Add osnap T S climtransects

[alicebarthel]

This is to plot T,S transects at OSNAP transects, based on the obs dataset [downloaded in 2022].
I based it on woce_transects.py
A couple of questions for @milenaveneziani @xylar:
I saw that WOCE is a collection of separate transects. Currently, the obs dataset has both East and West transects in one file, is it better to split them in pre-processing to give us separate plots?
I currently only added T,S (we have no measured density).
I did not add velocities in the variables but it is in the obs dataset. I suspected the process for plotting MPAS-O velocity at transect should be separate from tracer fields. Let me know if you think otherwise.

Checklist (for @xylar) before merging:

• copy OSNAP obs from /lcrc/group/e3sm/diagnostics/observations/Ocean/OSNAP to /lcrc/group/e3sm/public_html/diagnostics/observations/Ocean/OSNAP
• generate README from obs.xml and copy to /lcrc/group/e3sm/public_html/diagnostics/observations/Ocean/OSNAP
• sync OSNAP to Compy and Perlmutter

[milenaveneziani]

Currently, the obs dataset has both East and West transects in one file, is it better to split them in pre-processing to give us separate plots?

Yes, I would split them.

I suspected the process for plotting MPAS-O velocity at transect should be separate from tracer fields.

I'll let @xylar reply to this. My standalone script plots the mpas normalVelocity at the transect mask edges: so, no interpolation but unpleasant noisy visuals. I think Xylar has a more pleasant-to-the-eye alternative.

[xylar]

I suspected the process for plotting MPAS-O velocity at transect should be separate from tracer fields.

I'll let @xylar reply to this. My standalone script plots the mpas normalVelocity at the transect mask edges: so, no interpolation but unpleasant noisy visuals. I think Xylar has a more pleasant-to-the-eye alternative.

Hmm, I don't think we've plotted velocities at transects except for SOSE. There, we just plot zonal, meridional or magnitude. Instead, you would want velocity normal to the transect, which would require a computation based on the zonal and meridional components, and the angle of the transect edge with respect to east. I think that would be fun to do but it hasn't been done yet.

[milenaveneziani]

We actually don't plot velocities for the SOSE transects, only for the SOSE 2d maps.
But yes, I agree that plotting velocities is something that will take more effort and maybe should be done later.

[xylar]

We actually don't plot velocities for the SOSE transects, only for the SOSE 2d maps.

At least in the analysis I'm looking at, we do plot the velocities (zonal, meridional and magnitude):
https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.xylar/analysis_testing/anvil/fix_conda_mpi_on_hpc/main_py3.9/ocean/index.html#sose_transects

[alicebarthel]

An update on this:

1. I have been able to run the osnapTransects for all 3 combinations of grids (mpas, obs, and 5+uniform).
2. I am still finding some inconsistencies.

At the moment, runnning the analysis in a new directory fails with

srun: error: b577: tasks 1,10: Exited with exit code 1
Traceback (most recent call last):
  File "/gpfs/fs1/home/ac.abarthel/mpas-analysis/add_OSNAP_TSclimtransects/mpas_analysis/__main__.py", line 381, in add_task_and_subtasks
    analysisTask.setup_and_check()
  File "/gpfs/fs1/home/ac.abarthel/mpas-analysis/add_OSNAP_TSclimtransects/mpas_analysis/ocean/compute_transects_subtask.py", line 221, in setup_and_check
    super().setup_and_check()
  File "/gpfs/fs1/home/ac.abarthel/mpas-analysis/add_OSNAP_TSclimtransects/mpas_analysis/shared/climatology/remap_mpas_climatology_subtask.py", line 208, in setup_and_check
    self._setup_remappers()
  File "/gpfs/fs1/home/ac.abarthel/mpas-analysis/add_OSNAP_TSclimtransects/mpas_analysis/shared/climatology/remap_mpas_climatology_subtask.py", line 413, in _setup_remappers
    self.remappers[comparisonGridName] = get_remapper(
  File "/gpfs/fs1/home/ac.abarthel/mpas-analysis/add_OSNAP_TSclimtransects/mpas_analysis/shared/climatology/climatology.py", line 130, in get_remapper
    remapper.build_mapping_file(method=method, logger=logger,
  File "/lcrc/group/e3sm/ac.abarthel/mambaforge/envs/mpas_dev/lib/python3.10/site-packages/pyremap/remapper.py", line 303, in build_mapping_file
    subprocess.check_call(args, stdout=DEVNULL)
  File "/lcrc/group/e3sm/ac.abarthel/mambaforge/envs/mpas_dev/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['srun', '-n', '12', '/lcrc/group/e3sm/ac.abarthel/mambaforge/envs/mpas_dev/bin/ESMF_RegridWeightGen', '--source', '/lcrc/group/e3sm/ac.abarthel/analysis/debugl.A_WCYCL1850.ne4_oQU480.anvil/clim_3-5_ts_1-5/mapping/tmptykhhv_f/src_mesh.nc', '--destination', '/lcrc/group/e3sm/ac.abarthel/analysis/debugl.A_WCYCL1850.ne4_oQU480.anvil/clim_3-5_ts_1-5/mapping/tmptykhhv_f/dst_mesh.nc', '--weight', '/lcrc/group/e3sm/ac.abarthel/analysis/debugl.A_WCYCL1850.ne4_oQU480.anvil/clim_3-5_ts_1-5/mapping/map_oQU480_to_OSNAP_transects_5km_bilinear.nc', '--method', 'bilinear', '--netcdf4', '--no_log', '--src_loc', 'center', '--src_regional', '--dst_regional', '--ignore_unmapped']' returned non-zero exit status 1.
Warning: osnapTransects: remapTransects failed during check and will not be run
Warning: subtask of osnapTransects failed during check, so this task will not be run

but re-running it a second time (without purging) works and generates the plots. I am wondering if there is something in the order of operations that is off. After the first failure, the only logs available are:

ls /lcrc/group/e3sm/ac.abarthel/analysis/debugl.A_WCYCL1850.ne4_oQU480.anvil/clim_3-5_ts_1-5/logs
config.A_WCYCL1850.ne4_oQU480.anvil  taskProgress.log

[xylar]

@alicebarthel, you're getting close! Because you are using a development environment that doesn't support system MPI, you need these config options from the tutorial: https://mpas-dev.github.io/MPAS-Analysis/latest/tutorials/dev_getting_started.html#execute

That way, ESMF_RegridWeightGen won't be called with srun.

[alicebarthel]

Thanks @xylar ! It seems to work ok now that I added the [execute] section. Apologies for rushing through the set-up instead of reading the tutorial carefully. Just curious: Why does running it a second time solve the problem?

[alicebarthel]

A couple of questions/comments:

• The default grid for WOCE transects is: (horiz: 5, vert: uniform_0_to_4000m_at_10m). Do we want the same for the OSNAP transects or would something else make more sense?
• I plan to try to run the OSNAP analysis on a simulation with finer mesh.
• Do any of you want to try to run this analysis independently to do a check on it?

[xylar]

Just curious: Why does running it a second time solve the problem?

I have no idea. I wouldn't expect that to be the case.

[xylar]

The default grid for WOCE transects is: (horiz: 5, vert: uniform_0_to_4000m_at_10m). Do we want the same for the OSNAP transects or would something else make more sense?

Maybe too deep?

Do any of you want to try to run this analysis independently to do a check on it?

Yes, I'll run a test, too, if you think it's ready.

[xylar]

Apologies for rushing through the set-up instead of reading the tutorial carefully.

Absolutely no worries. It's confusing and I hope we can move to the approach in compass soon, where ESMF is built with system MPI.

[milenaveneziani]

The default grid for WOCE transects is: (horiz: 5, vert: uniform_0_to_4000m_at_10m). Do we want the same for the OSNAP transects or would something else make more sense?

I actually think this is OK, especially for OSNAP West. Here are two sections I plotted in the past, and they go to 3500 and 4000 m, respectively:

I can also give this a try.

[alicebarthel]

Ok, thanks @milenaveneziani! I may need to adjust the plot options like range. I did not see observed densities. Did you recalculate these or did you have them?

[alicebarthel]

FYI, 2 year clim for the v2.amoc.baseline (EC30to60)
https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.abarthel/E3SMv2/analysis/v2.amoc.baseline.ne30pg2_EC30to60E2r2/clim_100-101_ts_100-101/ocean/index.html#osnap

[xylar]

This is really great, @alicebarthel!

[xylar]

Things are looking great!

I ran the test suite and results are here:
https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.xasay-davis/analysis_testing/chrysalis/add_OSNAP_TSclimtransects/

In addition to a couple of very small code changes, the only thing left (other than the check-list at the top of this PR for me) are:

• add the task to the API docs here: https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.xasay-davis/analysis_testing/chrysalis/add_OSNAP_TSclimtransects/
• add a new file to the docs for this analysis (with an example image), similar to: https://github.com/MPAS-Dev/MPAS-Analysis/blob/develop/docs/users_guide/tasks/woceTransects.rst

[xylar]

I ran analysis on CRYO1950.ne30pg2_SOwISC12to60E2r4.N2Dependent.smep and it worked great:
https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.xylar/analysis/20221116.CRYO1950.ne30pg2_SOwISC12to60E2r4.N2Dependent.submeso.chrysalis/yrs121-130/ocean/index.html
It seems like we either have giant temperature biases in that run (very believable) or the bias plot has a pretty narrow range. What do you both think?

[milenaveneziani]

hmm, a bias of 2deg is pretty high in those regions.. We could extend the range to +-3 for the bias.

Do you understand why the range above 0 (in the vertical) is plotted?

@alicebarthel : no, I computed the sigma0's myself. They were not in the obs data set I used.

[milenaveneziani]

But I agree this is looking great. Thanks @alicebarthel!

[xylar]

Do you understand why the range above 0 (in the vertical) is plotted?

@milenaveneziani, I think that's because it plots up to the actual sea surface height, which goes at least slightly above zero. It doesn't bound the data range too tightly but instead leaves a little margin of error around it. We can manually set the vertical range in the config options if this bugs you.
https://github.com/alicebarthel/MPAS-Analysis/blob/add_OSNAP_TSclimtransects/mpas_analysis/default.cfg#L2619

[milenaveneziani]

yes, I would probably add vertical_bounds. Maybe 0 to -4000?
Thanks for reminding me about this option @xylar.

[milenaveneziani]

@alicebarthel: I think you can remove the WIP in the title and work in progress label for this one :)

[alicebarthel]

@cbegeman here is the draft PR I mentioned.

[alicebarthel]

@xylar I added the preprocessing notebook. Let me know if you have any issues with that format. I did modify the script (and added extra information such as the period, which maybe is confusing since the current version of the data does not have that naming convention). I can remove that extra bit, or replace the file and the file names in osnap_transect.py with the new version. I did check that the files generated were identical using filecmp.cmp(file1, file2, shallow=False).

[cbegeman]

@alicebarthel I reran this branch successfully. Thanks for getting this going again!

[alicebarthel]

@cbegeman thanks for testing this and showing interest, it provided the incentive needed!

[milenaveneziani]

I can test this, but not this week. Feel free to remove me as reviewer if you would like to get this merged asap.

Thanks @alicebarthel for working on it!

[xylar]

@alicebarthel, I copied the observations files into:

/lcrc/group/e3sm/public_html/diagnostics/observations/Ocean/OSNAP

I generated a README.md using https://github.com/alicebarthel/MPAS-Analysis/blob/add_OSNAP_TSclimtransects/mpas_analysis/obs/make_obs_readme_bibtex.py and copied it into that directory as well.

I'm running my test suite on Chrysalis right now to make sure things go smoothly. Everything looks good so far but I'll let you know how the tests go.

[xylar]

@alicebarthel, thanks so much for working hard on this and getting back to it to finish things off!

Everything looks great in my testing:

This is a super low res (QU240) simulation so the coarse plot is expected.

The full test results are here:
https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.xasay-davis/analysis_testing/chrysalis/add_OSNAP_TSclimtransects/

In building the docs:
https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.xasay-davis/analysis_testing/chrysalis/add_OSNAP_TSclimtransects/docs/
it looks like the new osnapTransects task is missing here:
https://github.com/alicebarthel/MPAS-Analysis/blob/add_OSNAP_TSclimtransects/docs/users_guide/analysis_tasks.rst?plain=1#L37

[xylar]

Looks great! Thanks @alicebarthel!

[alicebarthel]

Thanks @xylar for looking this over!

[milenaveneziani]

It does look great, @alicebarthel, thanks!

I can try testing this within Impacts, at the same time getting some possible nice pictures for your AMOC presentation, Alice. But for now please merge!