Add netCDF PIO capability for restarts and run-time history for dev/ufs-weather-model #1303
Conversation
* remove wav_grdout routine, now moved into wav_history_mod
* remove more cap stuff from w3iorsmd. only ww3 native filenaming is possible w/ binary restarts * remove ifdef w3_ascii from w3wavemd, since the ascii commit is not yet present in mesh cap branch * clean up config variable logic for filenaming
* nrqrs can be non-zero if also using the restart from binary option
* flpart needs to be set for either historync or not
* move restart and history nc blocks outside of ww3 time testing block. * add log print messages for reading and writing restarts
* get logging working correctly for ufs * fix noclobber variable and file name in wav_history * clarify some comments
* make binary history files match when restartnc=true * rework wav_restart_mod, which originally was designed to be able to read and write restarts for testing purposes from inside wav_comp_nuopc.
* verboselog is true by default, but can be set false by config * the header text for ww3 logging as it steps through time is now turned off in w3init and placed into the mesh cap. this allows the mesh cap to order the logging correctly
* move block where addrstflds was set to before call to w3init since restarts are read in w3init * ensure that if nml file lacks a specification of extra fields, the default value of "unset" will not be returned as a field name
* only ice is added for now * tab cleanup in w3grid
* need to send explicit array bounds for ice array since it is 0:nsea * all baselines b4b against f9531d0
DeniseWorthen
changed the title
* intialize floutg and floutg2 which are can be unintialized when waves are in slow loop and historync is true
|
@MatthewMasarik-NOAA Can I ask if you've tested the mesh you provided me in debug mode? When using the dev/ufs-weather-model branch, I am getting a failure in debug mode at LN 1388 in w3init Lines 1384 to 1398 in 7f548c7 |
|
Hi @DeniseWorthen, I have not tested this mesh in either standard or debug mode. I was provided this HR4 mesh from @AliS-Noaa and @JessicaMeixner-NOAA who have done the unstructured grid creation, and could probably give that information. You had mentioned offline that at one point that Ali had told you a grid was passing in standalone, but not in coupled. I had not been briefed on that. Do you know if anyone followed up on that at the time? Or can you confirm if this new mesh is the same as the one you were discussing with Ali? |
|
@MatthewMasarik-NOAA The remark about failing in coupled mode was from an informal lunch-time discussion I had w/ Ali, some time ago, where he simply mentioned in passing what he was working on. It was hardly a "briefing" I had on the subject. I have only the mesh you provided me, as well as one which Jiande gave me which he is currently using for HR4. |
@DeniseWorthen, understood. It was more than has been shared with me so you know more than I do at this point. |
|
I think we might be getting meshes confused as well. HR4 was supposed to originally have an update and use the new mesh from @AliS-Noaa - however, due to time it's using the same as HR3. A new HR with updated wave grids will happen. @sbanihash I think @DeniseWorthen has pointed out an oversight from us that we should test this new mesh with debug mode on - would you be able to assign someone to run the new mesh in debug mode? |
|
I believe I had the same failure w/ the HR4 files that Jiande provided me. But I will confirm and post back either yes or no. |
|
My run directory on Gaea is /gpfs/f5/nggps_emc/scratch/Denise.Worthen/ww3pio/datm.gfsv17 grep F90 err | grep w3 In this case, I'm using a mod_def created with the 'gefs' settings in the grid.inp file. This is a test case I built w/ a DATM+MOM+CICE. See /gpfs/f5/nggps_emc/scratch/Denise.Worthen/ww3pio/datm.500k for a test case that runs w/ the same executable but w/ a different mesh and mod_def (the global 500k mesh I had used during mesh cap development). |
Pull Request Summary
Enable netCDF restarts and run-time netCDF history using PIO.
Description
PIO is configured via configuration settings in WAV attributes for UFS and via share code for CESM. Defaults are provided for the stride, number of io tasks and rearranger settings. The PIO subsystem is managed and initialized in
wav_pio_mod.F90PIO/netCDF restarts are enabled by using
use_restartnc=truein the configuration attributes, otherwise native WW3 binary restarts will be written. If netCDF restarts are enabled, an additional option exists to restart from an existing binary restart (restart_from_binary=true). In this case, the binary restart filename needs the same form as the netCDF restarts (minus the .nc at the end).With netCDF restarts enabled, restarts will be written at the
rstwrfrequency defined in the cap, which will allow both the flexible (ie, non-interval) restarts and end-of-run restarts to be enabled in the same way as for other coupled components in UFS. These two features are not implemented in this work.The netCDF restarts are read and written in a new routine
wav_restart_mod.F90. Only two fields (vaandmapsta) are written to the restart file by default. As noted in the comment in #1298, when waves are in the slow-coupling loop, the ice field is also written. The addition of extra restart fields is generalized via the already-existingrestart%extranamelist.Runtime netCDF history is enabled by using
use_historync=truein the configuration attributes, otherwise native WW3 binary history files will be written. The PIO/netCDF capability builds on the existing serial netCDF capability in the mesh cap. The two existing routines used for this (w3iogoncdmd.F90, wav_grdout.F90) have been reduced to a new single netCDF history modulewav_history_mod.F90containing the same functionality.History files can be custom named (if
user_histname = true), otherwise they will be named in the native WW3 file naming. History files will be written at thehistwrfrequency. This frequency will be set using the normalhistory_n,history_optionsettings, if present. Otherwise, thefield%stridefrequency will be used to determine thehistwrfrequency. (This requires that the stride be set in units of seconds, which is less flexible than enabling the history_n,history_option settings.)w3wavemd, w3iorsmd and w3iogomd..Modifications to the three listed routines were required for the mesh cap to allow for additional flexibility of restart and history writing. The changes in w3iogomd were resolved w/ the PR to dev/ufs-weather-model for the Langmuir turbulence calculations (#1034). With the new restartnc and historync options via PIO, the changes to the other two routines (wavemd and iorsmd) are no longer required, meaning these two routines are now much more aligned with the same code in the develop branch. Since the dev/ufs-weather-model branch has not been updated for nearly a year, the modifications are best seen by comparing these two routines in this PR branch against the develop branch at 4d8c315
Issue(s) addressed
Commit Message
Check list
Testing
Using UFS-WM, a baseline was created and verified to pass for all tests which include a WAV component. Full testing as well as scalability results will be documented in the associated UFS-WM PR.