Add index to cpl_scalars to allow CSG or regional (single tile) mediator history files for ATM component in UFS #436

DeniseWorthen · 2024-03-05T15:59:52Z

Description of changes

Adds an index to cpl_scalars to allow 6 tiles of a CSG or a single tile of a regional ATM grid to be written as the required number of 2-d fields in the mediator history files. Replaces current implementation of this feature, which is limited to CSG and component level history files (ie, not history written by the history write phase).

Specific notes

The additional index of cpl_scalars is specified with the variable ScalarFieldIdxGridNTile.
Includes final changes that were required for CDEPS_INLINE functionality in UFS
Includes minor fixup of typos, removal of unnecessary string trims on coupling_mode .
Includes fix for UFS lnd->atm masking

Contributors other than yourself, if any:

CMEPS Issues Fixed (include github issue #): NOAA-EMC#111

UFS Issue ufs-community/ufs-weather-model#2171

Are changes expected to change answers? (specify if bfb, different at roundoff, more substantial)

No

Any User Interface Changes (namelist or namelist defaults changes)?

Only for UFS

Testing performed

Please describe the tests along with the target model and machine(s)
If possible, please also added hashes that were used in the testing

… air-sea flux computation (NOAA-EMC#107) * Update ufs/ccpp/data/MED_typedefs.meta.

Sync w/ ESCOMP, adding CDEPS inline capability to CMEPS

* add new cpl_scalar for mediator history files for tiled gridded domains, eg cube-sphere. Replaces existing use of config variables which restricted the use to 6-tiles domain * remove unnecessary trims, fix minor typos and indentation

…scalars

DeniseWorthen · 2024-03-05T16:01:10Z

@uturuncoglu I have this working in UFS when testing w/ a feature branch of FV3 to allow the export of the cpl_scalars from FV3. Would you please test in CESM. I believe it should work but I would prefer this to get tested and merged at ESCOMP first.

I also could add the required changes to the NOAHMP cap also.

uturuncoglu · 2024-03-06T16:38:43Z

@DeniseWorthen Sorry for late response. Last two days were full of meetings. Let me check this with CESM. I think I'll follow same approach and create baseline first and then check against it with your changes. @jedwards4b Please let me know if we have a baseline already.

DeniseWorthen · 2024-03-06T20:26:22Z

@uturuncoglu Thanks. Let me know if you run into any issues. Right now this is draft, but if you don't run into any issues, I'd like to commit it here first.

Note the PR also contains the final changes we made to get the CDEPS inline work committed for HAFS implementation.

uturuncoglu · 2024-03-06T20:27:57Z

@DeniseWorthen Okay. I was going to ask those changes. I'll update you once I finalize the tests.

uturuncoglu · 2024-03-07T16:27:11Z

@jedwards4b @fischer-ncar I created my own baseline using cesm2_3_alpha17b and compare agains it using the version of CMEPS in this PR and the tests are failing with following error. It seems that the tests could not find specific files. The baseline seems passing without any issue. So, maybe there is incotibility issue among component such as cime etc. but I am not sure why, I supposed that if you tag the model the components needs to be consistent. Right?

    FAIL ERR_Ld5.f09_t061.B1850MOM.derecho_intel.allactive-defaultio MEMCOMP [Errno 2] No such file or directory: '/glade/derecho/scratch/turuncu/baselines/20240306_cesm2_3_alpha17b/ERR_Ld5.f09_t061.B1850MOM.derecho_intel.allactive-defaultio/cpl-mem.log'
    FAIL ERR_Ld5.f09_t061.B1850MOM.derecho_intel.allactive-defaultio TPUTCOMP [Errno 2] No such file or directory: '/glade/derecho/scratch/turuncu/baselines/20240306_cesm2_3_alpha17b/ERR_Ld5.f09_t061.B1850MOM.derecho_intel.allactive-defaultio/cpl-tput.log'
    FAIL ERS_Ld5.f09_g17.I1850Clm50Sp.derecho_intel.clm-default MEMCOMP [Errno 2] No such file or directory: '/glade/derecho/scratch/turuncu/baselines/20240306_cesm2_3_alpha17b/ERS_Ld5.f09_g17.I1850Clm50Sp.derecho_intel.clm-default/cpl-mem.log'
    FAIL ERS_Ld5.f19_g17.I2000Clm51Bgc.derecho_intel.clm-default MEMCOMP [Errno 2] No such file or directory: '/glade/derecho/scratch/turuncu/baselines/20240306_cesm2_3_alpha17b/ERS_Ld5.f19_g17.I2000Clm51Bgc.derecho_intel.clm-default/cpl-mem.log'
    FAIL ERS_Ld5.ne30pg3_t232.BLT1850_v0c.derecho_intel.allactive-defaultio MEMCOMP [Errno 2] No such file or directory: '/glade/derecho/scratch/turuncu/baselines/20240306_cesm2_3_alpha17b/ERS_Ld5.ne30pg3_t232.BLT1850_v0c.derecho_intel.allactive-defaultio/cpl-mem.log'
    FAIL ERS_Ld5.T62_g37.DTEST.derecho_intel MEMCOMP [Errno 2] No such file or directory: '/glade/derecho/scratch/turuncu/baselines/20240306_cesm2_3_alpha17b/ERS_Ld5.T62_g37.DTEST.derecho_intel/cpl-mem.log'
    FAIL ERS_Ld5.T62_t061.CMOM.derecho_intel MEMCOMP [Errno 2] No such file or directory: '/glade/derecho/scratch/turuncu/baselines/20240306_cesm2_3_alpha17b/ERS_Ld5.T62_t061.CMOM.derecho_intel/cpl-mem.log'
    FAIL ERS_Ld5.T62_t061.GMOM.derecho_intel MEMCOMP [Errno 2] No such file or directory: '/glade/derecho/scratch/turuncu/baselines/20240306_cesm2_3_alpha17b/ERS_Ld5.T62_t061.GMOM.derecho_intel/cpl-mem.log'
    FAIL ERS_Ln9_C3.f19_g17_rx1.A.derecho_intel MEMCOMP [Errno 2] No such file or directory: '/glade/derecho/scratch/turuncu/baselines/20240306_cesm2_3_alpha17b/ERS_Ln9_C3.f19_g17_rx1.A.derecho_intel/cpl-mem.log'
    FAIL ERS_Ln9.f19_g17.X.derecho_intel MEMCOMP [Errno 2] No such file or directory: '/glade/derecho/scratch/turuncu/baselines/20240306_cesm2_3_alpha17b/ERS_Ln9.f19_g17.X.derecho_intel/cpl-mem.log'
    FAIL SMS_D_Ly1.f09_g17_gl4.T1850Gg.derecho_intel MEMCOMP [Errno 2] No such file or directory: '/glade/derecho/scratch/turuncu/baselines/20240306_cesm2_3_alpha17b/SMS_D_Ly1.f09_g17_gl4.T1850Gg.derecho_intel/cpl-mem.log'
  SMS_Ld2.ww3a.ADWAV.derecho_intel (Overall: FAIL) details:
    FAIL SMS_Ld2.ww3a.ADWAV.derecho_intel RUN time=16

FYI, the commands that I used to create baseline and check agains it,

./create_test --xml-testlist ../../components/cmeps/cime_config/testdefs/testlist_drv.xml --xml-machine derecho --xml-category nuopc --xml-compiler intel --generate 20240306_cesm2_3_alpha17b --baseline-root /glade/derecho/scratch/turuncu/baselines --xml-category aux_cmeps

./create_test --xml-testlist ../../components/cmeps/cime_config/testdefs/testlist_drv.xml --xml-machine derecho --xml-category nuopc --xml-compiler intel --compare 20240306_cesm2_3_alpha17b  --baseline-root /glade/derecho/scratch/turuncu/baselines --xml-category aux_cmeps

fischer-ncar · 2024-03-07T16:38:57Z

@uturuncoglu I wouldn't worry about these failures. There's issues with the generation of the cpl-mem.log and cpl-tput.log.

uturuncoglu · 2024-03-07T16:46:19Z

@fischer-ncar Thanks for the confirmation and help. I think this PR is fine to go to main branch. Please let me know if you need more test. Otherwise, I think @DeniseWorthen could go ahead and make this PR ready for review.

As a side note, I think these kinds of issues are annoying for the unexperienced users like me and make hard to test the CESM. Since, this is shared component with UFS, I think we need to have more smooth testing and maybe guideline for the users that want to contribute to CMEPS and CDEPS. So, they could test easily until we have automated testing capability.

DeniseWorthen · 2024-03-07T18:32:45Z

@uturuncoglu Thanks for your testing. I did want to apologize for combining multiple things in this PR. I'll refrain in the future.

uturuncoglu · 2024-03-07T18:49:10Z

@DeniseWorthen No worries. I think it is better to combine them since testing in CESM side takes time.

DeniseWorthen · 2024-03-09T16:50:11Z

There is a small fix required to write mediator fractions on ntiles for the case where the med_phases_history_write phase is used. I can add this through a later PR if preferred, but I wanted to make note of it. The change is

diff --git a/mediator/med_phases_history_mod.F90 b/mediator/med_phases_history_mod.F90
index 606b6159..52b20c03 100644
--- a/mediator/med_phases_history_mod.F90
+++ b/mediator/med_phases_history_mod.F90
@@ -336,12 +336,14 @@ contains
                 ! Write mediator fraction field bundles
                 if (ESMF_FieldBundleIsCreated(is_local%wrap%FBFrac(n),rc=rc)) then
                    call med_io_write(io_file, is_local%wrap%FBFrac(n), whead(m), wdata(m), &
-                        is_local%wrap%nx(n), is_local%wrap%ny(n), nt=1, pre='Med_frac_'//trim(compname(n)), rc=rc)
+                        is_local%wrap%nx(n), is_local%wrap%ny(n), nt=1, pre='Med_frac_'//trim(compname(n)), &
+                        ntile=is_local%wrap%ntile(n), rc=rc)
                    if (ChkErr(rc,__LINE__,u_FILE_u)) return
                 end if
                 ! Write component mediator area field bundles
                 call med_io_write(io_file, is_local%wrap%FBArea(n), whead(m), wdata(m), &
-                     is_local%wrap%nx(n), is_local%wrap%ny(n), nt=1, pre='MED_'//trim(compname(n)), rc=rc)
+                     is_local%wrap%nx(n), is_local%wrap%ny(n), nt=1, pre='MED_'//trim(compname(n)), &
+                     ntile=is_local%wrap%ntile(n), rc=rc)
              end do

              ! Write atm/ocn fluxes and ocean albedoes if field bundles are created

* testing of this feature w/ UFS noahmp lnd component, which currently runs on the CSG, found two issues. One to write the mediator fractions and areas on the tiles when using the single history file. A second fix is the mapping masking for lnd-atm coupling in UFS.

uturuncoglu

@DeniseWorthen This looks good to me. Let me try to run CESM tests one more time if you don't mind to be sure that we are not changing anything in CESM side.

mediator/med_phases_restart_mod.F90

uturuncoglu · 2024-03-13T16:21:59Z

@DeniseWorthen I have just start test. Will update you soon.

uturuncoglu · 2024-03-13T19:31:44Z

@DeniseWorthen I run CMEPS aux tests with cesm2_3_alpha17b and compare agains my previously generated baseline and it look fine. @fischer-ncar @jedwards4b please let me know if you want me to do more test. Other than that I think this is ready to merge.

uturuncoglu · 2024-03-13T21:44:28Z

@DeniseWorthen @jedwards4b @fischer-ncar Please wait before merging this. We might need to modify mask for lnd->atm for ufs. I'll update you soon. I think that the issue that we see around coastline is created by the masking. I'll try to set it differently and try again.

uturuncoglu · 2024-03-13T21:54:36Z

@DeniseWorthen If you don't mind could you modify

       if (n1 == complnd .and. n2 == compatm) then
          srcMaskValue = ispval_mask
          dstMaskValue = 0
       end if

as

       if (n1 == complnd .and. n2 == compatm) then
          srcMaskValue = ispval_mask
          dstMaskValue = ispval_mask
       end if

This will solve the coastline issue that we see in the results. I think, the 0 in destination mask is creating issue since there are fractional grid in UFS side so some grid points are fractional. If you want you could go without mask changes at all in this PR that is also fine and I could bring this later. Anyway, let me know what you think.

DeniseWorthen · 2024-03-13T22:28:13Z

@uturuncoglu I will make the change since I've got this CMEPS PR already open.

uturuncoglu · 2024-03-13T22:30:50Z

@DeniseWorthen Thanks. I think we could merge this but lets wait confirmation from @fischer-ncar and @jedwards4b.

DeniseWorthen · 2024-03-19T20:18:48Z

@uturuncoglu Are there any issues preventing this getting merged?

uturuncoglu · 2024-03-19T20:29:57Z

@DeniseWorthen As I know this Pr is fine to merge. @jedwards4b Can we merge this since this is preventing UFS level PR.

jedwards4b

LGTM

DeniseWorthen and others added 9 commits January 31, 2024 09:20

fix dummy arguments w/o values

4127636

Update ccpp-physics in CMEPS to consider sea surface ocean current in…

8cd4a32

… air-sea flux computation (NOAA-EMC#107) * Update ufs/ccpp/data/MED_typedefs.meta.

Update CMEPS (NOAA-EMC#110)

624920d

Sync w/ ESCOMP, adding CDEPS inline capability to CMEPS

Merge branch 'NOAA-EMC:emc/develop' into feature/nowarn

fa78b46

add cpl_scalar for tiled grids, other minor fixes

14878f4

* add new cpl_scalar for mediator history files for tiled gridded domains, eg cube-sphere. Replaces existing use of config variables which restricted the use to 6-tiles domain * remove unnecessary trims, fix minor typos and indentation

Merge remote-tracking branch 'ESCOMP/main' into feature/cplscalars

91cc86e

Merge remote-tracking branch 'origin/feature/nowarn' into feature/cpl…

f8fb572

…scalars

set ntile=0 when ntile scalar doesn't exist

3515360

fix restarts when ntile>0

ac5520f

This was referenced Mar 6, 2024

Enable cpl_scalars NOAA-EMC/fv3atm#794

Merged

Enable cpl_scalars export from ATM and NoahMP for use by CMEPS ufs-community/ufs-weather-model#2175

Merged

DeniseWorthen marked this pull request as ready for review March 7, 2024 18:31

DeniseWorthen mentioned this pull request Mar 11, 2024

add export of cpl_scalars NOAA-EMC/noahmp#8

Merged

add two fixes

45d63c4

* testing of this feature w/ UFS noahmp lnd component, which currently runs on the CSG, found two issues. One to write the mediator fractions and areas on the tiles when using the single history file. A second fix is the mapping masking for lnd-atm coupling in UFS.

DeniseWorthen requested review from uturuncoglu, jedwards4b and mvertens and removed request for uturuncoglu March 13, 2024 16:11

uturuncoglu approved these changes Mar 13, 2024

View reviewed changes

mediator/med_phases_restart_mod.F90 Show resolved Hide resolved

modify dstmask for lnd->atm in UFS

8309884

jedwards4b approved these changes Mar 19, 2024

View reviewed changes

jedwards4b merged commit 1825c2b into ESCOMP:main Mar 19, 2024
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add index to cpl_scalars to allow CSG or regional (single tile) mediator history files for ATM component in UFS #436

Add index to cpl_scalars to allow CSG or regional (single tile) mediator history files for ATM component in UFS #436

DeniseWorthen commented Mar 5, 2024 •

edited

Loading

DeniseWorthen commented Mar 5, 2024 •

edited

Loading

uturuncoglu commented Mar 6, 2024

DeniseWorthen commented Mar 6, 2024

uturuncoglu commented Mar 6, 2024

uturuncoglu commented Mar 7, 2024

fischer-ncar commented Mar 7, 2024

uturuncoglu commented Mar 7, 2024

DeniseWorthen commented Mar 7, 2024

uturuncoglu commented Mar 7, 2024

DeniseWorthen commented Mar 9, 2024 •

edited

Loading

uturuncoglu left a comment

uturuncoglu commented Mar 13, 2024

uturuncoglu commented Mar 13, 2024

uturuncoglu commented Mar 13, 2024 •

edited

Loading

uturuncoglu commented Mar 13, 2024

DeniseWorthen commented Mar 13, 2024

uturuncoglu commented Mar 13, 2024

DeniseWorthen commented Mar 19, 2024

uturuncoglu commented Mar 19, 2024

jedwards4b left a comment

Add index to cpl_scalars to allow CSG or regional (single tile) mediator history files for ATM component in UFS #436

Add index to cpl_scalars to allow CSG or regional (single tile) mediator history files for ATM component in UFS #436

Conversation

DeniseWorthen commented Mar 5, 2024 • edited Loading

Description of changes

Specific notes

Testing performed

DeniseWorthen commented Mar 5, 2024 • edited Loading

uturuncoglu commented Mar 6, 2024

DeniseWorthen commented Mar 6, 2024

uturuncoglu commented Mar 6, 2024

uturuncoglu commented Mar 7, 2024

fischer-ncar commented Mar 7, 2024

uturuncoglu commented Mar 7, 2024

DeniseWorthen commented Mar 7, 2024

uturuncoglu commented Mar 7, 2024

DeniseWorthen commented Mar 9, 2024 • edited Loading

uturuncoglu left a comment

Choose a reason for hiding this comment

uturuncoglu commented Mar 13, 2024

uturuncoglu commented Mar 13, 2024

uturuncoglu commented Mar 13, 2024 • edited Loading

uturuncoglu commented Mar 13, 2024

DeniseWorthen commented Mar 13, 2024

uturuncoglu commented Mar 13, 2024

DeniseWorthen commented Mar 19, 2024

uturuncoglu commented Mar 19, 2024

jedwards4b left a comment

Choose a reason for hiding this comment

DeniseWorthen commented Mar 5, 2024 •

edited

Loading

DeniseWorthen commented Mar 5, 2024 •

edited

Loading

DeniseWorthen commented Mar 9, 2024 •

edited

Loading

uturuncoglu commented Mar 13, 2024 •

edited

Loading