Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support coupled GEFS forecast, use mem000 for GEFS control #1755

Merged

Conversation

WalterKolczynski-NOAA
Copy link
Contributor

@WalterKolczynski-NOAA WalterKolczynski-NOAA commented Jul 18, 2023

Description

Makes the necessary updates to allow the workflow to run (non-wave) coupled forecasts. Some of the coupled functions in postdet had to be updated because they only checked for gfs and gdas runs, so necessary code was never executed.

Also updates so that the GEFS control uses mem000 as the memdir so it appears at the same level as the perturbed members. Relatedly, the default member number in coupled files is changed to 000 instead of 01.

Finally, an unrelated error in the earc job is corrected where a character from the middle of the line was moved to the beginning.

Resolves #1754

Type of change

  • GEFS coupled forecast-only on Orion
  • P8 coupled on Orion
  • Cycled on Orion

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • New and existing tests pass with my changes

Makes the necessary updates to allow the workflow to run (non-wave) coupled
forecasts. Some of the coupled functions in postdet had to be updated because
they only checked for gfs and gdas runs, so necessary code was never executed.

Also updates so that the GEFS control uses `mem000` as the memdir so it
appears at the same level as the perturbed members. Relatedly, the default
member number in coupled files is changed to 000 instead of 01.

Finally, an unrelated error in the earc job is corrected where a character
from the middle of the line was moved to the beginning.

Resolves NOAA-EMC#1754
@WalterKolczynski-NOAA WalterKolczynski-NOAA self-assigned this Jul 18, 2023
Copy link

@github-advanced-security github-advanced-security bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shellcheck found more than 10 potential problems in the proposed changes. Check the Files changed tab for more details.

@WalterKolczynski-NOAA
Copy link
Contributor Author

Low-res test pass. C384 fails for a reason that seems unrelated (I tried develop and it failed too).

@WalterKolczynski-NOAA WalterKolczynski-NOAA marked this pull request as ready for review July 18, 2023 14:00
@WalterKolczynski-NOAA WalterKolczynski-NOAA marked this pull request as draft July 18, 2023 14:02
Copy link
Contributor

@aerorahul aerorahul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

first pass.

parm/config/gefs/config.efcs Outdated Show resolved Hide resolved
ush/forecast_postdet.sh Outdated Show resolved Hide resolved
Copy link
Contributor

@aerorahul aerorahul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks better.
A couple of questions:

  1. see the review
  2. can we just link the config.XYZ files in parm/config/gefs from parm/config/gfs that are not different? I don't imagine things like config.ocn, config.ice, config.ufs, config.wave to be different. These are setting the various properties of the model for a choice of resolution and wavgrd set in config.base.

parm/config/gefs/config.base.emc.dyn Show resolved Hide resolved
@@ -0,0 +1,202 @@
#! /usr/bin/env bash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just link to config.wave from parm/config/gfs? Is there any distinction for wave between gfs and gefs for waves?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we will need to use different settings for GEFS. Can certain do that (and for aero/ice/ocn) until such time as we find we need to have different settings. Will leave it up to you.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to link in gefs to force them to be the same as gfs and really understand the requirements when they are presented why they need to be different. Otherwise there will be a tendency to just cram gefs specifics into this application.
I am not against the flexibility, but I would like to understand the reason and justification when that is provided.
This does not prevent us from maintaining a separate configuration for GEFS in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

scripts/exgdas_enkf_earc.sh Show resolved Hide resolved
Since the files are identical replace the component config files for
GEFS with links to the GFS versions.

Refs NOAA-EMC#1754
@@ -0,0 +1,29 @@
#! /usr/bin/env bash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can also be linked from parm/config/gfs

@aerorahul aerorahul self-requested a review July 20, 2023 17:40
Copy link
Contributor

@aerorahul aerorahul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.
There is one comment on linking config.defaults.s2sw from gfs into gefs.
I'll leave it to you if you want to address it here or later or not.

@WalterKolczynski-NOAA WalterKolczynski-NOAA marked this pull request as ready for review July 21, 2023 07:30
@WalterKolczynski-NOAA
Copy link
Contributor Author

Looks good. There is one comment on linking config.defaults.s2sw from gfs into gefs. I'll leave it to you if you want to address it here or later or not.

Took care of this one.

@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion labels Jul 21, 2023
@emcbot emcbot added CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera and removed CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Jul 21, 2023
@emcbot emcbot added the CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress label Jul 21, 2023
@emcbot
Copy link

emcbot commented Jul 21, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Fri Jul 21 07:32:56 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Fri Jul 21 07:38:14 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Fri Jul 21 08:06:55 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 08:07:00 UTC 2023 for experiment C48_S2S_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 08:07:04 UTC 2023 for experiment C96C48_hybatmDA_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 08:07:07 UTC 2023 for experiment C96_atm3DVar_9ba95014

@emcbot emcbot added CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress and removed CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion labels Jul 21, 2023
@emcbot
Copy link

emcbot commented Jul 21, 2023

Automated global-workflow Testing Results:

Machine: Orion
Start: Fri Jul 21 02:32:19 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Fri Jul 21 02:34:35 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Fri Jul 21 03:07:09 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 03:07:16 CDT 2023 for experiment C48_S2S_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 03:07:20 CDT 2023 for experiment C96_atm3DVar_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 03:07:24 CDT 2023 for experiment C96C48_hybatmDA_9ba95014

@emcbot
Copy link

emcbot commented Jul 21, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Fri Jul 21 07:32:56 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Fri Jul 21 07:38:14 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Fri Jul 21 08:06:55 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 08:07:00 UTC 2023 for experiment C48_S2S_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 08:07:04 UTC 2023 for experiment C96C48_hybatmDA_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 08:07:07 UTC 2023 for experiment C96_atm3DVar_9ba95014
Experiment C48_S2S_9ba95014 completed: *SUCCESS*
Experiment C48_S2S_9ba95014 Completed at Fri Jul 21 08:51:15 UTC 2023
with 18 successfully completed jobs

@emcbot
Copy link

emcbot commented Jul 21, 2023

Automated global-workflow Testing Results:

Machine: Orion
Start: Fri Jul 21 02:32:19 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Fri Jul 21 02:34:35 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Fri Jul 21 03:07:09 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 03:07:16 CDT 2023 for experiment C48_S2S_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 03:07:20 CDT 2023 for experiment C96_atm3DVar_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 03:07:24 CDT 2023 for experiment C96C48_hybatmDA_9ba95014
Experiment C48_S2S_9ba95014 completed: *SUCCESS*
Experiment C48_S2S_9ba95014 Completed at Fri Jul 21 03:52:09 CDT 2023
with 18 successfully completed jobs

@emcbot
Copy link

emcbot commented Jul 21, 2023

Automated global-workflow Testing Results:

Machine: Orion
Start: Fri Jul 21 02:32:19 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Fri Jul 21 02:34:35 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Fri Jul 21 03:07:09 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 03:07:16 CDT 2023 for experiment C48_S2S_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 03:07:20 CDT 2023 for experiment C96_atm3DVar_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 03:07:24 CDT 2023 for experiment C96C48_hybatmDA_9ba95014
Experiment C48_S2S_9ba95014 completed: *SUCCESS*
Experiment C48_S2S_9ba95014 Completed at Fri Jul 21 03:52:09 CDT 2023
with 18 successfully completed jobs
Experiment C96_atm3DVar_9ba95014 completed: *SUCCESS*
Experiment C96_atm3DVar_9ba95014 Completed at Fri Jul 21 05:26:13 CDT 2023
with 89 successfully completed jobs

@emcbot
Copy link

emcbot commented Jul 21, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Fri Jul 21 07:32:56 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Fri Jul 21 07:38:14 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Fri Jul 21 08:06:55 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 08:07:00 UTC 2023 for experiment C48_S2S_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 08:07:04 UTC 2023 for experiment C96C48_hybatmDA_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 08:07:07 UTC 2023 for experiment C96_atm3DVar_9ba95014
Experiment C48_S2S_9ba95014 completed: *SUCCESS*
Experiment C48_S2S_9ba95014 Completed at Fri Jul 21 08:51:15 UTC 2023
with 18 successfully completed jobs
Experiment C96C48_hybatmDA_9ba95014 completed: *SUCCESS*
Experiment C96C48_hybatmDA_9ba95014 Completed at Fri Jul 21 10:30:15 UTC 2023
with 151 successfully completed jobs

@emcbot
Copy link

emcbot commented Jul 21, 2023

Automated global-workflow Testing Results:

Machine: Orion
Start: Fri Jul 21 02:32:19 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Fri Jul 21 02:34:35 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Fri Jul 21 03:07:09 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 03:07:16 CDT 2023 for experiment C48_S2S_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 03:07:20 CDT 2023 for experiment C96_atm3DVar_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 03:07:24 CDT 2023 for experiment C96C48_hybatmDA_9ba95014
Experiment C48_S2S_9ba95014 completed: *SUCCESS*
Experiment C48_S2S_9ba95014 Completed at Fri Jul 21 03:52:09 CDT 2023
with 18 successfully completed jobs
Experiment C96_atm3DVar_9ba95014 completed: *SUCCESS*
Experiment C96_atm3DVar_9ba95014 Completed at Fri Jul 21 05:26:13 CDT 2023
with 89 successfully completed jobs
Experiment C96C48_hybatmDA_9ba95014 completed: *SUCCESS*
Experiment C96C48_hybatmDA_9ba95014 Completed at Fri Jul 21 06:12:10 CDT 2023
with 151 successfully completed jobs

@emcbot emcbot added CI-Orion-Passed **Bot use only** CI testing on Orion for this PR has completed successfully and removed CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress labels Jul 21, 2023
@emcbot
Copy link

emcbot commented Jul 21, 2023

Automated global-workflow Testing Results:

Machine: Orion
Start: Fri Jul 21 02:32:19 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Fri Jul 21 02:34:35 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Fri Jul 21 03:07:09 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 03:07:16 CDT 2023 for experiment C48_S2S_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 03:07:20 CDT 2023 for experiment C96_atm3DVar_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 03:07:24 CDT 2023 for experiment C96C48_hybatmDA_9ba95014
Experiment C48_S2S_9ba95014 completed: *SUCCESS*
Experiment C48_S2S_9ba95014 Completed at Fri Jul 21 03:52:09 CDT 2023
with 18 successfully completed jobs
Experiment C96_atm3DVar_9ba95014 completed: *SUCCESS*
Experiment C96_atm3DVar_9ba95014 Completed at Fri Jul 21 05:26:13 CDT 2023
with 89 successfully completed jobs
Experiment C96C48_hybatmDA_9ba95014 completed: *SUCCESS*
Experiment C96C48_hybatmDA_9ba95014 Completed at Fri Jul 21 06:12:10 CDT 2023
with 151 successfully completed jobs

@emcbot
Copy link

emcbot commented Jul 21, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Fri Jul 21 07:32:56 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Fri Jul 21 07:38:14 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Fri Jul 21 08:06:55 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 08:07:00 UTC 2023 for experiment C48_S2S_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 08:07:04 UTC 2023 for experiment C96C48_hybatmDA_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 08:07:07 UTC 2023 for experiment C96_atm3DVar_9ba95014
Experiment C48_S2S_9ba95014 completed: *SUCCESS*
Experiment C48_S2S_9ba95014 Completed at Fri Jul 21 08:51:15 UTC 2023
with 18 successfully completed jobs
Experiment C96C48_hybatmDA_9ba95014 completed: *SUCCESS*
Experiment C96C48_hybatmDA_9ba95014 Completed at Fri Jul 21 10:30:15 UTC 2023
with 151 successfully completed jobs
Experiment C96_atm3DVar_9ba95014 completed: *SUCCESS*
Experiment C96_atm3DVar_9ba95014 Completed at Fri Jul 21 11:18:16 UTC 2023
with 89 successfully completed jobs

1 similar comment
@emcbot
Copy link

emcbot commented Jul 21, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Fri Jul 21 07:32:56 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Fri Jul 21 07:38:14 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Fri Jul 21 08:06:55 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 08:07:00 UTC 2023 for experiment C48_S2S_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 08:07:04 UTC 2023 for experiment C96C48_hybatmDA_9ba95014
Created experiment:            *SUCCESS*
Case setup: Completed at Fri Jul 21 08:07:07 UTC 2023 for experiment C96_atm3DVar_9ba95014
Experiment C48_S2S_9ba95014 completed: *SUCCESS*
Experiment C48_S2S_9ba95014 Completed at Fri Jul 21 08:51:15 UTC 2023
with 18 successfully completed jobs
Experiment C96C48_hybatmDA_9ba95014 completed: *SUCCESS*
Experiment C96C48_hybatmDA_9ba95014 Completed at Fri Jul 21 10:30:15 UTC 2023
with 151 successfully completed jobs
Experiment C96_atm3DVar_9ba95014 completed: *SUCCESS*
Experiment C96_atm3DVar_9ba95014 Completed at Fri Jul 21 11:18:16 UTC 2023
with 89 successfully completed jobs

@emcbot emcbot added CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Jul 21, 2023
@WalterKolczynski-NOAA WalterKolczynski-NOAA merged commit 4f31327 into NOAA-EMC:develop Jul 21, 2023
4 checks passed
@WalterKolczynski-NOAA WalterKolczynski-NOAA deleted the feature/gefs_cpl branch July 21, 2023 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully CI-Orion-Passed **Bot use only** CI testing on Orion for this PR has completed successfully
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add coupled forecast support for GEFS
3 participants