Fix `SDATE_GFS` issue in cycled and forecast-only modes #1753

aerorahul · 2023-07-17T22:10:28Z

Description

On further examination, SDATE_GFS in the context of metp task is the first date that METP was run and started generating statistics. This is different when doing cycled (gfs_cyc determines SDATE_GFS). In the case of forecast-only, SDATE_GFS is always the date the first free-forecast was launched (SDATE)

While the fix by @RussTreadon-NOAA worked for cycling, it broke for forecast-only setups and needed to account for the other mode of operation.

Fixes #1748

Type of change

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Created XML for cycled and forecast-only experiments on Hera.

Checklist

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
My changes need updates to the documentation. I have made corresponding changes to the documentation
My changes generate no new warnings
New and existing tests pass with my changes
Any dependent changes have been merged and published

…and started generating statistics. This is different when doing cycled (gfs_cyc determines SDATE_GFS). In the case of forecast-only, SDATE_GFS is always the first free-forecast

emcbot · 2023-07-17T22:48:22Z

Automated global-workflow Testing Results:

Machine: Orion
Start: Mon Jul 17 17:15:49 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Mon Jul 17 17:17:40 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Mon Jul 17 17:48:08 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 17:48:11 CDT 2023 for experiment C48_S2S_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 17:48:16 CDT 2023 for experiment C96_atm3DVar_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 17:48:20 CDT 2023 for experiment C96C48_hybatmDA_a62f37ab

emcbot · 2023-07-17T22:49:10Z

Automated global-workflow Testing Results:

Machine: Hera
Start: Mon Jul 17 22:16:26 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Mon Jul 17 22:21:25 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Mon Jul 17 22:49:00 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 22:49:02 UTC 2023 for experiment C48_S2S_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 22:49:06 UTC 2023 for experiment C96C48_hybatmDA_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 22:49:08 UTC 2023 for experiment C96_atm3DVar_a62f37ab

emcbot · 2023-07-17T23:00:32Z

Automated global-workflow Testing Results:

Machine: Orion
Start: Mon Jul 17 17:15:49 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Mon Jul 17 17:17:40 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Mon Jul 17 17:48:08 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 17:48:11 CDT 2023 for experiment C48_S2S_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 17:48:16 CDT 2023 for experiment C96_atm3DVar_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 17:48:20 CDT 2023 for experiment C96C48_hybatmDA_a62f37ab
Experiment C96_atm3DVar_a62f37ab Terminated: *FAILED*
Experiment C96_atm3DVar_a62f37ab Terminated with 1 tasks failed at Mon Jul 17 18:00:27 CDT 2023
Error logs:
/work2/noaa/stmp/GFS_CI_ROOT/PR/1753/RUNTESTS/COMROT/C96_atm3DVar_a62f37ab/logs/2021122018/gdasfcst.log

emcbot · 2023-07-17T23:30:18Z

Automated global-workflow Testing Results:

Machine: Hera
Start: Mon Jul 17 22:16:26 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Mon Jul 17 22:21:25 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Mon Jul 17 22:49:00 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 22:49:02 UTC 2023 for experiment C48_S2S_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 22:49:06 UTC 2023 for experiment C96C48_hybatmDA_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 22:49:08 UTC 2023 for experiment C96_atm3DVar_a62f37ab
Experiment C48_S2S_a62f37ab completed: *SUCCESS*
Experiment C48_S2S_a62f37ab Completed at Mon Jul 17 23:30:18 UTC 2023
with 18 successfully completed jobs

RussTreadon-NOAA

Copied bugfix/1748 workflow/rocoto/gfs_tasks.py to HOMEgfs for cycled parallel on Hera. Regenerate EXPDIR xml. Find correct setting for SDATE_GFS in gfsmetp section

<envar><name>SDATE_GFS</name><value>2021080100</value></envar>

Who will confirm that change work for forecast only parallel?

See that CI-Orion Failed. Is the failure related to this change? Seems unlikely but will let g-w team look into this.

emcbot · 2023-07-18T01:12:18Z

Automated global-workflow Testing Results:

Machine: Hera
Start: Mon Jul 17 22:16:26 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Mon Jul 17 22:21:25 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Mon Jul 17 22:49:00 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 22:49:02 UTC 2023 for experiment C48_S2S_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 22:49:06 UTC 2023 for experiment C96C48_hybatmDA_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 22:49:08 UTC 2023 for experiment C96_atm3DVar_a62f37ab
Experiment C48_S2S_a62f37ab completed: *SUCCESS*
Experiment C48_S2S_a62f37ab Completed at Mon Jul 17 23:30:18 UTC 2023
with 18 successfully completed jobs
Experiment C96C48_hybatmDA_a62f37ab completed: *SUCCESS*
Experiment C96C48_hybatmDA_a62f37ab Completed at Tue Jul 18 01:12:18 UTC 2023
with 151 successfully completed jobs

emcbot · 2023-07-18T01:51:10Z

Automated global-workflow Testing Results:

Machine: Hera
Start: Mon Jul 17 22:16:26 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Mon Jul 17 22:21:25 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Mon Jul 17 22:49:00 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 22:49:02 UTC 2023 for experiment C48_S2S_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 22:49:06 UTC 2023 for experiment C96C48_hybatmDA_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 22:49:08 UTC 2023 for experiment C96_atm3DVar_a62f37ab
Experiment C48_S2S_a62f37ab completed: *SUCCESS*
Experiment C48_S2S_a62f37ab Completed at Mon Jul 17 23:30:18 UTC 2023
with 18 successfully completed jobs
Experiment C96C48_hybatmDA_a62f37ab completed: *SUCCESS*
Experiment C96C48_hybatmDA_a62f37ab Completed at Tue Jul 18 01:12:18 UTC 2023
with 151 successfully completed jobs
Experiment C96_atm3DVar_a62f37ab completed: *SUCCESS*
Experiment C96_atm3DVar_a62f37ab Completed at Tue Jul 18 01:51:10 UTC 2023
with 89 successfully completed jobs

emcbot · 2023-07-18T01:54:14Z

Automated global-workflow Testing Results:

Machine: Hera
Start: Mon Jul 17 22:16:26 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Mon Jul 17 22:21:25 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Mon Jul 17 22:49:00 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 22:49:02 UTC 2023 for experiment C48_S2S_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 22:49:06 UTC 2023 for experiment C96C48_hybatmDA_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jul 17 22:49:08 UTC 2023 for experiment C96_atm3DVar_a62f37ab
Experiment C48_S2S_a62f37ab completed: *SUCCESS*
Experiment C48_S2S_a62f37ab Completed at Mon Jul 17 23:30:18 UTC 2023
with 18 successfully completed jobs
Experiment C96C48_hybatmDA_a62f37ab completed: *SUCCESS*
Experiment C96C48_hybatmDA_a62f37ab Completed at Tue Jul 18 01:12:18 UTC 2023
with 151 successfully completed jobs
Experiment C96_atm3DVar_a62f37ab completed: *SUCCESS*
Experiment C96_atm3DVar_a62f37ab Completed at Tue Jul 18 01:51:10 UTC 2023
with 89 successfully completed jobs

aerorahul · 2023-07-18T13:58:04Z

@RussTreadon-NOAA
Thanks for the test.

@malloryprow
Did I correctly catch the use of SDATE_GFS in this PR description? Is that the right use of SDATE_GFS in the metp jobs?

malloryprow · 2023-07-18T15:07:58Z

@aerorahul Yup

TerrenceMcGuinness-NOAA · 2023-07-18T18:30:57Z

Looks like it might be that C96_atm3DVar fails on Orion because it wasn't checked out with the -g switch for cycling. That was a test branch on the master CI repo for the create_experment test. Let me update it to the develop CI master on both Orion and Hera.

emcbot · 2023-07-19T15:44:11Z

Automated global-workflow Testing Results:

Machine: Orion
Start: Wed Jul 19 10:12:15 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Wed Jul 19 10:14:02 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Wed Jul 19 10:43:53 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 19 10:43:59 CDT 2023 for experiment C48_S2S_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 19 10:44:04 CDT 2023 for experiment C96_atm3DVar_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 19 10:44:08 CDT 2023 for experiment C96C48_hybatmDA_a62f37ab

emcbot · 2023-07-19T16:26:16Z

Automated global-workflow Testing Results:

Machine: Orion
Start: Wed Jul 19 10:12:15 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Wed Jul 19 10:14:02 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Wed Jul 19 10:43:53 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 19 10:43:59 CDT 2023 for experiment C48_S2S_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 19 10:44:04 CDT 2023 for experiment C96_atm3DVar_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 19 10:44:08 CDT 2023 for experiment C96C48_hybatmDA_a62f37ab
Experiment C48_S2S_a62f37ab completed: *SUCCESS*
Experiment C48_S2S_a62f37ab Completed at Wed Jul 19 11:26:15 CDT 2023
with 18 successfully completed jobs

emcbot · 2023-07-19T18:10:32Z

Automated global-workflow Testing Results:

Machine: Orion
Start: Wed Jul 19 10:12:15 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Wed Jul 19 10:14:02 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Wed Jul 19 10:43:53 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 19 10:43:59 CDT 2023 for experiment C48_S2S_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 19 10:44:04 CDT 2023 for experiment C96_atm3DVar_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 19 10:44:08 CDT 2023 for experiment C96C48_hybatmDA_a62f37ab
Experiment C48_S2S_a62f37ab completed: *SUCCESS*
Experiment C48_S2S_a62f37ab Completed at Wed Jul 19 11:26:15 CDT 2023
with 18 successfully completed jobs
Experiment C96_atm3DVar_a62f37ab completed: *SUCCESS*
Experiment C96_atm3DVar_a62f37ab Completed at Wed Jul 19 13:10:31 CDT 2023
with 89 successfully completed jobs

emcbot · 2023-07-19T19:02:10Z

Automated global-workflow Testing Results:

Machine: Orion
Start: Wed Jul 19 10:12:15 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Wed Jul 19 10:14:02 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Wed Jul 19 10:43:53 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 19 10:43:59 CDT 2023 for experiment C48_S2S_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 19 10:44:04 CDT 2023 for experiment C96_atm3DVar_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 19 10:44:08 CDT 2023 for experiment C96C48_hybatmDA_a62f37ab
Experiment C48_S2S_a62f37ab completed: *SUCCESS*
Experiment C48_S2S_a62f37ab Completed at Wed Jul 19 11:26:15 CDT 2023
with 18 successfully completed jobs
Experiment C96_atm3DVar_a62f37ab completed: *SUCCESS*
Experiment C96_atm3DVar_a62f37ab Completed at Wed Jul 19 13:10:31 CDT 2023
with 89 successfully completed jobs
Experiment C96C48_hybatmDA_a62f37ab completed: *SUCCESS*
Experiment C96C48_hybatmDA_a62f37ab Completed at Wed Jul 19 14:02:09 CDT 2023
with 151 successfully completed jobs

emcbot · 2023-07-19T19:04:10Z

Automated global-workflow Testing Results:

Machine: Orion
Start: Wed Jul 19 10:12:15 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Wed Jul 19 10:14:02 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Wed Jul 19 10:43:53 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 19 10:43:59 CDT 2023 for experiment C48_S2S_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 19 10:44:04 CDT 2023 for experiment C96_atm3DVar_a62f37ab
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 19 10:44:08 CDT 2023 for experiment C96C48_hybatmDA_a62f37ab
Experiment C48_S2S_a62f37ab completed: *SUCCESS*
Experiment C48_S2S_a62f37ab Completed at Wed Jul 19 11:26:15 CDT 2023
with 18 successfully completed jobs
Experiment C96_atm3DVar_a62f37ab completed: *SUCCESS*
Experiment C96_atm3DVar_a62f37ab Completed at Wed Jul 19 13:10:31 CDT 2023
with 89 successfully completed jobs
Experiment C96C48_hybatmDA_a62f37ab completed: *SUCCESS*
Experiment C96C48_hybatmDA_a62f37ab Completed at Wed Jul 19 14:02:09 CDT 2023
with 151 successfully completed jobs

SDATE_GFS in the context of metp is the first date that METP was run …

a62f37a

…and started generating statistics. This is different when doing cycled (gfs_cyc determines SDATE_GFS). In the case of forecast-only, SDATE_GFS is always the first free-forecast

aerorahul requested review from RussTreadon-NOAA, WalterKolczynski-NOAA and malloryprow July 17, 2023 22:10

WalterKolczynski-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion labels Jul 17, 2023

emcbot added CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress and removed CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Jul 17, 2023

emcbot added CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed and removed CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress labels Jul 17, 2023

RussTreadon-NOAA reviewed Jul 18, 2023

View reviewed changes

emcbot added CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Jul 18, 2023

WalterKolczynski-NOAA approved these changes Jul 18, 2023

View reviewed changes

TerrenceMcGuinness-NOAA added CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion and removed CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed labels Jul 19, 2023

emcbot added CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion and removed CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion labels Jul 19, 2023

TerrenceMcGuinness-NOAA added CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion and removed CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion labels Jul 19, 2023

emcbot added CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion and removed CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion labels Jul 19, 2023

TerrenceMcGuinness-NOAA added CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion and removed CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion labels Jul 19, 2023

emcbot removed the CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress label Jul 19, 2023

emcbot added the CI-Orion-Passed **Bot use only** CI testing on Orion for this PR has completed successfully label Jul 19, 2023

WalterKolczynski-NOAA changed the title ~~Fix SDATE_GFS issue in cycled and forecast-only modes.~~ Fix SDATE_GFS issue in cycled and forecast-only modes Jul 20, 2023

WalterKolczynski-NOAA merged commit cebad69 into develop Jul 20, 2023
5 checks passed

WalterKolczynski-NOAA deleted the bugfix/1748 branch July 20, 2023 01:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix `SDATE_GFS` issue in cycled and forecast-only modes #1753

Fix `SDATE_GFS` issue in cycled and forecast-only modes #1753

aerorahul commented Jul 17, 2023

emcbot commented Jul 17, 2023

emcbot commented Jul 17, 2023

emcbot commented Jul 17, 2023

emcbot commented Jul 17, 2023

RussTreadon-NOAA left a comment

emcbot commented Jul 18, 2023

emcbot commented Jul 18, 2023

emcbot commented Jul 18, 2023

aerorahul commented Jul 18, 2023

malloryprow commented Jul 18, 2023

TerrenceMcGuinness-NOAA commented Jul 18, 2023 •

edited

Loading

emcbot commented Jul 19, 2023

emcbot commented Jul 19, 2023

emcbot commented Jul 19, 2023

emcbot commented Jul 19, 2023

emcbot commented Jul 19, 2023

Fix SDATE_GFS issue in cycled and forecast-only modes #1753

Fix SDATE_GFS issue in cycled and forecast-only modes #1753

Conversation

aerorahul commented Jul 17, 2023

emcbot commented Jul 17, 2023

emcbot commented Jul 17, 2023

emcbot commented Jul 17, 2023

emcbot commented Jul 17, 2023

RussTreadon-NOAA left a comment

Choose a reason for hiding this comment

emcbot commented Jul 18, 2023

emcbot commented Jul 18, 2023

emcbot commented Jul 18, 2023

aerorahul commented Jul 18, 2023

malloryprow commented Jul 18, 2023

TerrenceMcGuinness-NOAA commented Jul 18, 2023 • edited Loading

emcbot commented Jul 19, 2023

emcbot commented Jul 19, 2023

emcbot commented Jul 19, 2023

emcbot commented Jul 19, 2023

emcbot commented Jul 19, 2023

Fix `SDATE_GFS` issue in cycled and forecast-only modes #1753

Fix `SDATE_GFS` issue in cycled and forecast-only modes #1753

TerrenceMcGuinness-NOAA commented Jul 18, 2023 •

edited

Loading