Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding a CI for using Replay ICs with an offset hour of 3 #2788

Merged

Conversation

NeilBarton-NOAA
Copy link
Contributor

Description

This PR adds a CI testing using C96mx100 resolution and the S2SWA app. 2 perturb members are included with the control member.

Currently, this PR is dependent on
#2778
#2755

as the above PRs ensure the product tasks are being triggered in the workflow.

I could likely bypass the dependency #2755 if desired.

I will keep this PR as a draft until further discussion on the dependencies.

Current test ICs are located on hera at
/scratch2/NCEPDEV/stmp3/Neil.Barton/ICs/REPLAY_ICs/CI/2020110100

Type of change

  • New feature (adds functionality)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO

How has this been tested?

This is currently being tested on HERA.

  • Forecast-only on Hera

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • I have made corresponding changes to the documentation if necessary

jobs/JGLOBAL_WAVE_POST_PNT Fixed Show fixed Hide fixed
jobs/JGLOBAL_WAVE_POST_PNT Fixed Show fixed Hide fixed
parm/config/gefs/config.base Fixed Show fixed Hide fixed
parm/config/gefs/config.base Fixed Show fixed Hide fixed
@NeilBarton-NOAA
Copy link
Contributor Author

ready for review. This will have a conflict with @KateFriedman-NOAA 's PR #2651

NeilBarton-NOAA and others added 7 commits August 14, 2024 20:45
Fix a bug in a variable check in the extractvars task where parenthesis in the variable name will prompt a false warning message.
Ice variables are extracted from the history files instead of the product files. This is necessary because there are certain ice variables that are needed that are only available in the history files and not the product files.
@NeilBarton-NOAA
Copy link
Contributor Author

CI test fix updated. Please put through CI tests again

@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed labels Oct 3, 2024
@emcbot emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Oct 3, 2024
@emcbot
Copy link

emcbot commented Oct 5, 2024

CI Passed on Hera in Build# 7
Built and ran in directory /scratch1/NCEPDEV/global/CI/2788


Experiment C96_S2SWA_gefs_replay_ics_922eadff Completed 1 Cycles: *SUCCESS* at Sat Oct  5 03:55:04 UTC 2024
Experiment C48_ATM_922eadff Completed 1 Cycles: *SUCCESS* at Sat Oct  5 03:58:13 UTC 2024
Experiment C48_S2SW_922eadff Completed 1 Cycles: *SUCCESS* at Sat Oct  5 05:15:57 UTC 2024
Experiment C48mx500_3DVarAOWCDA_922eadff Completed 2 Cycles: *SUCCESS* at Sat Oct  5 08:38:15 UTC 2024
Experiment C96_atm3DVar_922eadff Completed 3 Cycles: *SUCCESS* at Sat Oct  5 09:47:09 UTC 2024
Experiment C48_S2SWA_gefs_922eadff Completed 1 Cycles: *SUCCESS* at Sat Oct  5 09:52:54 UTC 2024
Experiment C96C48_ufs_hybatmDA_922eadff Completed 2 Cycles: *SUCCESS* at Sat Oct  5 09:59:28 UTC 2024
Experiment C96C48_hybatmDA_922eadff Completed 3 Cycles: *SUCCESS* at Sat Oct  5 10:24:11 UTC 2024
Experiment C96C48_hybatmaerosnowDA_922eadff Completed 3 Cycles: *SUCCESS* at Sat Oct  5 10:46:08 UTC 2024

@WalterKolczynski-NOAA WalterKolczynski-NOAA added the CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules label Oct 7, 2024
@emcbot emcbot added CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully and removed CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress labels Oct 7, 2024
@emcbot
Copy link

emcbot commented Oct 7, 2024

CI Passed on Hercules in Build# 8
Built and ran in directory /work2/noaa/stmp/CI/HERCULES/2788


Experiment C48_ATM_922eadff Completed 1 Cycles: *SUCCESS* at Mon Oct  7 12:51:11 CDT 2024
Experiment C96_S2SWA_gefs_replay_ics_922eadff Completed 1 Cycles: *SUCCESS* at Mon Oct  7 12:57:19 CDT 2024
Experiment C96C48_hybatmDA_922eadff Completed 3 Cycles: *SUCCESS* at Mon Oct  7 13:51:59 CDT 2024
Experiment C96_atm3DVar_922eadff Completed 3 Cycles: *SUCCESS* at Mon Oct  7 13:57:47 CDT 2024
Experiment C48_S2SWA_gefs_922eadff Completed 1 Cycles: *SUCCESS* at Mon Oct  7 14:59:19 CDT 2024
Experiment C48_S2SW_922eadff Completed 1 Cycles: *SUCCESS* at Mon Oct  7 15:22:48 CDT 2024

@WalterKolczynski-NOAA WalterKolczynski-NOAA merged commit a42c833 into NOAA-EMC:develop Oct 8, 2024
10 of 11 checks passed
@KateFriedman-NOAA
Copy link
Member

@NeilBarton-NOAA I've pulled the updated IC set into ICSDIR on Hera. See here: /scratch1/NCEPDEV/global/glopara/data/ICSDIR/C96mx100/20240610

I pulled the files through the symlinks I saw within your set (rsync -azvL), so as to not have any symlinks to personal files. I also placed it under a timestamp folder. Let me know if the ICs look good on Hera and we'll sync them to the other platforms. After we sync them I will also remove the prior set that I pulled in that wasn't yet under the timestamp folder (/scratch1/NCEPDEV/global/glopara/data/ICSDIR/C96mx100/gefs.20201101).

@NeilBarton-NOAA Checking back on this. Please confirm the above ICs are good-to-go. Thanks!

@NeilBarton-NOAA
Copy link
Contributor Author

@KateFriedman-NOAA The CI tests passed and the code has been merged. The ICs are good-to-go

@KateFriedman-NOAA
Copy link
Member

Thanks for confirming @NeilBarton-NOAA , particularly for our records. I will proceed with deleting the non-timestamped copy (scratch1/NCEPDEV/global/glopara/data/ICSDIR/C96mx100/gefs.20201101).

@NeilBarton-NOAA
Copy link
Contributor Author

@KateFriedman-NOAA I miss understood your question. The CI points to the timestamped ICs

https://github.com/NOAA-EMC/global-workflow/blob/develop/ci/cases/pr/C96_S2SWA_gefs_replay_ics.yaml

If removing the time-stamp folder, the yaml file will need to be edited.

@KateFriedman-NOAA
Copy link
Member

@NeilBarton-NOAA All good, I removed the non-timestamped one. The timestamped one will remain and be used in the CI test.

EricSinsky-NOAA added a commit that referenced this pull request Oct 28, 2024
<!--
  *** PLEASE READ ***

  Any PRs not following this template will be closed.

  Please delete all these comments before submitting the PR.

Please use a short (<60 char), descriptive title for the PR title above.
It should complete the sentence "If merged, this PR will _____".
Capitalize the first word and do not end with a period.

  No content should appear above the "Description" header.

If this PR is not merge-ready (e.g. it depends on other PRs not yet
merged), please mark it as draft until it is ready.

  PRs should meet these guidelines:
  - Each PR should address ONE topic and have an associated issue.
  - No hard-coded paths or personal directories.
  - No temporary or backup files should be committed (including logs).
- Any code that you disabled by being commented out should be removed or
reenabled.
-->
# Description
<!-- This description will become the commit message for the PR. -->
<!--
  Solely pointing to an issue is not an adequate description!

  Please use this format for your description:

Describe your changes. Focus on the *what* and *why*. The *how* will be
evident from the changes. In particular, be sure to note any interface
changes, such as command line syntax, that will need to be communicated
to users.

At the end of your description, please be sure to add the issue this PR
solves using the word "Resolves". If there are any issues that are
related but not yet resolved (including in other repos), you may use
"Refs".

  Resolves #1234
  Refs #4321
  Refs NOAA-EMC/repo#5678
-->

This PR brings recent changes from the develop branch to the GEFS
reforecast branch. This PR updates the GEFS reforecast branch to develop
hash ac3cde5 (10/11/2024). This version
of global-workflow uses the ufs-weather-model hash
[6a4e09e](https://github.com/ufs-community/ufs-weather-model/tree/6a4e09e94773ffa39ce7ab6a54a885efada91f21)
(9/9/2024).

Furthermore, this PR ensures the following adjustments for the
reforecast:

- [x] Speed up rocoto by grouping post job
- [x] Optimize PE configuration
- [x]  Remove duplicate OCNSPPT and EPBL settings
- [x] Set restart_interval to fhmax
- [x] Turn off SHUM in config.efcs
- [x] Set FHMIN_WAV to 3 in config.base 
- [x] Turn off ATM history file output
- [x] Change  HMS=${cyc}0000 to HMS=030000 in Wavepostpnt script (#2788)
- [x] Include YYYYMMDDHH (PDY) in job name
- [x] Change CA seed based on case and cyc for control member and
perturbed members
- [x] Fix post ensemble info
- [x] Add tob to ocean products (#2995 )
- [x] Move PEVPR from b group to a group for atmos products (#2995)
- [x] Add option to download initial condition from HPSS
- [x] Add ability to download and stage replay analysis from AWS, which
is needed for the repair_replay task
- [x] Add capability to run forecasts in 7-day intervals  (#2928)
- [x] Update defaults.yaml so that many of the reforecast-specific
settings can be used by default

<!-- For more on writing good commit messages, see
https://cbea.ms/git-commit/ -->

# Type of change
- [ ] Bug fix (fixes something broken)
- [ ] New feature (adds functionality)
- [x] Maintenance (code refactor, clean-up, new CI test, etc.)

# Change characteristics
<!-- Choose YES or NO from each of the following and delete the other
-->
- Is this a breaking change (a change in existing functionality)? NO
- Does this change require a documentation update? NO
- Does this change require an update to any of the following submodules?
NO
  - [ ] EMC verif-global <!-- NOAA-EMC/EMC_verif-global#1234 -->
  - [ ] GDAS <!-- NOAA-EMC/GDASApp#1234 -->
  - [ ] GFS-utils <!-- NOAA-EMC/gfs-utils#1234 -->
  - [ ] GSI <!-- NOAA-EMC/GSI#1234 -->
  - [ ] GSI-monitor <!-- NOAA-EMC/GSI-Monitor#1234 -->
  - [ ] GSI-utils <!-- NOAA-EMC/GSI-Utils#1234 -->
  - [ ] UFS-utils <!-- ufs-community/UFS_UTILS#1234 -->
  - [ ] UFS-weather-model <!-- ufs-community/ufs-weather-model#1234 -->
  - [ ] wxflow <!-- NOAA-EMC/wxflow#1234 -->

# How has this been tested?
<!-- Please list any test you conducted, including the machine.

Example:
- Clone and build on WCOSS
- Cycled test on Orion
- Forecast-only on Hera
-->

This branch is being tested on WCOSS2. When testing has succeeded, this
PR will be marked as ready for review.

# Checklist
- [ ] Any dependent changes have been merged and published
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have documented my code, including function, input, and output
descriptions
- [ ] My changes generate no new warnings
- [ ] New and existing tests pass with my changes
- [ ] This change is covered by an existing CI test or a new one has
been added
- [ ] I have made corresponding changes to the system documentation if
necessary

---------

Co-authored-by: Wei Huang <[email protected]>
Co-authored-by: Kate Friedman <[email protected]>
Co-authored-by: Cory Martin <[email protected]>
Co-authored-by: Andrew.Tangborn <[email protected]>
Co-authored-by: Walter Kolczynski - NOAA <[email protected]>
Co-authored-by: AndrewEichmann-NOAA <[email protected]>
Co-authored-by: DavidBurrows-NCO <[email protected]>
Co-authored-by: AnningCheng-NOAA <[email protected]>
Co-authored-by: David Huber <[email protected]>
Co-authored-by: Rahul Mahajan <[email protected]>
Co-authored-by: AntonMFernando-NOAA <[email protected]>
Co-authored-by: BoCui-NOAA <[email protected]>
Co-authored-by: DavidNew-NOAA <[email protected]>
Co-authored-by: Jeffrey Whitaker <[email protected]>
Co-authored-by: mingshichen-noaa <[email protected]>
Co-authored-by: Jiarui Dong <[email protected]>
Co-authored-by: David Huber <[email protected]>
Co-authored-by: Guillaume Vernieres <[email protected]>
Co-authored-by: RussTreadon-NOAA <[email protected]>
Co-authored-by: Innocent Souopgui <[email protected]>
Co-authored-by: Neil Barton <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants