Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[develop] Update forecast metric test script #882

Merged
merged 3 commits into from
Aug 14, 2023

Conversation

clouden90
Copy link
Contributor

DESCRIPTION OF CHANGES:

Update srw_metric_example.sh to remove the explicit setting of MET_INSTALL_DIR and MET_BIN_EXEC environment variables. (This change comes after #826, where these variables are sourced directly from the modulefiles.)

TESTS CONDUCTED:

To start, git clone the feature branch, then navigate to the ufs-srweather-app folder and proceed to check out the external repositories. Set the following environment variables: WORKSPACE (path to your ufs-srweather-app folder), SRW_PLATFORM (e.g., hera), SRW_COMPILER (e.g., intel), and SRW_PROJECT (e.g., nems). Once these are set, run the command ./.cicd/scripts/srw_metric_example.sh. This process is expected to take around 2 hours to complete. Afterward, you should find an output file named skill-score.out that contains the skill score index. Tests have been conducted on Hera and Jenkins pipeline.

  • hera.intel
  • orion.intel
  • cheyenne.intel
  • cheyenne.gnu
  • gaea.intel
  • jet.intel
  • wcoss2.intel
  • NOAA Cloud (indicate which platform)
  • Jenkins
  • fundamental test suite
  • comprehensive tests (specify which if a subset was used)

CHECKLIST

  • My code follows the style guidelines in the Contributor's Guide
  • I have performed a self-review of my own code using the Code Reviewer's Guide
  • I have commented my code, particularly in hard-to-understand areas
  • My changes need updates to the documentation. I have made corresponding changes to the documentation
  • My changes do not require updates to the documentation (explain).
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • Any dependent changes have been merged and published

LABELS (optional):

A Code Manager needs to add the following labels to this PR:

  • Work In Progress
  • bug
  • enhancement
  • documentation
  • release
  • high priority
  • run_ci
  • run_we2e_fundamental_tests
  • run_we2e_comprehensive_tests
  • Needs Cheyenne test
  • Needs Jet test
  • Needs Hera test
  • Needs Orion test
  • help wanted

CONTRIBUTORS (optional):

Copy link
Collaborator

@RatkoVasic-NOAA RatkoVasic-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

Copy link
Collaborator

@MichaelLueken MichaelLueken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@clouden90 -

These changes look good to me! However, I did note that the old regional_workflow conda environment is still being loaded. In PR #793, the conda environment was changed to workflow_tools. It would be a good idea to change line 49 so that it reads:

conda activate workflow_tools

The test was run with both regional_workflow and workflow_tools. Both tests successfully passed. Will approve after regional_workflow has been updated to workflow_tools.

.cicd/scripts/srw_metric_example.sh Show resolved Hide resolved
Copy link
Collaborator

@MichaelLueken MichaelLueken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @clouden90! Approving now.

@MichaelLueken MichaelLueken added the run_we2e_coverage_tests Run the coverage set of SRW end-to-end tests label Aug 10, 2023
@MichaelLueken
Copy link
Collaborator

The Cheyenne Intel WE2E coverage tests were manually run on Hera and all successfully passed:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
custom_GFDLgrid__GFDLgrid_USE_NUM_CELLS_IN_FILENAMES_eq_FALSE      COMPLETE              11.63
grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16_plot     COMPLETE              30.07
grid_RRFS_CONUS_25km_ics_NAM_lbcs_NAM_suite_GFS_v16                COMPLETE              19.78
grid_RRFS_CONUScompact_13km_ics_HRRR_lbcs_RAP_suite_HRRR           COMPLETE              25.56
grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta    COMPLETE               8.59
grid_SUBCONUS_Ind_3km_ics_HRRR_lbcs_HRRR_suite_HRRR                COMPLETE              16.46
pregen_grid_orog_sfc_climo                                         COMPLETE               7.76
specify_template_filenames                                         COMPLETE               7.54
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE             127.39

The Cheyenne GNU WE2E coverage tests were manually run on Hera and all successfully passed:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
grid_CONUS_25km_GFDLgrid_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16      COMPLETE              21.67
grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1beta      COMPLETE             234.41
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp  COMPLETE             109.27
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v17_p8_plot  COMPLETE              27.35
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_HRRR             COMPLETE              36.51
grid_RRFS_CONUScompact_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16   COMPLETE              23.94
grid_RRFS_NA_13km_ics_FV3GFS_lbcs_FV3GFS_suite_RAP                 COMPLETE             320.20
grid_SUBCONUS_Ind_3km_ics_NAM_lbcs_NAM_suite_GFS_v16               COMPLETE              51.59
specify_EXTRN_MDL_SYSBASEDIR_ICS_LBCS                              COMPLETE              10.36
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE             835.30

Unfortunately, the hfv3gfs allocations on Jet are oversubscribed and the Jenkins tests were unable to run due to the disk quota being exceeded. Currently running the tests manually on Jet as well. Will update once complete.

@MichaelLueken
Copy link
Collaborator

The Jet tests were manually run and have successfully completed:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
community                                                          COMPLETE              17.90
custom_ESGgrid                                                     COMPLETE              15.75
custom_GFDLgrid                                                    COMPLETE              12.61
get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio_2021032018         COMPLETE              10.22
get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_netcdf_2022060112_48h     COMPLETE              50.57
get_from_HPSS_ics_RAP_lbcs_RAP                                     COMPLETE              12.92
grid_RRFS_AK_3km_ics_FV3GFS_lbcs_FV3GFS_suite_HRRR                 COMPLETE             230.62
grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16_plot     COMPLETE              39.33
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2        COMPLETE               8.20
grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1beta       COMPLETE             512.88
nco_grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR       COMPLETE              10.88
process_obs                                                        COMPLETE               0.45
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE             922.33

Now moving forward with merging this work.

@MichaelLueken MichaelLueken merged commit 6b57195 into ufs-community:develop Aug 14, 2023
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working run_we2e_coverage_tests Run the coverage set of SRW end-to-end tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants