Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[develop] Add Sonarqube stage to the Jenkinsfile for PLATFORM-926 #935

Merged

Conversation

MichaelLueken
Copy link
Collaborator

@MichaelLueken MichaelLueken commented Oct 9, 2023

DESCRIPTION OF CHANGES:

The EPIC Platform team is tasked with adding a Sonarqube gate to the SRW App's Jenkins pipeline. @ankimball has added the Sonarqube pipeline into Jenkins, and requested that the Sonarqube stage be added to the Jenkinsfile in the SRW App's GitHub repository. This work is adding the Sonarqube stage to the .cicd/Jenkinsfile.

Additionally, a fix has been made to allow the Functional Workflow Task Tests stage to successfully run on Hera GNU (30 minute wall time isn't enough for the the community test to finish using GNU compiled executables).

Type of change

  • New feature (non-breaking change which adds functionality)

TESTS CONDUCTED:

This is an update to the Jenkinsfile only. There is no need to test this on any platform except for Jenkins.

  • Jenkins

DEPENDENCIES:

None

DOCUMENTATION:

None

ISSUE:

PLATFORM-926

CHECKLIST

  • My code follows the style guidelines in the Contributor's Guide
  • I have performed a self-review of my own code using the Code Reviewer's Guide
  • My changes do not require updates to the documentation (explain).
    This is an update to the Jenkinsfile only. No documentation changes are required.
  • My changes generate no new warnings
  • New and existing tests pass with my changes

CONTRIBUTORS (optional):

@ankimball for providing the Sonarqube stage modifications.

@MichaelLueken MichaelLueken added the enhancement New feature or request label Oct 9, 2023
@MichaelLueken MichaelLueken added the run_we2e_coverage_tests Run the coverage set of SRW end-to-end tests label Oct 9, 2023
@MichaelLueken
Copy link
Collaborator Author

All tests successfully passed, with the exception of custom_ESGgrid_Central_Asia_3km:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used
----------------------------------------------------------------------------------------------------
custom_ESGgrid_Central_Asia_3km                                    DEAD                   5.95
get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_grib2_2019061200          COMPLETE               6.48
get_from_HPSS_ics_GDAS_lbcs_GDAS_fmt_netcdf_2022040400_ensemble_2  COMPLETE             763.07
get_from_HPSS_ics_HRRR_lbcs_RAP                                    COMPLETE              14.21
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2        COMPLETE               6.51
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16_plot     COMPLETE              13.18
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_RAP                 COMPLETE              10.31
grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v15p2        COMPLETE               6.75
grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2         COMPLETE             234.58
grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16           COMPLETE             304.77
grid_RRFS_CONUScompact_3km_ics_HRRR_lbcs_RAP_suite_HRRR            COMPLETE             325.30
pregen_grid_orog_sfc_climo                                         COMPLETE               7.81
----------------------------------------------------------------------------------------------------
Total                                                              DEAD                1698.92

A rerun of the test successfully passed:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
custom_ESGgrid_Central_Asia_3km                                    COMPLETE              29.64
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE              29.64

Currently testing a fix for the occasional issues encountered with the Functional Workflow Task Tests. Once a fix that works on Hera GNU, Hera Intel, Orion, and Hercules is found, the change will be pushed and this PR will be opened.

…age complete and pass successfully on Hera GNU
@MichaelLueken
Copy link
Collaborator Author

An update was added to allow the Functional Workflow Task Tests to run and pass successfully on Hera GNU. The Jenkins tests were run on Hera GNU and Hercules (previous modifications to correct this behavior caused issues on Hercules) and all tests passed. While submitting the tests on these two machines, tests were also submitted on Jet, which had to be aborted both time.

Hera GNU:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used
----------------------------------------------------------------------------------------------------
custom_ESGgrid_Peru_12km                                           COMPLETE              29.21
get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio_2019061200         COMPLETE              15.39
get_from_NOMADS_ics_FV3GFS_lbcs_FV3GFS                             COMPLETE              19.92
grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_HRRR             COMPLETE              55.96
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1beta      COMPLETE              34.48
grid_SUBCONUS_Ind_3km_ics_HRRR_lbcs_RAP_suite_WoFS_v0              COMPLETE              23.11
long_fcst                                                          COMPLETE              76.48
MET_verification_only_vx                                           COMPLETE               0.39
MET_ensemble_verification_only_vx_time_lag                         COMPLETE               8.36
nco_grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16      COMPLETE              66.42
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE             329.72

Hercules:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
custom_GFDLgrid__GFDLgrid_USE_NUM_CELLS_IN_FILENAMES_eq_FALSE      COMPLETE               9.43
grid_CONUS_25km_GFDLgrid_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16      COMPLETE              12.31
grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1beta      COMPLETE              31.23
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v17_p8_plot  COMPLETE              19.07
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_HRRR             COMPLETE              25.68
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_RAP              COMPLETE              57.79
grid_RRFS_CONUScompact_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16   COMPLETE              15.45
grid_RRFS_NA_13km_ics_FV3GFS_lbcs_FV3GFS_suite_RAP                 COMPLETE              66.83
grid_SUBCONUS_Ind_3km_ics_NAM_lbcs_NAM_suite_GFS_v16               COMPLETE              29.37
MET_verification_only_vx                                           COMPLETE               0.37
specify_EXTRN_MDL_SYSBASEDIR_ICS_LBCS                              COMPLETE               9.62
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE             277.15

Opening this PR now.

Copy link
Collaborator

@RatkoVasic-NOAA RatkoVasic-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

Copy link
Collaborator

@EdwardSnyder-NOAA EdwardSnyder-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@MichaelLueken
Copy link
Collaborator Author

Thanks for the reviews, @RatkoVasic-NOAA and @EdwardSnyder-NOAA! Merging this in now.

@MichaelLueken MichaelLueken merged commit f9dd5e1 into ufs-community:develop Oct 13, 2023
4 of 6 checks passed
@MichaelLueken MichaelLueken deleted the feature/add_sonarqube_stage branch October 13, 2023 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request run_we2e_coverage_tests Run the coverage set of SRW end-to-end tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants