Skip to content

Commit

Permalink
rrfs_ci: Update to top of authoritative develop (#156)
Browse files Browse the repository at this point in the history
* Add support on NSSL/Odin (#227)

* Add support on NSSL/Odin

* Add wlfow_odin.env and modify detect_machine.sh

* update comment

* Detect explicitly odin1/odin2

* Add python module to cheyenne build environments (#232)

* Update SRW Documentation (#212)

* updated docs

* added git submodule

* fix formatting

* added new submodule commits

* fixed ref links

* finished Intro

* finish Components & Intro edits

* edited Rocoto workflow section of Quickstart

* added minor hpc submodule commits

* Updates to Rocoto Workflow in Quick Start

* add to HPC-stack intro

* submodule updates

* added submodule docs edits

* hpc-stack updates & formatting fixes

* hpc-stack intro edits

* bibtex attempted fix

* add hpc-stack module edits

* update sphinxcontrib version

* add .readthedocs.yaml file

* update .readthedocs.yaml file

* update .readthedocs.yaml file

* update conf.py

* updates .readthedocs.yaml with submodules

* updates .readthedocs.yaml with submodules

* submodule updates

* submodule updates

* minor Intro edits

* minor Intro edits

* minor Intro edits

* submodule updates

* fixed typos in QS

* QS updates

* QS updates

* QS updates

* updates to InputOutput and QS

* fix I/O doc typos

* pull updates to hpc-stack docs

* pull updates to hpc-stack docs

* fix table wrapping

* updates to QS for cloud

* fix QS export statements

* fix QS export statements

* QS edits on bind, config

* add bullet points to notes

* running without rocoto

* add HPC-Stack submodule w/docs

* split QS into container/non-container approaches

* added filepath changes for running in container on Orion, et al.

* edits to overview and container QS

* moved CodeReposAndDirs.rst info to the Introduction & deleted file

* continued edits to SRWAppOverview

* combine overview w/non-container docs

* finish merging non-container guide & SRWOverview, rename/remove files, update FAQ

* minor edits for Intro & QS

* updates to BuildRun doc through 3.8.1

* edits to Build/Run and Components

* remove .gitignore

* fix Ch 3 title, 4 supported platform levels note

* fix typos, add term links

* other minor fixes/suggestions implemented

* updated Intro based on feedback; changed SRW to SRW App throughout

* update comment to Intro citation

* add user-defined vertical levels to future work

* Add instructions for srw_common module load

* fix typo

* update Intro & BuildRunSRW based on Mark's feedback

* minor intro updates

* 1st round of jwolff's edits

* 2nd round of jwolff updates

* update QS intro

* fix minor physics details

* update citation and physics suite name

* add compute node allocation info to QS

* add authoritative hpc-stack docs to Intro

Co-authored-by: gspetro <[email protected]>

* Update hashes of all components to latest versions (#233)

* Add a Contributor's Guide feature to the docs (#228)

* create contributor's guide

* add guidelines for making good PR

* good pull request edits

* 1st Draft of COntributor's Guide

* minor formatting edits

* Add instructions for srw_common module load

* fix module files guidance

* fix typo

* update Intro & BuildRunSRW based on Mark's feedback

* minor intro updates

* add info on documentation requirements

* 1st round of J. Beck's edits

* add textonly issue template + minor typo-type fixes

* minor edits/formatting

* remove pylintrc requirement

* fixed .yaml & other minor

Co-authored-by: gspetro <[email protected]>

* Updates to parameters in config_defaults .rst files (#237)

* edit config intro & platform environment sections

* edit sections on cron & directory parameters, platform & parameters for running without a workflow manager

* edit NCO, file-separator, filename params, add some METplus and model config params

* ConfigWorkflow.rst revisions, added METplus to Components, grid info

* add grid config details

* changes to readme.md

* RTD readme.md edits

* create MacOS install/build instructions

* update task run and grid parameters

* fixed file params & workflow task params

* 1st draft of ConfigParameters.inc

* minor edits

* add stochastic physics var details

* update FVCOM, thread affinity params

* halo_blend, ens, crtm, custom post, subhourly updates

* update HPC-Stack submodule/docs

* Rocoto WF tasks & params

* workflow tasks/params, debug, verbose, pre-existing dir, predefined grid

* move Stochastic physics to CCP section; write component edits

* comp'l forecast, grid gen, NOMADS, user-staged files

* METplus, model config & forecast params, separator

* 2nd draft complete

* physics updates

* remove MacInstall empty file

* undo hpc-stack submodule update (save for separate PR)

* undo hpc-stack install doc update (save for separate PR)

* revisions to SPP & LSM physics

* minor edits

* update comments in LAM Grid chapter

* update LSM_SPP_EACH_STEP

* revert LSM_SPP_EACH_STEP to original definition

* combine config info into one doc instead of two

Co-authored-by: gspetro <[email protected]>

* Added functionality for MacOS X (#242)

* Added functionality for MacOS X

 Functionality for MacOS, updated module list in srw_common

* Update build_macos_gnu.env

* Update srw_common

* Update build_macos_gnu.env

The env/build_mac_gnu.env does not load srw_common module, but instead loads individual HPC-stack modules built locally on the Mac that contain higher-versions of some packages. This avoids conflicts with SRW builds for other platforms.

* Update srw_common

corrected the version of the gftl-shared

* Update build_macos_gnu.env

No need to load libpng module found in srw_common, as this is not being built as a part of the hpc-stack on MacOS X, rather installed system-wide.

Co-authored-by: Natalie Perlin <[email protected]>

* Add gaea to supported platforms (#236)

* fixes for gaea

* updates for gaea

* tweak for build env

* 2nd tweak for build env

* Fixes for slurm

* another fix for env

* added version for cmake

* Update Externals.cfg

* Update wflow_gaea.env

* tweak

* pulling Externals.cfg explicitly from develop

* temporarily removing externals

* Checked out directly from develop

* Bug fix with singularity env files (#245)

* Replace bash env files with modules  (#238)

* Pass machine name to build scripts.

* Use modules environment instead of shell scripts.

* Leave conda activation to the user.

* Remove set_machine script.

* Rename env to modulefiles

* Minor fix.

* Minor fix

* Take out *module purge* from modufiles and put it in devbuild.sh

* Activate conda directly in signularity modulefile.

* Minor fixes.

* Add Gaea modulefiles.

* Restore odin env files.

* Bug fixes in singularity modulefiles.

* Move activation of Lmod to devbuild.sh

* Don't do 'module purge' on cray systems

* Put Lmod initialization code in separate script.

* Go back to using modulefile for odin.

* Optionally pass machine name to lmod-setup.sh

* Modify odin wflow modulefile.

* Allow unknown platforms in devbuild.sh

* Update documentation.

* Move cmake init out of lmod-setup.sh on odin

* Also update markup language build documentation.

* Lmod setup script for both bash and tcsh login shells.

* Some fixes for tcsh login shell.

* Add singularity platform to lmod-setup

* update hash of regional workflow (#247)

* Update WE2E documentation (#241)

## DESCRIPTION OF CHANGES: 
This updates the documentation for how to use the WE2E testing system.

## TESTS CONDUCTED: 
Compiled the documentation using `sphinx-build` and viewed the resulting html.

## DEPENDENCIES:
PR #[745](ufs-community/regional_workflow#745) in regional_workflow.

## CONTRIBUTORS: 
@gspetro

* fixes for gaea modules (#248)

* Update regional_workflow hash, add shortcuts for common devbuild.sh options (#251)

* Modifications to `CODEOWNERS` file (#252)

## DESCRIPTION OF CHANGES: 
Make the following modifications to the github `CODEOWNERS` file:
1) Add several EPIC staff (@gspetro-NOAA, @natalie-perlin, and @EdwardSnyder-NOAA) so they are notified of all PRs and can review them.
2) Remove duplicate entries.
3) Remove users who will no longer be working with the repo (thus far only @jwolff-ncar) .

## TESTS CONDUCTED: 
None.

* Add verification tasks to documentation (#243)


* METplus, model config & forecast params, separator

* remove MacInstall empty file

* undo hpc-stack submodule update (save for separate PR)

* undo hpc-stack install doc update (save for separate PR)

* combine config info into one doc instead of two

* remove ConfigParameters.inc (contents now appear in ConfigWorkflow.rst)

* add VX tables, config info, & Rocoto output tables

* add module use/load statements, fix typos

* varied minor details

* add workflow svg diagram

* condense VX task table using ##

* update README

* add png and revert hpc-stack commits until PR#240 (mac docs) is approved

* jwolff edits

* add info on run_vx.local

Co-authored-by: gspetro <[email protected]>

* Add NOAA cloud platforms to SRW (#221)

* updates for noaacloud

* working version

* pointing to noaa-epic for testing

* changes for noaacloud

* switched to load-any

* fix for regional_workflow pointer (#260)

* update hash of regional workflow (#261)

* Feature/cheyenne fix (#258)

* Adding a github action to build on cheyenne with intel

* fixing yml

* fixes for missing load-any on cheyenne

* added pio as well

* Update .github/workflows/build.yml

Co-authored-by: Will Mayfield <[email protected]>

Co-authored-by: Will Mayfield <[email protected]>

* tweaks for build/run on gaea (#254)

* tweaks for build/run on gaea

* fixed path

* Updates for PR

* Check-in Jenkinsfile and unified scripts (#253)

* Add Jenkinsfile that includes a build and test pipeline, which
leverages the unified build and test scripts
* Supported platforms are Cheyenne, Gaea, and Orion
* Supported compilers are GNU and Intel

* Fix for miniconda3 load on Hera (#257)

Pin down the version of miniconda3 on Hera, and do not append to the module path

* Updates to Remaining Chapters (6 & 8-12) of SRW Docs/User's Guide (#255)

* updated docs

* added git submodule

* fix formatting

* added new submodule commits

* fixed ref links

* finished Intro

* finish Components & Intro edits

* edited Rocoto workflow section of Quickstart

* added minor hpc submodule commits

* Updates to Rocoto Workflow in Quick Start

* add to HPC-stack intro

* submodule updates

* added submodule docs edits

* hpc-stack updates & formatting fixes

* hpc-stack intro edits

* bibtex attempted fix

* add hpc-stack module edits

* update sphinxcontrib version

* add .readthedocs.yaml file

* update .readthedocs.yaml file

* update .readthedocs.yaml file

* update conf.py

* updates .readthedocs.yaml with submodules

* updates .readthedocs.yaml with submodules

* submodule updates

* submodule updates

* minor Intro edits

* minor Intro edits

* minor Intro edits

* submodule updates

* fixed typos in QS

* QS updates

* QS updates

* QS updates

* updates to InputOutput and QS

* fix I/O doc typos

* pull updates to hpc-stack docs

* pull updates to hpc-stack docs

* fix table wrapping

* updates to QS for cloud

* fix QS export statements

* fix QS export statements

* QS edits on bind, config

* add bullet points to notes

* running without rocoto

* add HPC-Stack submodule w/docs

* split QS into container/non-container approaches

* added filepath changes for running in container on Orion, et al.

* edits to overview and container QS

* moved CodeReposAndDirs.rst info to the Introduction & deleted file

* continued edits to SRWAppOverview

* combine overview w/non-container docs

* finish merging non-container guide & SRWOverview, rename/remove files, update FAQ

* minor edits for Intro & QS

* updates to BuildRun doc through 3.8.1

* edits to Build/Run and Components

* remove .gitignore

* fix Ch 3 title, 4 supported platform levels note

* fix typos, add term links

* other minor fixes/suggestions implemented

* updated Intro based on feedback; changed SRW to SRW App throughout

* update comment to Intro citation

* add user-defined vertical levels to future work

* Add instructions for srw_common module load

* fix typo

* update Intro & BuildRunSRW based on Mark's feedback

* minor intro updates

* 1st round of jwolff's edits

* 2nd round of jwolff updates

* update QS intro

* fix minor physics details

* update citation and physics suite name

* add compute node allocation info to QS

* add authoritative hpc-stack docs to Intro

* edit config intro & platform environment sections

* edit sections on cron & directory parameters, platform & parameters for running without a workflow manager

* edit NCO, file-separator, filename params, add some METplus and model config params

* ConfigWorkflow.rst revisions, added METplus to Components, grid info

* add grid config details

* changes to readme.md

* RTD readme.md edits

* create MacOS install/build instructions

* update task run and grid parameters

* fixed file params & workflow task params

* 1st draft of ConfigParameters.inc

* minor edits

* add stochastic physics var details

* update FVCOM, thread affinity params

* halo_blend, ens, crtm, custom post, subhourly updates

* update HPC-Stack submodule/docs

* remove extra macinstall document

* Rocoto WF tasks & params

* workflow tasks/params, debug, verbose, pre-existing dir, predefined grid

* move Stochastic physics to CCP section; write component edits

* comp'l forecast, grid gen, NOMADS, user-staged files

* METplus, model config & forecast params, separator

* 2nd draft complete

* physics updates

* remove MacInstall empty file

* undo hpc-stack submodule update (save for separate PR)

* undo hpc-stack install doc update (save for separate PR)

* revert hpc-stack submodule update

* revisions to SPP & LSM physics

* minor edits

* update comments in LAM Grid chapter

* update LSM_SPP_EACH_STEP

* revert LSM_SPP_EACH_STEP to original definition

* combine config info into one doc instead of two

* remove ConfigParameters.inc (contents now appear in ConfigWorkflow.rst)

* update hpc-stack docs submodule

* odds & ends

* add VX tables, config info, & Rocoto output tables

* add module use/load statements, fix typos

* varied minor details

* add workflow svg diagram

* edits to rocoto ch

* updates to Rocoto chapter

* fix minor formatting/wording issues

* updates to LAMgrid chapter

* LAM Grid edits

* LAM ch: user-defined grid section

* add UPP Product tables ch 6

* I/O edits & glossary terms

* I/O Pt2

* I/O changes

* include updated images

* update docs to reflect changes in PR #238

* Graphics Ch-1st pass

* minor updates to Graphics

* minor updates to Graphics

* edit ConfigNewPlatform sections 1-4

* ConfigNewPlatform edits

* resolve merge conflicts

* I/O ch edits

* I/O edits

* more I/O edits

* hpc-stack submodule updates

* add HPC-Stack MacOs info

* WE2E edits & tables

* fix typo

* minor grammar/typos

* merge conflict resolution

* merge conflict resolution

* fix grid name

* remove resolved comments

* add compact grids

* file path updates & info for HPC-Stack

* add SRW prereqs to Intro

* change ConfigNewPlatform to a non-container quickstart

* clean up non-container quickstart

* update build options for non-container QS

* update file paths & WE2E

* minor fixes

* update I/O & Gaea file paths

* update error in non-container QS

* add warning for users w/o Rocoto

* add UPP Satellite Product instructions

* Xlink for UPP satellite output info

Co-authored-by: gspetro <[email protected]>

* Update compiler prerequisite in docs (#267)

* updated docs

* added git submodule

* fix formatting

* added new submodule commits

* fixed ref links

* finished Intro

* finish Components & Intro edits

* edited Rocoto workflow section of Quickstart

* added minor hpc submodule commits

* Updates to Rocoto Workflow in Quick Start

* add to HPC-stack intro

* submodule updates

* added submodule docs edits

* hpc-stack updates & formatting fixes

* hpc-stack intro edits

* bibtex attempted fix

* add hpc-stack module edits

* update sphinxcontrib version

* add .readthedocs.yaml file

* update .readthedocs.yaml file

* update .readthedocs.yaml file

* update conf.py

* updates .readthedocs.yaml with submodules

* updates .readthedocs.yaml with submodules

* submodule updates

* submodule updates

* minor Intro edits

* minor Intro edits

* minor Intro edits

* submodule updates

* fixed typos in QS

* QS updates

* QS updates

* QS updates

* updates to InputOutput and QS

* fix I/O doc typos

* pull updates to hpc-stack docs

* pull updates to hpc-stack docs

* fix table wrapping

* updates to QS for cloud

* fix QS export statements

* fix QS export statements

* QS edits on bind, config

* add bullet points to notes

* running without rocoto

* add HPC-Stack submodule w/docs

* split QS into container/non-container approaches

* added filepath changes for running in container on Orion, et al.

* edits to overview and container QS

* moved CodeReposAndDirs.rst info to the Introduction & deleted file

* continued edits to SRWAppOverview

* combine overview w/non-container docs

* finish merging non-container guide & SRWOverview, rename/remove files, update FAQ

* minor edits for Intro & QS

* updates to BuildRun doc through 3.8.1

* edits to Build/Run and Components

* remove .gitignore

* fix Ch 3 title, 4 supported platform levels note

* fix typos, add term links

* other minor fixes/suggestions implemented

* updated Intro based on feedback; changed SRW to SRW App throughout

* update comment to Intro citation

* add user-defined vertical levels to future work

* Add instructions for srw_common module load

* fix typo

* update Intro & BuildRunSRW based on Mark's feedback

* minor intro updates

* 1st round of jwolff's edits

* 2nd round of jwolff updates

* update QS intro

* fix minor physics details

* update citation and physics suite name

* add compute node allocation info to QS

* add authoritative hpc-stack docs to Intro

* edit config intro & platform environment sections

* edit sections on cron & directory parameters, platform & parameters for running without a workflow manager

* edit NCO, file-separator, filename params, add some METplus and model config params

* ConfigWorkflow.rst revisions, added METplus to Components, grid info

* add grid config details

* changes to readme.md

* RTD readme.md edits

* create MacOS install/build instructions

* update task run and grid parameters

* fixed file params & workflow task params

* 1st draft of ConfigParameters.inc

* minor edits

* add stochastic physics var details

* update FVCOM, thread affinity params

* halo_blend, ens, crtm, custom post, subhourly updates

* update HPC-Stack submodule/docs

* remove extra macinstall document

* Rocoto WF tasks & params

* workflow tasks/params, debug, verbose, pre-existing dir, predefined grid

* move Stochastic physics to CCP section; write component edits

* comp'l forecast, grid gen, NOMADS, user-staged files

* METplus, model config & forecast params, separator

* 2nd draft complete

* physics updates

* remove MacInstall empty file

* undo hpc-stack submodule update (save for separate PR)

* undo hpc-stack install doc update (save for separate PR)

* revert hpc-stack submodule update

* revisions to SPP & LSM physics

* minor edits

* update comments in LAM Grid chapter

* update LSM_SPP_EACH_STEP

* revert LSM_SPP_EACH_STEP to original definition

* combine config info into one doc instead of two

* remove ConfigParameters.inc (contents now appear in ConfigWorkflow.rst)

* update hpc-stack docs submodule

* odds & ends

* add VX tables, config info, & Rocoto output tables

* add module use/load statements, fix typos

* varied minor details

* add workflow svg diagram

* edits to rocoto ch

* updates to Rocoto chapter

* fix minor formatting/wording issues

* updates to LAMgrid chapter

* LAM Grid edits

* LAM ch: user-defined grid section

* add UPP Product tables ch 6

* I/O edits & glossary terms

* I/O Pt2

* I/O changes

* include updated images

* update docs to reflect changes in PR #238

* Graphics Ch-1st pass

* minor updates to Graphics

* minor updates to Graphics

* edit ConfigNewPlatform sections 1-4

* ConfigNewPlatform edits

* resolve merge conflicts

* I/O ch edits

* I/O edits

* more I/O edits

* hpc-stack submodule updates

* add HPC-Stack MacOs info

* WE2E edits & tables

* fix typo

* minor grammar/typos

* merge conflict resolution

* merge conflict resolution

* fix grid name

* remove resolved comments

* add compact grids

* file path updates & info for HPC-Stack

* add SRW prereqs to Intro

* change ConfigNewPlatform to a non-container quickstart

* clean up non-container quickstart

* update build options for non-container QS

* update file paths & WE2E

* minor fixes

* update I/O & Gaea file paths

* update error in non-container QS

* add warning for users w/o Rocoto

* add UPP Satellite Product instructions

* Xlink for UPP satellite output info

* clarify compiler prereqs

Co-authored-by: gspetro <[email protected]>

Co-authored-by: Yunheng Wang <[email protected]>
Co-authored-by: Will Mayfield <[email protected]>
Co-authored-by: Gillian Petro <[email protected]>
Co-authored-by: gspetro <[email protected]>
Co-authored-by: Michael Kavulich <[email protected]>
Co-authored-by: Natalie Perlin <[email protected]>
Co-authored-by: Natalie Perlin <[email protected]>
Co-authored-by: Mark Potts <[email protected]>
Co-authored-by: danielabdi-noaa <[email protected]>
Co-authored-by: Chan-Hoo.Jeon-NOAA <[email protected]>
Co-authored-by: gsketefian <[email protected]>
Co-authored-by: Jesse McFarland <[email protected]>
  • Loading branch information
13 people authored Jun 7, 2022
1 parent 52c10ab commit 3915859
Show file tree
Hide file tree
Showing 87 changed files with 6,376 additions and 3,203 deletions.
169 changes: 169 additions & 0 deletions .cicd/Jenkinsfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
pipeline {
agent none

options {
skipDefaultCheckout(true)
}

parameters {
// Allow job runner to filter based on platform
// choice(name: 'SRW_PLATFORM_FILTER', choices: ['all', 'cheyenne', 'gaea', 'orion', 'pcluster_noaa_v2_use1', 'azcluster_noaa', 'gcluster_noaa_v2_usc1'], description: 'Specify the platform(s) to use')
choice(name: 'SRW_PLATFORM_FILTER', choices: ['all', 'cheyenne', 'gaea', 'orion'], description: 'Specify the platform(s) to use')
// Allow job runner to filter based on compiler
choice(name: 'SRW_COMPILER_FILTER', choices: ['all', 'gnu', 'intel'], description: 'Specify the compiler(s) to use to build')
}

stages {
/*
// Start the NOAA Parallel Works clusters, if necessary
stage('Start Parallel Works Clusters') {
matrix {
// Start all clusters by default or only the specified cluster given by SRW_PLATFORM_FILTER
when {
anyOf {
expression { params.SRW_PLATFORM_FILTER == 'all' }
expression { params.SRW_PLATFORM_FILTER == env.SRW_PLATFORM }
}
}
axes {
axis {
name 'SRW_PLATFORM'
values 'pcluster_noaa_v2_use1', 'azcluster_noaa', 'gcluster_noaa_v2_usc1'
}
}
stages {
// Call the parallel-works-jenkins-client/start-cluster job using SRW_PLATFORM for the
// PW_CLUSTER_NAME parameter
stage('Start Cluster') {
steps {
build job: 'parallel-works-jenkins-client/start-cluster', parameters: [string(name: 'PW_CLUSTER_NAME', value: env.SRW_PLATFORM), string(name: 'PW_CLUSTER_SSH_KEY', value: '~/.ssh/id_rsa'), string(name: 'JAVA_VERSION', value: '11')]
}
}
}
}
}
*/

// Build and test the SRW application on all supported platforms using the supported compilers for each platform
stage('Build and Test') {
matrix {
// Run on all platform/compiler combinations by default or build and test only on the platform(s) and
// compiler(s) specified by SRW_PLATFORM_FILTER and SRW_COMPILER_FILTER
when {
allOf {
anyOf {
expression { params.SRW_PLATFORM_FILTER == 'all' }
expression { params.SRW_PLATFORM_FILTER == env.SRW_PLATFORM }
}

anyOf {
expression { params.SRW_COMPILER_FILTER == 'all' }
expression { params.SRW_COMPILER_FILTER == env.SRW_COMPILER }
}
}
}

axes {
axis {
name 'SRW_PLATFORM'
// values 'cheyenne', 'gaea', 'orion', 'pcluster_noaa_v2_use1', 'azcluster_noaa', 'gcluster_noaa_v2_usc1'
values 'cheyenne', 'gaea', 'orion'
}

axis {
name 'SRW_COMPILER'
values 'gnu', 'intel'
}
}

excludes {
// Exclude GNU from platforms that don't support it
exclude {
axis {
name 'SRW_PLATFORM'
values 'gaea', 'orion'
}

axis {
name 'SRW_COMPILER'
values 'gnu'
}
}
}

agent {
label env.SRW_PLATFORM
}

environment {
BUILD_VERSION = "${env.SRW_PLATFORM}-${env.SRW_COMPILER}-${env.BRANCH_NAME}-${env.BUILD_NUMBER}"
BUILD_NAME = "ufs-srweather-app_${env.BUILD_VERSION}"
}

stages {
// Clean the workspace, checkout the repository, and run checkout_externals
stage('Initialize') {
steps {
echo "Initializing SRW (${env.SRW_COMPILER}) build environment on ${env.SRW_PLATFORM}"
cleanWs()
checkout scm
sh '"${WORKSPACE}/manage_externals/checkout_externals"'
}
}

// Run the unified build script; if successful create a tarball of the build and upload to S3
stage('Build') {
steps {
echo "Building SRW (${env.SRW_COMPILER}) on ${env.SRW_PLATFORM}"
sh 'bash --login "${WORKSPACE}/.cicd/scripts/srw_build.sh"'
}

post {
success {
sh 'tar --create --gzip --verbose --file "${WORKSPACE}/${BUILD_NAME}.tgz" bin include lib share'
s3Upload consoleLogLevel: 'INFO', dontSetBuildResultOnFailure: false, dontWaitForConcurrentBuildCompletion: false, entries: [[bucket: 'woc-epic-jenkins-artifacts', excludedFile: '', flatten: false, gzipFiles: false, keepForever: false, managedArtifacts: true, noUploadOnFailure: true, selectedRegion: 'us-east-1', showDirectlyInBrowser: false, sourceFile: "${env.BUILD_NAME}.tgz", storageClass: 'STANDARD', uploadFromSlave: false, useServerSideEncryption: false]], pluginFailureResultConstraint: 'FAILURE', profileName: 'main', userMetadata: []
}
}
}

// Run the unified test script
stage('Test') {
steps {
echo "Testing SRW (${env.SRW_COMPILER}) on ${env.SRW_PLATFORM}"
sh 'bash --login "${WORKSPACE}/.cicd/scripts/srw_test.sh"'
}
}
}
}
}
}

/*
post {
always {
// Stop any Parallel Works clusters that were started during the pipeline execution
script {
def pw_clusters = ['pcluster_noaa_v2_use1', 'azcluster_noaa', 'gcluster_noaa_v2_usc1']
def clusters = []
// Determine which clusters need to be stopped, if any
if (params.SRW_PLATFORM_FILTER == 'all') {
clusters = pw_clusters
} else if (params.SRW_PLATFORM_FILTER in pw_clusters) {
clusters = [params.SRW_PLATFORM_FILTER]
} else {
echo 'No Parallel Works clusters were used in build'
}
for (int i = 0; i < clusters.size(); ++i) {
// Call the parallel-works-jenkins-client/stop-cluster job using clusters[i] for the
// PW_CLUSTER_NAME parameter
build job: 'parallel-works-jenkins-client/stop-cluster', parameters: [string(name: 'PW_CLUSTER_NAME', value: clusters[i])]
}
}
}
}
*/
}
31 changes: 31 additions & 0 deletions .cicd/scripts/srw_build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#!/usr/bin/env bash
#
# A unified build script for the SRW application. This script is expected to
# build the SRW application for all supported platforms.
#
set -e -u -x

script_dir="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" > /dev/null 2>&1 && pwd)"

# Get repository root from Jenkins WORKSPACE variable if set, otherwise, set
# relative to script directory.
declare workspace
if [[ -n "${WORKSPACE}" ]]; then
workspace="${WORKSPACE}"
else
workspace="$(cd -- "${script_dir}/../.." && pwd)"
fi

build_dir="${workspace}/build"

# Set build related environment variables and load required modules.
source "${workspace}/etc/lmod-setup.sh" "${SRW_PLATFORM}"
module use "${workspace}/modulefiles"
module load "build_${SRW_PLATFORM}_${SRW_COMPILER}"

# Compile SRW application and install to repository root.
mkdir "${build_dir}"
pushd "${build_dir}"
cmake -DCMAKE_INSTALL_PREFIX="${workspace}" "${workspace}"
make -j "${MAKE_JOBS}"
popd
21 changes: 21 additions & 0 deletions .cicd/scripts/srw_test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/usr/bin/env bash
#
# A unified test script for the SRW application. This script is expected to
# test the SRW application for all supported platforms. NOTE: At this time,
# this script is a placeholder for a more robust test framework.
#
set -e -u -x

script_dir="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" > /dev/null 2>&1 && pwd)"

# Get repository root from Jenkins WORKSPACE variable if set, otherwise, set
# relative to script directory.
declare workspace
if [[ -n "${WORKSPACE}" ]]; then
workspace="${WORKSPACE}"
else
workspace="$(cd -- "${script_dir}/../.." && pwd)"
fi

# Verify that there is a non-zero sized weather model executable.
[[ -s "${workspace}/bin/ufs_model" ]] || [[ -s "${workspace}/bin/NEMS.exe" ]]
1 change: 0 additions & 1 deletion .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
# These owners will be the default owners for everything in the repo.
#* @defunkt
* @robgonzalezpita @venitahagerty @christopherwharrop-NOAA @christinaholtNOAA

# Order is important. The last matching pattern has the most precedence.
# So if a pull request only touches javascript files, only these owners
# will be requested to review.
Expand Down
20 changes: 20 additions & 0 deletions .github/ISSUE_TEMPLATE/textonly_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
name: Text-only request
about: Suggest an idea for this project
title: ''
labels: textonly
assignees: ''

---

## Description
Provide a clear and concise description of the problem to be solved.

## Solution
Add a clear and concise description of the proposed solution.

## Alternatives (optional)
If applicable, add a description of any alternative solutions or features you've considered.

## Related to (optional)
Directly reference any issues or PRs in this or other repositories that this is related to, and describe how they are related.
21 changes: 21 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: Build SRW
on: [push]
jobs:
Explore-GitHub-Actions:
runs-on: [self-hosted, ncar]
steps:
- run: echo "🎉 The job was automatically triggered by a ${{ github.event_name }} event."
- run: echo "🐧 This job is now running on a ${{ runner.os }} server hosted by GitHub!"
- run: echo "🔎 The name of your branch is ${{ github.ref }} and your repository is ${{ github.repository }}."
- name: Check out repository code
uses: actions/checkout@v2
- run: |
cd ufs-srweather-app
./manage_externals/checkout_externals
module use modulefiles
module load build_cheyenne_intel
mdkir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=.. ..
make -j 4
- run: echo "🍏 This job's status is ${{ job.status }}."
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "hpc-stack-mod"]
path = hpc-stack-mod
url = https://github.com/NOAA-EMC/hpc-stack.git
35 changes: 35 additions & 0 deletions .readthedocs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# .readthedocs.yaml
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

# Required
version: 2

# Set the version of Python and other tools you might need
build:
os: ubuntu-20.04
tools:
python: "3.9"
# You can also specify other tool versions:
# nodejs: "16"
# rust: "1.55"
# golang: "1.17"

# Build documentation in the docs/ directory with Sphinx
sphinx:
configuration: docs/UsersGuide/source/conf.py

# If using Sphinx, optionally build your docs in additional formats such as PDF
# formats:
# - pdf

# Optionally declare the Python requirements required to build your docs
python:
install:
- requirements: docs/UsersGuide/requirements.txt

submodules:
include:
- hpc-stack-mod
recursive: true

6 changes: 3 additions & 3 deletions Externals.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ protocol = git
repo_url = https://github.com/ufs-community/UFS_UTILS
# Specify either a branch name or a hash but not both.
#branch = develop
hash = f30740e
hash = 31271f7
local_path = src/UFS_UTILS
required = True

Expand All @@ -20,7 +20,7 @@ protocol = git
repo_url = https://github.com/ufs-community/ufs-weather-model
# Specify either a branch name or a hash but not both.
#branch = develop
hash = e593349
hash = 96dffa1
local_path = src/ufs-weather-model
required = True

Expand All @@ -29,7 +29,7 @@ protocol = git
repo_url = https://github.com/NOAA-EMC/UPP
# Specify either a branch name or a hash but not both.
#branch = develop
hash = 4a16052
hash = 394917e
local_path = src/UPP
required = True

Expand Down
7 changes: 3 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,11 @@

The Unified Forecast System (UFS) is a community-based, coupled, comprehensive Earth modeling system. It is designed to be the source system for NOAA’s operational numerical weather prediction applications while enabling research, development, and contribution opportunities for the broader weather enterprise. For more information about the UFS, visit the UFS Portal at https://ufscommunity.org/.

The UFS can be configured for multiple applications (see a complete list at https://ufscommunity.org/#/science/aboutapps). The configuration described here is the UFS Short-Range Weather (SRW) Application, which targets predictions of atmospheric behavior on a limited spatial domain and on time scales from less than an hour out to several days. The development branch of the application is continually evolving as the system undergoes open development. The SRW App v1.0.0 represents a snapshot of this continuously evolving system. The SRW App includes a prognostic atmospheric model, pre- and post-processing, and a community workflow for running the system end-to-end.
The UFS includes multiple applications (see a complete list at https://ufscommunity.org/science/aboutapps/) that support different forecast durations and spatial domains. This documentation describes the UFS Short-Range Weather (SRW) Application, which targets predictions of atmospheric behavior on a limited spatial domain and on time scales from minutes to several days. The development branch of the application is continually evolving as the system undergoes open development. The SRW App release branches represent a snapshot of this continuously evolving system. The SRW Application includes a prognostic atmospheric model, pre- and post-processing, and a community workflow for running the system end-to-end. These components are documented within the User's Guide and supported through a community forum (https://forums.ufscommunity.org/).

The UFS SRW App User's Guide associated with the development branch is at: https://ufs-srweather-app.readthedocs.io/en/latest/, while that specific to the SRW App v1.0.0 release can be found at: https://ufs-srweather-app.readthedocs.io/en/ufs-v1.0.0/. The repository is at: https://github.com/ufs-community/ufs-srweather-app.
The UFS SRW App User's Guide associated with the development branch can be found at: https://ufs-srweather-app.readthedocs.io/en/develop/, while the guide specific to the SRW App v1.0.1 release can be found at: https://ufs-srweather-app.readthedocs.io/en/ufs-v1.0.1/. The GitHub repository link is: https://github.com/ufs-community/ufs-srweather-app.

For instructions on how to clone the repository, build the code, and run the workflow, see:
https://github.com/ufs-community/ufs-srweather-app/wiki/Getting-Started

UFS Development Team. (2021, March 4). Unified Forecast System (UFS) Short-Range Weather (SRW) Application (Version v1.0.0). Zenodo. https://doi.org/10.5281/zenodo.4534994

UFS Development Team. (2021, March 4). Unified Forecast System (UFS) Short-Range Weather (SRW) Application (Version v1.0.0). Zenodo. https://doi.org/10.5281/zenodo.4534994
Loading

0 comments on commit 3915859

Please sign in to comment.