Skip to content

Automated Setup, Build, and Run Instructions

Jump to bottom
G. Dylan Dickerson edited this page Apr 24, 2024 · 5 revisions

For simplicity, we have written a few scripts to automate the setup, build, and run of all five supported compsets in Earthworks on Cheyenne and Derecho: Earthworks scripts repo

To obtain these scripts run:

git clone [email protected]:NCAR/Earthworks_scripts.git

Script Usage

These scripts are intended mostly for batch building/submitting multiple versions of the same compset on the same machine with various resolutions and compilers with one call of the appropriate script.

Example basic usage:

./<script_name> --srcroot <path_to_earthworks_src_dir> --casedir <path_to_create_case_in>

Use ./<script_name> --help to see other options the scripts support.

Changing resolution, compiler, and tasks

You can control these either with arguments when calling the script or by editing the script directly. Note that arguments take precedence over code edits.

These scripts will create a case for each compiler, resolution, and task combination requested. See the end of this section for an example.

With arguments to the *_CBR.sh

  • The --res= argument is used to decide which resolution(s) to create, build, and run case(s) for. It accepts a bash array of valid resolutions. Valid resolutions for this release are: 120, 60, 30, 15
  • The --compiler= argument is used to determine which compiler(s) to create, build, and run case(s) for. It also accepts a bash array. The values allowed for this depends on your machine definition.
  • The --ntasks= argument holds determines the number of tasks to use when running case(s). It accepts a bash array of integers. There is no checking if these values are appropriate by the script. Jobs submitted with too few tasks may not start or behave as expected. Jobs submitted with too many tasks could be wasteful.

By editing the script

  • The RESS variable near line 21 of a CBR script controls the resolution(s) the case(s) will be created for. You can edit it to something like RESS=("120" "60" "30" "15") to create, build, and submit for the 4 supported resolutions.
  • The C_SUITES variable near line 19 of a CBR script controls the compiler(s) the case(s) will be compiled with. You can edit it to something like C_SUITES=("nvhpc" "gnu") to run the script with multiple compilers. Compilers available for this command depends on your machine definition.
  • Additionally, NTASKSS="" near line 23 can be modified with multiple values so each resolution/compiler requested with multiple tasks (MPI ranks).

Example use and beginning output with unrealistic task requests (don't use these tasks settings as any basis for your own testing):

$ ./F2000climo_derecho_CBR.sh --srcroot ~/EarthWorks --casesdir ~/cases --res=(120 15) --compiler=(nvhpc gnu) --ntasks=(128 10752)
                                                                                         
Submitting EarthWorks jobs to test F2000climo compset on ${HOSTNAME} for ${USER}                                                                                                
Starting at Wed Nov  8 18:14:31 MST 2023                                                                                                                                     
Using:                                                                                                                                                                       
        EarthWorks Repo at                   "~/EarthWorks"                                                  
        Creating cases in                    "~/cases"
        Case bld and run directories will be "/glade/derecho/scratch/${USER}/${CASENAME}"
        Running with compilers               "nvhpc,gnu"
        On MPAS-A grids (km)                 "120,15"
        CREATE=true     BUILD=true     RUN=true

CASENAMEs:  COMPSET.GRID.MACHINE.COMPILER.NTASKS
        F2000climo.mpasa120.derecho.nvhpc.128
        F2000climo.mpasa120.derecho.nvhpc.10752
        F2000climo.mpasa015.derecho.nvhpc.128
        F2000climo.mpasa015.derecho.nvhpc.10752
        F2000climo.mpasa120.derecho.gnu.128 
        F2000climo.mpasa120.derecho.gnu.10752
        F2000climo.mpasa015.derecho.gnu.128 
        F2000climo.mpasa015.derecho.gnu.10752
-------------------------------------------------------------------------

More would be outputted based on the build and submission as those steps occur.

Clone this wiki locally