Skip to content

Commit

Permalink
Unify CDEPS gfs, cfsr, and gefs datm datamodes + Improve error checki…
Browse files Browse the repository at this point in the history
…ng in rt.sh (2388) + Add ability to read increment files on native cubed sphere grid (2304) (#2389)

* UFSWM -  Unify CDEPS gfs, cfsr, and gefs datm datamodes. Add ability to read increment files on native cubed sphere grid.  Update rt scripts to fix checking the job exit status from Slurm and PBS schedulers.
  * FV3 - Add namelist parameter for cubed sphere increment read and update atmos_cubed_sphere hash for GFDL 
    Atmospheric Cubed Sphere PR #[342](NOAA-GFDL/GFDL_atmos_cubed_sphere#342)
     * atmos_cubed_sphere - Add ability to read increments from files on native cubed sphere grid
  * CDEPS - Unify CDEPS gfs, cfsr, and gefs datm datamodes. Rename CDEPS-related FILENAME_BASE to FILEBASE_DATM in 
     tests. Sunset gfs_hafs datamode.
  • Loading branch information
NickSzapiro-NOAA committed Aug 27, 2024
1 parent fcc9f84 commit 94a3cd7
Show file tree
Hide file tree
Showing 64 changed files with 2,593 additions and 2,922 deletions.
3 changes: 0 additions & 3 deletions CDEPS-interface/cdeps_files.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -41,14 +41,11 @@ list(APPEND cdeps_dshr_files

list(APPEND cdeps_datm_files
CDEPS/datm/atm_comp_nuopc.F90
CDEPS/datm/datm_datamode_cfsr_mod.F90
CDEPS/datm/datm_datamode_clmncep_mod.F90
CDEPS/datm/datm_datamode_core2_mod.F90
CDEPS/datm/datm_datamode_cplhist_mod.F90
CDEPS/datm/datm_datamode_era5_mod.F90
CDEPS/datm/datm_datamode_gefs_mod.F90
CDEPS/datm/datm_datamode_gfs_mod.F90
CDEPS/datm/datm_datamode_gfs_hafs_mod.F90
CDEPS/datm/datm_datamode_jra_mod.F90
CDEPS/datm/datm_datamode_simple_mod.F90
)
Expand Down
2 changes: 1 addition & 1 deletion FV3
17 changes: 5 additions & 12 deletions build.sh
Original file line number Diff line number Diff line change
@@ -1,17 +1,10 @@
#!/bin/bash
set -eu
uname_s=$(uname -s)
if [[ ${uname_s} == Darwin ]]; then
UFS_MODEL_DIR=$(greadlink -f -n "${BASH_SOURCE[0]}")
UFS_MODEL_DIR=$(dirname "${UFS_MODEL_DIR}")
UFS_MODEL_DIR=$(cd "${UFS_MODEL_DIR}" && pwd -P)
else
UFS_MODEL_DIR=$(readlink -f -n "${BASH_SOURCE[0]}")
UFS_MODEL_DIR=$(dirname "${UFS_MODEL_DIR}")
UFS_MODEL_DIR=$(cd "${UFS_MODEL_DIR}" && pwd -P)
fi
echo "UFS MODEL DIR: ${UFS_MODEL_DIR}"

SCRIPT_REALPATH=$(realpath "${BASH_SOURCE[0]}")
UFS_MODEL_DIR=$(dirname "${SCRIPT_REALPATH}")
readonly UFS_MODEL_DIR
echo "UFS MODEL DIR: ${UFS_MODEL_DIR}"

export CC=${CC:-mpicc}
export CXX=${CXX:-mpicxx}
Expand All @@ -26,4 +19,4 @@ for i in ${CMAKE_FLAGS}; do ARR_CMAKE_FLAGS+=("${i}") ; done
cmake "${UFS_MODEL_DIR}" "${ARR_CMAKE_FLAGS[@]}"
# Turn off OpenMP threading for parallel builds
# to avoid exhausting the number of user processes
OMP_NUM_THREADS=1 make -j "${BUILD_JOBS:-4}" "VERBOSE=${BUILD_VERBOSE:-}"
OMP_NUM_THREADS=1 make -j "${BUILD_JOBS:-4}" "VERBOSE=${BUILD_VERBOSE:-}"
8 changes: 0 additions & 8 deletions tests/ci/Jenkinsfile.combined
Original file line number Diff line number Diff line change
Expand Up @@ -312,14 +312,6 @@ def generateStage(nodeLabel) {
echo "Testing concluded...removing labels for $machine from $GIT_URL"
git remote -v | grep -w sshorigin > /dev/null 2>&1 && git remote remove sshorigin > /dev/null 2>&1
git remote add sshorigin $SSH_ORIGIN > /dev/null 2>&1

if [[ -f $WORKSPACE/tests/logs/RT-run-$machine.log ]] ; then
git add $WORKSPACE/tests/logs/RT-run-$machine.log
fi
git commit --allow-empty -m "[AutoRT] $machine Job Failed!\n\n\n on-behalf-of @ufs-community <[email protected]>"

git pull sshorigin $FORK_BRANCH
git push sshorigin HEAD:$FORK_BRANCH

tar --create --gzip --verbose --dereference --file "${machine_name_logs}.tgz" ${WORKSPACE}/tests/logs/*.log

Expand Down
10 changes: 2 additions & 8 deletions tests/compile.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,8 @@ function trim {

SECONDS=0

uname_s=$(uname -s)
if [[ ${uname_s} == Darwin ]]; then
greadlnk=$(greadlink -f -n "${BASH_SOURCE[0]}" )
MYDIR=$(cd "$(dirname "${greadlnk}" )" && pwd -P)
else
readlnk=$(readlink -f -n "${BASH_SOURCE[0]}" )
MYDIR=$(cd "$(dirname "${readlnk}" )" && pwd -P)
fi
SCRIPT_REALPATH=$(realpath "${BASH_SOURCE[0]}")
MYDIR=$(dirname "${SCRIPT_REALPATH}")
readonly MYDIR

# ----------------------------------------------------------------------
Expand Down
9 changes: 5 additions & 4 deletions tests/default_vars.sh
Original file line number Diff line number Diff line change
Expand Up @@ -559,6 +559,7 @@ export MODEL_INITIALIZATION=false
export WARM_START=.false.
export READ_INCREMENT=.false.
export RES_LATLON_DYNAMICS="''"
export INCREMENT_FILE_ON_NATIVE_GRID=.false.
export NGGPS_IC=.true.
export EXTERNAL_IC=.true.
export MAKE_NH=.true.
Expand Down Expand Up @@ -1332,10 +1333,10 @@ export_datm_cdeps ()
export INPUT_NML=input.mom6.nml.IN
export DIAG_TABLE=diag_table_template
export DATM_SRC=CFSR
export FILENAME_BASE=cfsr.
export MESH_ATM=${FILENAME_BASE//.}_mesh.nc
export atm_datamode=${DATM_SRC}
export stream_files=INPUT/${FILENAME_BASE}201110.nc
export FILEBASE_DATM=cfsr
export MESH_ATM=${FILEBASE_DATM}_mesh.nc
export atm_datamode=GEFS
export stream_files=INPUT/${FILEBASE_DATM}.201110.nc
export EXPORT_ALL=.false.
export STREAM_OFFSET=0

Expand Down
3 changes: 3 additions & 0 deletions tests/error-test.conf
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ COMPILE | atm_dyn32 | intel | -DAPP=ATM -DCCPP_SUITES=FV3_GFS_v16,FV3_GFS_v16_fl
# This should succeed
RUN | control_c48.v2.sfc | | baseline |

# This should fail due to wall clock timeout
RUN | control_c48.v2.sfc_timeout | | baseline |

# These tests should always fail, and prevent the workflow from completing.
RUN | fail_to_copy | | baseline |
RUN | fail_to_run | | baseline |
Expand Down
48 changes: 24 additions & 24 deletions tests/logs/OpnReqTests_control_p8_hera.log
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
Wed Aug 21 22:05:02 UTC 2024
Mon Aug 26 19:59:54 UTC 2024
Start Operation Requirement Test


baseline dir = /scratch1/NCEPDEV/stmp4/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_bit_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_3986824/bit_base_bit_base
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_255239/bit_base_bit_base
Checking test bit_base results ....
Moving baseline bit_base files ....
Moving sfcf000.nc .........OK
Expand Down Expand Up @@ -51,14 +51,14 @@ Moving baseline bit_base files ....
Moving RESTART/20210323.060000.sfc_data.tile5.nc .........OK
Moving RESTART/20210323.060000.sfc_data.tile6.nc .........OK

0: The total amount of wall time = 280.831970
0: The maximum resident set size (KB) = 1448916
0: The total amount of wall time = 282.094574
0: The maximum resident set size (KB) = 1459728

Test bit_base PASS


baseline dir = /scratch1/NCEPDEV/stmp4/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_dbg_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_3986824/dbg_base_dbg_base
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_255239/dbg_base_dbg_base
Checking test dbg_base results ....
Moving baseline dbg_base files ....
Moving sfcf000.nc .........OK
Expand Down Expand Up @@ -106,14 +106,14 @@ Moving baseline dbg_base files ....
Moving RESTART/20210323.060000.sfc_data.tile5.nc .........OK
Moving RESTART/20210323.060000.sfc_data.tile6.nc .........OK

0: The total amount of wall time = 992.336985
0: The maximum resident set size (KB) = 1437772
0: The total amount of wall time = 985.479617
0: The maximum resident set size (KB) = 1417884

Test dbg_base PASS


baseline dir = /scratch1/NCEPDEV/stmp4/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_3986824/dcp_dcp
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_255239/dcp_dcp
Checking test dcp results ....
Comparing sfcf000.nc .....USING NCCMP......OK
Comparing sfcf021.nc .....USING NCCMP......OK
Expand Down Expand Up @@ -160,14 +160,14 @@ Checking test dcp results ....
Comparing RESTART/20210323.060000.sfc_data.tile5.nc .....USING NCCMP......OK
Comparing RESTART/20210323.060000.sfc_data.tile6.nc .....USING NCCMP......OK

0: The total amount of wall time = 255.618968
0: The maximum resident set size (KB) = 1425456
0: The total amount of wall time = 248.873990
0: The maximum resident set size (KB) = 1437132

Test dcp PASS


baseline dir = /scratch1/NCEPDEV/stmp4/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_3986824/mpi_mpi
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_255239/mpi_mpi
Checking test mpi results ....
Comparing sfcf000.nc .....USING NCCMP......OK
Comparing sfcf021.nc .....USING NCCMP......OK
Expand Down Expand Up @@ -214,14 +214,14 @@ Checking test mpi results ....
Comparing RESTART/20210323.060000.sfc_data.tile5.nc .....USING NCCMP......OK
Comparing RESTART/20210323.060000.sfc_data.tile6.nc .....USING NCCMP......OK

0: The total amount of wall time = 250.392410
0: The maximum resident set size (KB) = 1437096
0: The total amount of wall time = 252.449329
0: The maximum resident set size (KB) = 1422488

Test mpi PASS


baseline dir = /scratch1/NCEPDEV/stmp4/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_3986824/rst_rst
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_255239/rst_rst
Checking test rst results ....
Comparing sfcf000.nc .....USING NCCMP......OK
Comparing sfcf021.nc .....USING NCCMP......OK
Expand Down Expand Up @@ -268,14 +268,14 @@ Checking test rst results ....
Comparing RESTART/20210323.060000.sfc_data.tile5.nc .....USING NCCMP......OK
Comparing RESTART/20210323.060000.sfc_data.tile6.nc .....USING NCCMP......OK

0: The total amount of wall time = 257.380091
0: The maximum resident set size (KB) = 1423788
0: The total amount of wall time = 252.763505
0: The maximum resident set size (KB) = 1423640

Test rst PASS


baseline dir = /scratch1/NCEPDEV/stmp4/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_3986824/std_base_std_base
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_255239/std_base_std_base
Checking test std_base results ....
Moving baseline std_base files ....
Moving sfcf000.nc .........OK
Expand Down Expand Up @@ -323,14 +323,14 @@ Moving baseline std_base files ....
Moving RESTART/20210323.060000.sfc_data.tile5.nc .........OK
Moving RESTART/20210323.060000.sfc_data.tile6.nc .........OK

0: The total amount of wall time = 267.250624
0: The maximum resident set size (KB) = 1421992
0: The total amount of wall time = 253.485325
0: The maximum resident set size (KB) = 1438440

Test std_base PASS


baseline dir = /scratch1/NCEPDEV/stmp4/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/control_p8_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_3986824/thr_thr
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_255239/thr_thr
Checking test thr results ....
Comparing sfcf000.nc .....USING NCCMP......OK
Comparing sfcf021.nc .....USING NCCMP......OK
Expand Down Expand Up @@ -377,11 +377,11 @@ Checking test thr results ....
Comparing RESTART/20210323.060000.sfc_data.tile5.nc .....USING NCCMP......OK
Comparing RESTART/20210323.060000.sfc_data.tile6.nc .....USING NCCMP......OK

0: The total amount of wall time = 257.163548
0: The maximum resident set size (KB) = 1434436
0: The total amount of wall time = 247.766378
0: The maximum resident set size (KB) = 1411920

Test thr PASS

OPERATION REQUIREMENT TEST WAS SUCCESSFUL
Wed Aug 21 23:13:54 UTC 2024
Elapsed time: 01h:08m:53s. Have a nice day!
Mon Aug 26 21:14:01 UTC 2024
Elapsed time: 01h:14m:07s. Have a nice day!
24 changes: 12 additions & 12 deletions tests/logs/OpnReqTests_cpld_control_nowave_noaero_p8_hera.log
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
Wed Aug 21 20:56:23 UTC 2024
Mon Aug 26 18:51:21 UTC 2024
Start Operation Requirement Test


baseline dir = /scratch1/NCEPDEV/stmp4/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/cpld_control_c96_noaero_p8_dbg_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_3387682/dbg_base_dbg_base
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_2593550/dbg_base_dbg_base
Checking test dbg_base results ....
Moving baseline dbg_base files ....
Moving sfcf021.tile1.nc .........OK
Expand Down Expand Up @@ -66,14 +66,14 @@ Moving baseline dbg_base files ....
Moving RESTART/iced.2021-03-23-21600.nc .........OK
Moving RESTART/ufs.cpld.cpl.r.2021-03-23-21600.nc .........OK

0: The total amount of wall time = 1337.925051
0: The maximum resident set size (KB) = 1527960
0: The total amount of wall time = 1361.846982
0: The maximum resident set size (KB) = 1537900

Test dbg_base PASS


baseline dir = /scratch1/NCEPDEV/stmp4/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/cpld_control_c96_noaero_p8_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_3387682/rst_rst
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_2593550/rst_rst
Checking test rst results ....
Comparing sfcf021.tile1.nc .....USING NCCMP......OK
Comparing sfcf021.tile2.nc .....USING NCCMP......OK
Expand Down Expand Up @@ -135,14 +135,14 @@ Checking test rst results ....
Comparing RESTART/iced.2021-03-23-21600.nc .....USING NCCMP......OK
Comparing RESTART/ufs.cpld.cpl.r.2021-03-23-21600.nc .....USING NCCMP......OK

0: The total amount of wall time = 429.025844
0: The maximum resident set size (KB) = 1528532
0: The total amount of wall time = 715.515398
0: The maximum resident set size (KB) = 1525500

Test rst PASS


baseline dir = /scratch1/NCEPDEV/stmp4/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/cpld_control_c96_noaero_p8_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_3387682/std_base_std_base
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_2593550/std_base_std_base
Checking test std_base results ....
Moving baseline std_base files ....
Moving sfcf021.tile1.nc .........OK
Expand Down Expand Up @@ -205,11 +205,11 @@ Moving baseline std_base files ....
Moving RESTART/iced.2021-03-23-21600.nc .........OK
Moving RESTART/ufs.cpld.cpl.r.2021-03-23-21600.nc .........OK

0: The total amount of wall time = 441.109791
0: The maximum resident set size (KB) = 1516924
0: The total amount of wall time = 435.784809
0: The maximum resident set size (KB) = 1528568

Test std_base PASS

OPERATION REQUIREMENT TEST WAS SUCCESSFUL
Wed Aug 21 22:05:01 UTC 2024
Elapsed time: 01h:08m:39s. Have a nice day!
Mon Aug 26 19:59:54 UTC 2024
Elapsed time: 01h:08m:34s. Have a nice day!
30 changes: 15 additions & 15 deletions tests/logs/OpnReqTests_regional_control_hera.log
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
Wed Aug 21 18:19:56 UTC 2024
Mon Aug 26 16:17:09 UTC 2024
Start Operation Requirement Test


baseline dir = /scratch1/NCEPDEV/stmp4/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/regional_control_bit_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_65311/bit_base_bit_base
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_3681501/bit_base_bit_base
Checking test bit_base results ....
Moving baseline bit_base files ....
Moving dynf000.nc .........OK
Expand All @@ -15,14 +15,14 @@ Moving baseline bit_base files ....
Moving NATLEV.GrbF00 .........OK
Moving NATLEV.GrbF06 .........OK

0: The total amount of wall time = 1391.521656
0: The maximum resident set size (KB) = 777236
0: The total amount of wall time = 1404.274425
0: The maximum resident set size (KB) = 769724

Test bit_base PASS


baseline dir = /scratch1/NCEPDEV/stmp4/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/regional_control_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_65311/dcp_dcp
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_3681501/dcp_dcp
Checking test dcp results ....
Comparing dynf000.nc .....USING NCCMP......OK
Comparing dynf006.nc .....USING NCCMP......OK
Expand All @@ -33,14 +33,14 @@ Checking test dcp results ....
Comparing NATLEV.GrbF00 .....USING CMP......OK
Comparing NATLEV.GrbF06 .....USING CMP......OK

0: The total amount of wall time = 2169.925001
0: The maximum resident set size (KB) = 743380
0: The total amount of wall time = 2178.286214
0: The maximum resident set size (KB) = 751032

Test dcp PASS


baseline dir = /scratch1/NCEPDEV/stmp4/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/regional_control_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_65311/std_base_std_base
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_3681501/std_base_std_base
Checking test std_base results ....
Moving baseline std_base files ....
Moving dynf000.nc .........OK
Expand All @@ -52,14 +52,14 @@ Moving baseline std_base files ....
Moving NATLEV.GrbF00 .........OK
Moving NATLEV.GrbF06 .........OK

0: The total amount of wall time = 2169.154728
0: The maximum resident set size (KB) = 733176
0: The total amount of wall time = 2198.737097
0: The maximum resident set size (KB) = 734276

Test std_base PASS


baseline dir = /scratch1/NCEPDEV/stmp4/role.epic/FV3_OPNREQ_TEST/OPNREQ_TEST/regional_control_std_base_gnu
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_65311/thr_thr
working dir = /scratch1/NCEPDEV/stmp2/role.epic/FV3_OPNREQ_TEST/opnReqTest_3681501/thr_thr
Checking test thr results ....
Comparing dynf000.nc .....USING NCCMP......OK
Comparing dynf006.nc .....USING NCCMP......OK
Expand All @@ -70,11 +70,11 @@ Checking test thr results ....
Comparing NATLEV.GrbF00 .....USING CMP......OK
Comparing NATLEV.GrbF06 .....USING CMP......OK

0: The total amount of wall time = 2203.506465
0: The maximum resident set size (KB) = 733136
0: The total amount of wall time = 2321.006507
0: The maximum resident set size (KB) = 742260

Test thr PASS

OPERATION REQUIREMENT TEST WAS SUCCESSFUL
Wed Aug 21 20:56:22 UTC 2024
Elapsed time: 02h:36m:27s. Have a nice day!
Mon Aug 26 18:51:20 UTC 2024
Elapsed time: 02h:34m:14s. Have a nice day!
Loading

0 comments on commit 94a3cd7

Please sign in to comment.