Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[develop] Change conda env for AQM on Hera to shared one and fix input data issue on WCOSS2 #844

Merged
merged 9 commits into from
Jul 5, 2023
Merged

[develop] Change conda env for AQM on Hera to shared one and fix input data issue on WCOSS2 #844

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
d3be7eb
replace python env with shared loc
chan-hoo Jun 22, 2023
a1cc776
remove unnecessary env
chan-hoo Jun 22, 2023
c5fa090
Merge branch 'develop' of https://github.com/chan-hoo/ufs-srweather-a…
chan-hoo Jun 22, 2023
f4d9560
add test_aqm_input_basedir
chan-hoo Jun 24, 2023
0b22f4c
adjust line space
chan-hoo Jun 24, 2023
cdcebcb
update aqm-utils
chan-hoo Jun 24, 2023
ca7d96b
update conda env for orion
chan-hoo Jun 26, 2023
aea11e0
Merge branch 'develop' of https://github.com/chan-hoo/ufs-srweather-a…
chan-hoo Jun 30, 2023
8638e02
update aqm_fire_dir with the latest change
chan-hoo Jun 30, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Externals.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ protocol = git
repo_url = https://github.com/NOAA-EMC/AQM-utils
# Specify either a branch name or a hash but not both.
#branch = develop
hash = 0a86f73
hash = 17c7957
local_path = sorc/AQM-utils
required = True

Expand Down
2 changes: 1 addition & 1 deletion modulefiles/tasks/hera/aqm_ics.local.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
load("miniconda_online-cmaq")
load("miniconda_regional_workflow_cmaq")
load(pathJoin("nco", os.getenv("nco_ver") or "4.9.3"))
2 changes: 1 addition & 1 deletion modulefiles/tasks/hera/fire_emission.local.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
load("hpss")
load("miniconda_online-cmaq")
load("miniconda_regional_workflow_cmaq")
load(pathJoin("nco", os.getenv("nco_ver") or "4.9.3"))
5 changes: 0 additions & 5 deletions modulefiles/tasks/hera/miniconda_online-cmaq.lua
View file
Open in desktop

This file was deleted.

4 changes: 4 additions & 0 deletions modulefiles/tasks/hera/miniconda_regional_workflow_cmaq.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
prepend_path("MODULEPATH","/scratch1/NCEPDEV/nems/role.epic/miniconda3/modulefiles")
load(pathJoin("miniconda3", os.getenv("miniconda3_ver") or "4.12.0"))

setenv("SRW_ENV", "regional_workflow_cmaq")
2 changes: 1 addition & 1 deletion modulefiles/tasks/hera/nexus_emission.local.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
load(pathJoin("nco", os.getenv("nco_ver") or "4.9.3"))
load("miniconda_online-cmaq")
load("miniconda_regional_workflow_cmaq")
2 changes: 1 addition & 1 deletion modulefiles/tasks/hera/nexus_post_split.local.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
load("miniconda_online-cmaq")
load("miniconda_regional_workflow_cmaq")
load(pathJoin("nco", os.getenv("nco_ver") or "4.9.3"))
2 changes: 1 addition & 1 deletion modulefiles/tasks/hera/point_source.local.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
load("miniconda_online-cmaq")
load("miniconda_regional_workflow_cmaq")
5 changes: 4 additions & 1 deletion tests/WE2E/run_WE2E_tests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,6 @@ def run_we2e_tests(homedir, args) -> None:
if args.verbose_tests:
test_cfg['workflow'].update({"VERBOSE": args.verbose_tests})


logging.debug(f"Overwriting WE2E-test-specific settings for test \n{test_name}\n")

if 'task_get_extrn_ics' in test_cfg:
Expand All @@ -191,6 +190,10 @@ def run_we2e_tests(homedir, args) -> None:
if 'verification' in test_cfg:
logging.debug(test_cfg['verification'])

if 'cpl_aqm_parm' in test_cfg:
test_aqm_input_basedir = machine_defaults['platform']['TEST_AQM_INPUT_BASEDIR']
test_cfg['cpl_aqm_parm']['AQM_FIRE_DIR'] = f"{test_aqm_input_basedir}/RAVE_fire"

logging.debug(f"Writing updated config.yaml for test {test_name}\n"\
"based on specified command-line arguments:\n")
logging.debug(cfg_to_yaml_str(test_cfg))
Expand Down
1 change: 1 addition & 0 deletions ush/config_defaults.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -372,6 +372,7 @@ platform:
#-----------------------------------------------------------------------
#
TEST_EXTRN_MDL_SOURCE_BASEDIR: ""
TEST_AQM_INPUT_BASEDIR: ""
TEST_PREGEN_BASEDIR: ""
TEST_ALT_EXTRN_MDL_SYSBASEDIR_ICS: ""
TEST_ALT_EXTRN_MDL_SYSBASEDIR_LBCS: ""
Expand Down
6 changes: 0 additions & 6 deletions ush/load_modules_run_task.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,12 +192,6 @@ if [ -n "${SRW_ENV:-}" ] ; then
set -u
fi

if [ -n "${AQM_ENV:-}" ] ; then
set +u
source "${AQM_ENV_FP}/${AQM_ENV}/bin/activate"
set -u
fi

#
#-----------------------------------------------------------------------
#
Expand Down
1 change: 1 addition & 0 deletions ush/machine/cheyenne.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ platform:
RUN_CMD_AQMLBC: mpirun -np ${NUMTS}
PRE_TASK_CMDS: '{ ulimit -s unlimited; ulimit -a; }'
TEST_EXTRN_MDL_SOURCE_BASEDIR: /glade/work/epicufsrt/contrib/UFS_SRW_data/develop/input_model_data
TEST_AQM_INPUT_BASEDIR: /glade/work/epicufsrt/contrib/UFS_SRW_data/develop/aqm_data
TEST_PREGEN_BASEDIR: /glade/work/epicufsrt/contrib/UFS_SRW_data/develop/FV3LAM_pregen
TEST_ALT_EXTRN_MDL_SYSBASEDIR_ICS: /glade/work/epicufsrt/contrib/UFS_SRW_data/develop/dummy_FV3GFS_sys_dir
TEST_ALT_EXTRN_MDL_SYSBASEDIR_LBCS: /glade/work/epicufsrt/contrib/UFS_SRW_data/develop/dummy_FV3GFS_sys_dir
Expand Down
1 change: 1 addition & 0 deletions ush/machine/hera.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ platform:
SCHED_NATIVE_CMD: --export=NONE
PRE_TASK_CMDS: '{ ulimit -s unlimited; ulimit -a; }'
TEST_EXTRN_MDL_SOURCE_BASEDIR: /scratch1/NCEPDEV/nems/role.epic/UFS_SRW_data/develop/input_model_data
TEST_AQM_INPUT_BASEDIR: /scratch1/NCEPDEV/nems/role.epic/UFS_SRW_data/develop/aqm_data
TEST_PREGEN_BASEDIR: /scratch1/NCEPDEV/nems/role.epic/UFS_SRW_data/develop/FV3LAM_pregen
TEST_ALT_EXTRN_MDL_SYSBASEDIR_ICS: /scratch1/NCEPDEV/nems/role.epic/UFS_SRW_data/develop/dummy_FV3GFS_sys_dir
TEST_ALT_EXTRN_MDL_SYSBASEDIR_LBCS: /scratch1/NCEPDEV/nems/role.epic/UFS_SRW_data/develop/dummy_FV3GFS_sys_dir
Expand Down
1 change: 1 addition & 0 deletions ush/machine/orion.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ platform:
SCHED_NATIVE_CMD: --export=NONE
PRE_TASK_CMDS: '{ ulimit -s unlimited; ulimit -a; }'
TEST_EXTRN_MDL_SOURCE_BASEDIR: /work/noaa/epic-ps/role-epic-ps/UFS_SRW_data/develop/input_model_data
TEST_AQM_INPUT_BASEDIR: /work/noaa/epic-ps/role-epic-ps/UFS_SRW_data/develop/aqm_data
TEST_PREGEN_BASEDIR: /work/noaa/epic-ps/role-epic-ps/UFS_SRW_data/develop/FV3LAM_pregen
FIXaer: /work/noaa/epic-ps/role-epic-ps/UFS_SRW_data/develop/fix/fix_aer
FIXgsi: /work/noaa/epic-ps/role-epic-ps/UFS_SRW_data/develop/fix/fix_gsi
Expand Down
1 change: 1 addition & 0 deletions ush/machine/wcoss2.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ platform:
SCHED_NATIVE_CMD: -l place=excl
PRE_TASK_CMDS: '{ ulimit -s unlimited; ulimit -a; }'
TEST_EXTRN_MDL_SOURCE_BASEDIR: /lfs/h2/emc/lam/noscrub/UFS_SRW_App/develop/input_model_data
TEST_AQM_INPUT_BASEDIR: /lfs/h2/emc/lam/noscrub/UFS_SRW_App/develop/aqm_data
TEST_PREGEN_BASEDIR: /lfs/h2/emc/lam/noscrub/UFS_SRW_App/develop/FV3LAM_pregen
FIXaer: /lfs/h2/emc/lam/noscrub/UFS_SRW_App/develop/fix/fix_aer
FIXgsm: /lfs/h2/emc/lam/noscrub/UFS_SRW_App/develop/fix/fix_am
Expand Down