Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update for JCB policies and stage DA job files with Jinja2-templates #2700

Merged
merged 91 commits into from
Jul 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
b5eb345
Use rename_atm of jcb parm erpos
danholdaway May 29, 2024
3a8f527
Initial commit
DavidNew-NOAA May 31, 2024
6904982
Update atm_berror_bump_fix.yaml.j2
DavidNew-NOAA Jun 3, 2024
0b37fe5
Update atm_berror_gsibec_fix.yaml.j2
DavidNew-NOAA Jun 3, 2024
d770b63
Update atm_berror_identity_fix.yaml.j2
DavidNew-NOAA Jun 3, 2024
0187bba
Update atm_bkg_fix.yaml.j2
DavidNew-NOAA Jun 3, 2024
4dcd47a
Rename some things
DavidNew-NOAA Jun 3, 2024
e61963d
Merge branch 'feature/stage_from_yaml' of https://github.com/DavidNew…
DavidNew-NOAA Jun 3, 2024
f91297d
Rename some things
DavidNew-NOAA Jun 3, 2024
983d720
Add changes to analysis python scripts
DavidNew-NOAA Jun 3, 2024
9436f4b
update hashes and config
danholdaway Jun 4, 2024
97a63a7
Merge remote-tracking branch 'upsteam/develop' into feature/rename_atm
danholdaway Jun 4, 2024
6e44a5f
Merge branch 'develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 4, 2024
2490049
Merge branch 'develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 4, 2024
3022393
Minor fix
DavidNew-NOAA Jun 4, 2024
cc1a3b6
Minor update
DavidNew-NOAA Jun 4, 2024
2097116
Minor update
DavidNew-NOAA Jun 4, 2024
5327711
Coding norms
DavidNew-NOAA Jun 4, 2024
2702ef0
Remove jcb from GW submodules
danholdaway Jun 4, 2024
b6456be
Remove JCB
danholdaway Jun 4, 2024
2a5f301
adjust link to point to new jcb location in gdasapp
danholdaway Jun 4, 2024
9f425a6
Update GDASapp Hash
danholdaway Jun 4, 2024
4d36caf
Move some stuff around
DavidNew-NOAA Jun 4, 2024
55358e5
Merge remote-tracking branch 'upsteam/develop' into feature/rename_atm
danholdaway Jun 4, 2024
9e357f9
gdas hash
danholdaway Jun 4, 2024
ba2d12d
Add time loop for some staging
DavidNew-NOAA Jun 5, 2024
d0d4429
bump gdas
danholdaway Jun 6, 2024
ee1aa30
bump gdas
danholdaway Jun 6, 2024
2138df1
Merge branch 'NOAA-EMC:develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 6, 2024
09ec021
hashes
danholdaway Jun 6, 2024
a5db89e
update gdas hash
danholdaway Jun 6, 2024
b71d285
Merge remote-tracking branch 'upstream/develop' into feature/move_jcb
danholdaway Jun 6, 2024
1bafbe8
bring russ change in
danholdaway Jun 6, 2024
e52e0de
follow russ for GSI utils hash
danholdaway Jun 6, 2024
4d3150c
revert gsi utils
danholdaway Jun 7, 2024
9c4f77b
Merge branch 'develop' into feature/move_jcb
danholdaway Jun 7, 2024
f70b0ad
Remove all references to mytask.config and mytask.runtime_config
DavidNew-NOAA Jun 7, 2024
3f8ba42
Merge branch 'develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 7, 2024
49f8593
Update wxflow hash
DavidNew-NOAA Jun 7, 2024
2e56698
Merge branch 'develop' into feature/move_jcb
danholdaway Jun 11, 2024
b6213a5
Merge branch 'develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 11, 2024
2bd9949
Tab out Jinja2 templates for readability
DavidNew-NOAA Jun 11, 2024
46f81e5
Merge branch 'develop' into feature/move_jcb
danholdaway Jun 12, 2024
458aa34
Merge branch 'develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 12, 2024
eebeb4b
Merge branch 'develop' into feature/move_jcb
danholdaway Jun 12, 2024
488a3de
update gdas hash
danholdaway Jun 13, 2024
d9501c5
Merge branch 'feature/move_jcb' of https://github.com/danholdaway/glo…
danholdaway Jun 13, 2024
2cb6664
Use replace_tmpl filter
DavidNew-NOAA Jun 13, 2024
bd7f770
Update wxflow hash
DavidNew-NOAA Jun 13, 2024
b714cca
Merge remote-tracking branch 'upstream/develop' into feature/move_jcb
danholdaway Jun 13, 2024
4de3199
Merge branch 'develop' into feature/move_jcb
WalterKolczynski-NOAA Jun 13, 2024
f76bef4
update to latest gdas app
danholdaway Jun 13, 2024
23971d2
mod
danholdaway Jun 13, 2024
d5cb822
add back ufsutils
danholdaway Jun 13, 2024
f8a18be
Update wxflow hash and fix big in lgetkf staging file
DavidNew-NOAA Jun 14, 2024
4b21d8a
Merge branch 'develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 17, 2024
c6223c4
Merge branch 'develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 17, 2024
18c596c
Merge branch 'feature/stage_from_yaml' of https://github.com/DavidNew…
DavidNew-NOAA Jun 17, 2024
2c8feb5
Simplify Jinja2 templates a bit
DavidNew-NOAA Jun 17, 2024
2c5ef59
Replace CDUMP with RUN
DavidNew-NOAA Jun 17, 2024
dcfc40f
Fix bug
DavidNew-NOAA Jun 17, 2024
a3d0d06
Merge branch 'feature/rename_atm' of https://github.com/danholdaway/g…
RussTreadon-NOAA Jun 18, 2024
584762a
update gdas.cd hash to bring in updated jcb, jcb-gdas, and jcb-algori…
RussTreadon-NOAA Jun 18, 2024
ee4397a
Merge branch 'NOAA-EMC:develop' into feature/rename_atm
RussTreadon-NOAA Jun 19, 2024
39e719d
Merge branch 'NOAA-EMC:develop' into feature/rename_atm
RussTreadon-NOAA Jun 20, 2024
9d1c2ec
Merge branch 'develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 20, 2024
dd6613b
remove path specific logic to identify MSU machines (#2695)
RussTreadon-NOAA Jun 20, 2024
826c3c3
add ocnanalecen to HERCULES.env (#2699)
RussTreadon-NOAA Jun 20, 2024
c14607f
fix shellcheck error in detect_machine.sh (#2695)
RussTreadon-NOAA Jun 20, 2024
9973c70
fix pycodestyle error in hosts.py (#2695)
RussTreadon-NOAA Jun 20, 2024
ddcd376
Updates
DavidNew-NOAA Jun 20, 2024
7aa041e
Refactor aero_prepobs.py like other tasks
DavidNew-NOAA Jun 20, 2024
e175b25
Update wxflow hash
DavidNew-NOAA Jun 20, 2024
b00d31e
Merge remote-tracking branch 'upstream/feature/stage_from_yaml' into …
RussTreadon-NOAA Jun 21, 2024
7cc86d9
update gdas.cd to resolve wxflow issues and update jcb hashes (#2699)
RussTreadon-NOAA Jun 21, 2024
f053ea1
Merge branch 'develop' into feature/rename_atm
RussTreadon-NOAA Jun 21, 2024
8663123
add wxflow to PYTHONPATH for ufsda jobs; update prep_snow_obs script …
RussTreadon-NOAA Jun 22, 2024
076a59d
simply wxflow path for ufsda jobs (#2654)
RussTreadon-NOAA Jun 23, 2024
19f35e9
update gdas.cd hash to fix issues with aero and soca DA (#2700)
RussTreadon-NOAA Jun 23, 2024
3768b9d
do not set wxflowPATH in rocoto j-job drivers, add wxflowPATH to load…
RussTreadon-NOAA Jun 24, 2024
32a2002
update gdas.cd hash to simplify wxflowpath in GDASApp ctests (#2700)
RussTreadon-NOAA Jun 24, 2024
d1d88a6
Merge branch 'develop' into feature/rename_atm
aerorahul Jun 24, 2024
8fc02e2
add g-w python to PYTHONPATH on WCOSS2 in upp.sh (#2700)
RussTreadon-NOAA Jun 24, 2024
bc33c2d
Merge branch 'develop' into feature/rename_atm
RussTreadon-NOAA Jun 25, 2024
8da0821
Merge branch 'develop' into feature/rename_atm
aerorahul Jun 26, 2024
3e84035
Merge branch 'NOAA-EMC:develop' into feature/rename_atm
RussTreadon-NOAA Jun 26, 2024
d70afac
Merge branch 'develop' into feature/rename_atm
RussTreadon-NOAA Jun 26, 2024
72bf7d8
Merge branch 'develop' into feature/rename_atm
RussTreadon-NOAA Jun 27, 2024
c1ef4b3
comment out find lines for purge_every_days (#2719)
RussTreadon-NOAA Jun 28, 2024
f551018
Update scripts/exglobal_cleanup.sh
aerorahul Jun 28, 2024
dcec081
Merge branch 'develop' into feature/rename_atm
RussTreadon-NOAA Jun 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions env/HERCULES.env
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,16 @@ case ${step} in
[[ ${NTHREADS_OCNANAL} -gt ${nth_max} ]] && export NTHREADS_OCNANAL=${nth_max}
export APRUN_OCNANAL="${launcher} -n ${npe_ocnanalrun} --cpus-per-task=${NTHREADS_OCNANAL}"
;;
"ocnanalecen")

export APRUNCFP="${launcher} -n \$ncmd ${mpmd_opt}"

nth_max=$((npe_node_max / npe_node_ocnanalecen))

export NTHREADS_OCNANALECEN=${nth_ocnanalecen:-${nth_max}}
[[ ${NTHREADS_OCNANALECEN} -gt ${nth_max} ]] && export NTHREADS_OCNANALECEN=${nth_max}
export APRUN_OCNANALECEN="${launcher} -n ${npe_ocnanalecen} --cpus-per-task=${NTHREADS_OCNANALECEN}"
;;
"ocnanalchkpt")

export APRUNCFP="${launcher} -n \$ncmd ${mpmd_opt}"
Expand Down
5 changes: 0 additions & 5 deletions jobs/rocoto/aeroanlfinal.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,6 @@ status=$?
export job="aeroanlfinal"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow utilities and tasks
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${wxflowPATH}"
export PYTHONPATH
###############################################################
# Execute the JJOB
"${HOMEgfs}/jobs/JGLOBAL_AERO_ANALYSIS_FINALIZE"
Expand Down
6 changes: 0 additions & 6 deletions jobs/rocoto/aeroanlinit.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,6 @@ status=$?
export job="aeroanlinit"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow utilities and tasks
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${wxflowPATH}"
export PYTHONPATH

###############################################################
# Execute the JJOB
"${HOMEgfs}/jobs/JGLOBAL_AERO_ANALYSIS_INITIALIZE"
Expand Down
6 changes: 0 additions & 6 deletions jobs/rocoto/aeroanlrun.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,6 @@ status=$?
export job="aeroanlrun"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow utilities and tasks
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${wxflowPATH}"
export PYTHONPATH

###############################################################
# Execute the JJOB
"${HOMEgfs}/jobs/JGLOBAL_AERO_ANALYSIS_RUN"
Expand Down
5 changes: 0 additions & 5 deletions jobs/rocoto/atmanlfinal.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,6 @@ status=$?
export job="atmanlfinal"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow utilities and tasks
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${wxflowPATH}"
export PYTHONPATH
###############################################################
# Execute the JJOB
"${HOMEgfs}/jobs/JGLOBAL_ATM_ANALYSIS_FINALIZE"
Expand Down
6 changes: 0 additions & 6 deletions jobs/rocoto/atmanlfv3inc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,6 @@ status=$?
export job="atmanlfv3inc"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow utilities and tasks
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${wxflowPATH}"
export PYTHONPATH

###############################################################
# Execute the JJOB
"${HOMEgfs}/jobs/JGLOBAL_ATM_ANALYSIS_FV3_INCREMENT"
Expand Down
6 changes: 0 additions & 6 deletions jobs/rocoto/atmanlinit.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,6 @@ status=$?
export job="atmanlinit"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow utilities and tasks
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${wxflowPATH}"
export PYTHONPATH

###############################################################
# Execute the JJOB
"${HOMEgfs}/jobs/JGLOBAL_ATM_ANALYSIS_INITIALIZE"
Expand Down
6 changes: 0 additions & 6 deletions jobs/rocoto/atmanlvar.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,6 @@ status=$?
export job="atmanlvar"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow utilities and tasks
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${wxflowPATH}"
export PYTHONPATH

###############################################################
# Execute the JJOB
"${HOMEgfs}/jobs/JGLOBAL_ATM_ANALYSIS_VARIATIONAL"
Expand Down
5 changes: 0 additions & 5 deletions jobs/rocoto/atmensanlfinal.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,6 @@ status=$?
export job="atmensanlfinal"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow utilities and tasks
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${wxflowPATH}"
export PYTHONPATH
###############################################################
# Execute the JJOB
"${HOMEgfs}/jobs/JGLOBAL_ATMENS_ANALYSIS_FINALIZE"
Expand Down
6 changes: 0 additions & 6 deletions jobs/rocoto/atmensanlfv3inc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,6 @@ status=$?
export job="atmensanlfv3inc"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow utilities and tasks
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${wxflowPATH}"
export PYTHONPATH

###############################################################
# Execute the JJOB
"${HOMEgfs}/jobs/JGLOBAL_ATMENS_ANALYSIS_FV3_INCREMENT"
Expand Down
6 changes: 0 additions & 6 deletions jobs/rocoto/atmensanlinit.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,6 @@ status=$?
export job="atmensanlinit"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow utilities and tasks
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${wxflowPATH}"
export PYTHONPATH

###############################################################
# Execute the JJOB
"${HOMEgfs}/jobs/JGLOBAL_ATMENS_ANALYSIS_INITIALIZE"
Expand Down
6 changes: 0 additions & 6 deletions jobs/rocoto/atmensanlletkf.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,6 @@ status=$?
export job="atmensanlletkf"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow utilities and tasks
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${wxflowPATH}"
export PYTHONPATH

###############################################################
# Execute the JJOB
"${HOMEgfs}/jobs/JGLOBAL_ATMENS_ANALYSIS_LETKF"
Expand Down
6 changes: 0 additions & 6 deletions jobs/rocoto/oceanice_products.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,6 @@ source "${HOMEgfs}/ush/preamble.sh"
status=$?
if (( status != 0 )); then exit "${status}"; fi

###############################################################
# setup python path for workflow utilities and tasks
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${wxflowPATH}"
export PYTHONPATH

export job="oceanice_products"
export jobid="${job}.$$"

Expand Down
5 changes: 2 additions & 3 deletions jobs/rocoto/prepatmiodaobs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,10 @@ export job="prepatmobs"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow and ioda utilities
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src"
# setup python path for ioda utilities
# shellcheck disable=SC2311
pyiodaPATH="${HOMEgfs}/sorc/gdas.cd/build/lib/python$(detect_py_ver)/"
PYTHONPATH="${pyiodaPATH}:${wxflowPATH}:${PYTHONPATH}"
PYTHONPATH="${pyiodaPATH}:${PYTHONPATH}"
export PYTHONPATH

###############################################################
Expand Down
6 changes: 0 additions & 6 deletions jobs/rocoto/prepobsaero.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,6 @@ status=$?
export job="prepobsaero"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow utilities and tasks
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${wxflowPATH}"
export PYTHONPATH

###############################################################
# Execute the JJOB
"${HOMEgfs}/jobs/JGLOBAL_PREP_OBS_AERO"
Expand Down
5 changes: 2 additions & 3 deletions jobs/rocoto/prepsnowobs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,11 @@ export job="prepsnowobs"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow utilities and tasks
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src"
# setup python path for ioda utilities
# shellcheck disable=SC2311
pyiodaPATH="${HOMEgfs}/sorc/gdas.cd/build/lib/python$(detect_py_ver)/"
gdasappPATH="${HOMEgfs}/sorc/gdas.cd/sorc/iodaconv/src:${pyiodaPATH}"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${wxflowPATH}:${gdasappPATH}"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}:${gdasappPATH}"
export PYTHONPATH

###############################################################
Expand Down
6 changes: 0 additions & 6 deletions jobs/rocoto/snowanl.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,6 @@ status=$?
export job="snowanl"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow utilities and tasks
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${wxflowPATH}"
export PYTHONPATH

###############################################################
# Execute the JJOB
"${HOMEgfs}/jobs/JGLOBAL_SNOW_ANALYSIS"
Expand Down
12 changes: 6 additions & 6 deletions jobs/rocoto/upp.sh
Original file line number Diff line number Diff line change
Expand Up @@ -29,18 +29,18 @@ if [[ "${MACHINE_ID}" = "wcoss2" ]]; then
module load python/3.8.6
module load crtm/2.4.0 # TODO: This is only needed when UPP_RUN=goes. Is there a better way to handle this?
set_trace

# Add wxflow to PYTHONPATH
wxflowPATH="${HOMEgfs}/ush/python"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${HOMEgfs}/ush:${wxflowPATH}"
export PYTHONPATH

else
. "${HOMEgfs}/ush/load_fv3gfs_modules.sh"
status=$?
if (( status != 0 )); then exit "${status}"; fi
fi

###############################################################
# setup python path for workflow utilities and tasks
wxflowPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/wxflow/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${wxflowPATH}"
export PYTHONPATH

export job="upp"
export jobid="${job}.$$"

Expand Down
7 changes: 5 additions & 2 deletions parm/config/gfs/config.atmanl
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,17 @@ export INTERP_METHOD='barycentric'
if [[ ${DOHYBVAR} = "YES" ]]; then
# shellcheck disable=SC2153
export CASE_ANL=${CASE_ENS}
export BERROR_YAML="background_error_hybrid_${STATICB_TYPE}_${LOCALIZATION_TYPE}"
export BERROR_YAML="atmosphere_background_error_hybrid_${STATICB_TYPE}_${LOCALIZATION_TYPE}"
else
export CASE_ANL=${CASE}
export BERROR_YAML="background_error_static_${STATICB_TYPE}"
export BERROR_YAML="atmosphere_background_error_static_${STATICB_TYPE}"
fi

export CRTM_FIX_YAML="${PARMgfs}/gdas/atm_crtm_coeff.yaml.j2"
export JEDI_FIX_YAML="${PARMgfs}/gdas/atm_jedi_fix.yaml.j2"
export VAR_BKG_STAGING_YAML="${PARMgfs}/gdas/staging/atm_var_bkg.yaml.j2"
export BERROR_STAGING_YAML="${PARMgfs}/gdas/staging/atm_berror_${STATICB_TYPE}.yaml.j2"
export FV3ENS_STAGING_YAML="${PARMgfs}/gdas/staging/atm_var_fv3ens.yaml.j2"

export layout_x_atmanl=@LAYOUT_X_ATMANL@
export layout_y_atmanl=@LAYOUT_Y_ATMANL@
Expand Down
1 change: 1 addition & 0 deletions parm/config/gfs/config.atmensanl
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ export INTERP_METHOD='barycentric'

export CRTM_FIX_YAML="${PARMgfs}/gdas/atm_crtm_coeff.yaml.j2"
export JEDI_FIX_YAML="${PARMgfs}/gdas/atm_jedi_fix.yaml.j2"
export LGETKF_BKG_STAGING_YAML="${PARMgfs}/gdas/staging/atm_lgetkf_bkg.yaml.j2"

export layout_x_atmensanl=@LAYOUT_X_ATMENSANL@
export layout_y_atmensanl=@LAYOUT_Y_ATMENSANL@
Expand Down
8 changes: 8 additions & 0 deletions parm/gdas/staging/atm_berror_gsibec.yaml.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{% set fname_list = ['gfs_gsi_global.nml', 'gsi-coeffs-gfs-global.nc4'] %}

mkdir:
- '{{ DATA }}/berror'
copy:
{% for fname in fname_list %}
- ['{{ HOMEgfs }}/fix/gdas/gsibec/{{ CASE_ANL }}/{{ fname }}', '{{ DATA }}/berror']
{% endfor %}
32 changes: 32 additions & 0 deletions parm/gdas/staging/atm_lgetkf_bkg.yaml.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
{% set ftype_list = ['fv_core.res', 'fv_srf_wnd.res', 'fv_tracer.res', 'phy_data', 'sfc_data'] %}
{% set time_list = [current_cycle] %}

mkdir:
{% for imem in range(1,NMEM_ENS+1) %}
{% set memchar = 'mem%03d' | format(imem) %}
{% set tmpl_dict = ({ '${ROTDIR}': ROTDIR,
'${RUN}': RUN,
'${YMD}': current_cycle | to_YMD,
'${HH}': current_cycle | strftime('%H'),
'${MEMDIR}': memchar }) %}
- '{{ DATA }}/bkg/{{ memchar }}'
- '{{ DATA }}/anl/{{ memchar }}'
- '{{ COM_ATMOS_ANALYSIS_TMPL | replace_tmpl(tmpl_dict) }}'
{% endfor %}
copy:
{% for time in time_list %}
{% for imem in range(1,NMEM_ENS+1) %}
{% set memchar = 'mem%03d' | format(imem) %}
{% set tmpl_dict = ({ '${ROTDIR}': ROTDIR,
'${RUN}': 'enkfgdas',
'${YMD}': previous_cycle | to_YMD,
'${HH}': previous_cycle | strftime('%H'),
'${MEMDIR}': memchar }) %}
- ['{{ COM_ATMOS_RESTART_TMPL | replace_tmpl(tmpl_dict) }}/{{ time | to_fv3time }}.coupler.res', '{{ DATA }}/bkg/{{ memchar }}/']
{% for ftype in ftype_list %}
{% for itile in range(1,7) %}
- ['{{ COM_ATMOS_RESTART_TMPL | replace_tmpl(tmpl_dict) }}/{{ time | to_fv3time }}.{{ ftype }}.tile{{ itile }}.nc', '{{ DATA }}/bkg/{{ memchar }}/']
{% endfor %}
{% endfor %}
{% endfor %}
{% endfor %}
14 changes: 14 additions & 0 deletions parm/gdas/staging/atm_var_bkg.yaml.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{% set ftype_list = ['fv_core.res', 'fv_srf_wnd.res', 'fv_tracer.res', 'phy_data', 'sfc_data'] %}
{% set time_list = [current_cycle] %}

mkdir:
- '{{ DATA }}/bkg'
copy:
{% for time in time_list %}
- ['{{ COM_ATMOS_RESTART_PREV }}/{{ time | to_fv3time }}.coupler.res', '{{ DATA }}/bkg/']
{% for ftype in ftype_list %}
{% for itile in range(1,ntiles+1) %}
- ['{{ COM_ATMOS_RESTART_PREV }}/{{ time | to_fv3time }}.{{ ftype }}.tile{{ itile }}.nc', '{{ DATA }}/bkg/']
{% endfor %}
{% endfor %}
{% endfor %}
24 changes: 24 additions & 0 deletions parm/gdas/staging/atm_var_fv3ens.yaml.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{% set ftype_list = ['fv_core.res', 'fv_srf_wnd.res', 'fv_tracer.res', 'phy_data', 'sfc_data'] %}
{% set time_list = [current_cycle] %}

mkdir:
{% for imem in range(1,NMEM_ENS+1) %}
- '{{ DATA }}/ens/{{ 'mem%03d' | format(imem) }}'
{% endfor %}
copy:
{% for time in time_list %}
{% for imem in range(1,NMEM_ENS+1) %}
{% set memchar = 'mem%03d' | format(imem) %}
{% set tmpl_dict = ({ '${ROTDIR}': ROTDIR,
'${RUN}': 'enkfgdas',
'${YMD}': previous_cycle | to_YMD,
'${HH}': previous_cycle | strftime('%H'),
'${MEMDIR}': memchar }) %}
- ['{{ COM_ATMOS_RESTART_TMPL | replace_tmpl(tmpl_dict) }}/{{ time | to_fv3time }}.coupler.res', '{{ DATA }}/ens/{{ memchar }}/']
{% for ftype in ftype_list %}
{% for itile in range(1,ntiles+1) %}
- ['{{ COM_ATMOS_RESTART_TMPL | replace_tmpl(tmpl_dict) }}/{{ time | to_fv3time }}.{{ ftype }}.tile{{ itile }}.nc', '{{ DATA }}/ens/{{ memchar }}/']
{% endfor %}
{% endfor %}
{% endfor %}
{% endfor %}
6 changes: 3 additions & 3 deletions scripts/exglobal_cleanup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@ rm -rf "${DATAROOT}/${RUN}efcs"*"${PDY:-}${cyc}"

# Search and delete files/directories from DATAROOT/ older than ${purge_every_days} days
# purge_every_days should be a positive integer
purge_every_days=3
#purge_every_days=3

# Find and delete files older than ${purge_every_days} days
find "${DATAROOT}/"* -type f -mtime "+${purge_every_days}" -exec rm -f {} \;
#find "${DATAROOT}/"* -type f -mtime "+${purge_every_days}" -exec rm -f {} \;

# Find and delete directories older than ${purge_every_days} days
find "${DATAROOT}/"* -type d -mtime "+${purge_every_days}" -exec rm -rf {} \;
#find "${DATAROOT}/"* -type d -mtime "+${purge_every_days}" -exec rm -rf {} \;

echo "Cleanup ${DATAROOT} completed!"
###############################################################
Expand Down
2 changes: 1 addition & 1 deletion scripts/exglobal_prep_snow_obs.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,5 @@
# Instantiate the snow prepare task
SnowAnl = SnowAnalysis(config)
SnowAnl.prepare_GTS()
if f"{ SnowAnl.runtime_config.cyc }" == '18':
if f"{ SnowAnl.task_config.cyc }" == '18':
SnowAnl.prepare_IMS()
2 changes: 1 addition & 1 deletion sorc/gdas.cd
Submodule gdas.cd updated 51 files
+19 −0 README.md
+1 −1 build.sh
+25 −4 ci/driver.sh
+27 −2 ci/gw_driver.sh
+2 −2 ci/hera.sh
+0 −3 modulefiles/GDAS/gaea.intel.lua
+0 −3 modulefiles/GDAS/hera.intel.lua
+0 −3 modulefiles/GDAS/noaacloud.intel.lua
+10 −9 modulefiles/GDAS/orion.intel.lua
+5 −2 parm/aero/berror/aero_diffusionparm.yaml.j2
+33 −0 parm/aero/berror/staticb_diffusion.yaml.j2
+49 −57 parm/atm/jcb-base.yaml.j2
+1 −1 parm/atm/jcb-prototype_lgetkf.yaml.j2
+0 −356 parm/atm/obs/config/satwind_modis_aqua.yaml.j2
+0 −368 parm/atm/obs/config/satwind_modis_terra.yaml.j2
+0 −289 parm/atm/obs/config/satwind_viirs_n20.yaml.j2
+0 −289 parm/atm/obs/config/satwind_viirs_npp.yaml.j2
+1 −1 parm/jcb-algorithms
+1 −1 parm/jcb-gdas
+5 −0 parm/soca/marine-jcb-3dfgat.yaml.j2
+72 −0 parm/soca/marine-jcb-base.yaml
+67 −1 scripts/exgdas_global_marine_analysis_prep.py
+13 −3 scripts/exgdas_global_marine_analysis_vrfy.py
+1 −1 sorc/jcb
+9 −0 test/aero/genyaml_3dvar.sh
+1 −1 test/aero/global-workflow/jjob_var_final.sh
+1 −1 test/aero/global-workflow/jjob_var_init.sh
+1 −1 test/aero/global-workflow/jjob_var_run.sh
+2 −2 test/atm/global-workflow/jcb-prototype_3dvar.yaml.j2
+2 −2 test/atm/global-workflow/jcb-prototype_lgetkf.yaml.j2
+1 −1 test/atm/global-workflow/jjob_ens_final.sh
+1 −1 test/atm/global-workflow/jjob_ens_inc.sh
+1 −1 test/atm/global-workflow/jjob_ens_init.sh
+1 −1 test/atm/global-workflow/jjob_ens_run.sh
+1 −1 test/atm/global-workflow/jjob_var_final.sh
+1 −1 test/atm/global-workflow/jjob_var_inc.sh
+1 −1 test/atm/global-workflow/jjob_var_init.sh
+1 −1 test/atm/global-workflow/jjob_var_run.sh
+6 −17 test/soca/gw/CMakeLists.txt
+9 −0 test/soca/socaincr2mom6.sh
+3 −3 ush/eva/marine_gdas_plots.yaml
+1 −4 ush/examples/run_scripts/soca1deg.yaml
+0 −45 ush/soca/examples/run_jjobs_container.yaml
+0 −50 ush/soca/examples/run_jjobs_hera.yaml
+0 −50 ush/soca/examples/run_jjobs_hera_025.yaml
+56 −56 ush/soca/marine_recenter.py
+29 −29 ush/soca/prep_ocean_obs.py
+4 −4 ush/soca/prep_ocean_obs_utils.py
+2 −3 ush/soca/run_jjobs.py
+94 −33 ush/soca/soca_vrfy.py
+3 −3 utils/soca/gdas_soca_diagb.h
2 changes: 1 addition & 1 deletion sorc/wxflow
Loading
Loading