Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidate VMINMON and cleanup monitor scripts #2088

Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
ef25518
Rename and delete VMINMON scripts
KateFriedman-NOAA Nov 27, 2023
0aabdcf
Monitor fix file updates
KateFriedman-NOAA Nov 27, 2023
0fe3072
Consolidate new GLOBAL VMINMON scripts
KateFriedman-NOAA Nov 27, 2023
be25d08
Export gsistat for ex-script
KateFriedman-NOAA Nov 27, 2023
6f24765
Update fix subfolder to "mon"
KateFriedman-NOAA Nov 27, 2023
7b42fed
Clean up the GSI monitor scripts some more
KateFriedman-NOAA Nov 27, 2023
80e0590
Resolve SC2086 shellcheck warnings for VMINMON
KateFriedman-NOAA Nov 27, 2023
2e76dbc
Remove PDY and cyc arguments for VMINMON scripts
KateFriedman-NOAA Nov 27, 2023
b6cd965
Address reviewer comments with changes
KateFriedman-NOAA Nov 27, 2023
b48772f
Update GDATE setting in JGDAS_ATMOS_VERFOZN
KateFriedman-NOAA Nov 28, 2023
2da4311
Update exit err catch in JGDAS_ATMOS_VERFRAD
KateFriedman-NOAA Nov 28, 2023
3f90e29
Move fix to parm
KateFriedman-NOAA Nov 28, 2023
054cb39
Further cleanup of monitor scripts
KateFriedman-NOAA Nov 28, 2023
43e1fa0
Address shellcheck warnings
KateFriedman-NOAA Nov 28, 2023
80c92c7
Update PDY parsing in ush/radmon_verf_angle.sh
KateFriedman-NOAA Nov 28, 2023
9a3400e
Update PDY parsing in ush/radmon_verf_bcor.sh
KateFriedman-NOAA Nov 28, 2023
3b9c3a9
Merge branch 'NOAA-EMC:develop' into feature/combine_vminmon_scripts
KateFriedman-NOAA Nov 28, 2023
909f7b4
Compress sourcing of parm file in configs
KateFriedman-NOAA Nov 28, 2023
1dc638d
Address SC2086 and SC2250 shellcheck warnings
KateFriedman-NOAA Nov 28, 2023
8f1f6c6
Address SC2086 shellcheck warnings in exgdas_atmos_verfrad.sh
KateFriedman-NOAA Nov 28, 2023
a04fbd2
Cleanup ozn_xtrct.sh a bit
KateFriedman-NOAA Nov 28, 2023
1ae1bce
Remove invocation of script that no longer exists
KateFriedman-NOAA Nov 28, 2023
ba6b2f9
Address SC2086 in exglobal_atmos_vminmon.sh
KateFriedman-NOAA Nov 28, 2023
2da1a9d
A few corrections to radmon scripts
KateFriedman-NOAA Nov 28, 2023
0787ee9
Add {} into scripts/exgdas_atmos_verfrad.sh
KateFriedman-NOAA Nov 29, 2023
9600f0a
Add shellcheck disable to jobs/JGDAS_ATMOS_VERFOZN
KateFriedman-NOAA Nov 29, 2023
d280a02
Add shellcheck disable for SC2153 for monitor jobs
KateFriedman-NOAA Nov 29, 2023
59a5cc4
Merge branch 'NOAA-EMC:develop' into feature/combine_vminmon_scripts
KateFriedman-NOAA Nov 29, 2023
c9f755c
Merge branch 'NOAA-EMC:develop' into feature/combine_vminmon_scripts
KateFriedman-NOAA Nov 29, 2023
c5cef17
Update qdate in ush/radmon_verf_time.sh
KateFriedman-NOAA Dec 1, 2023
3187b30
Apply suggestions from code review
KateFriedman-NOAA Dec 1, 2023
d33bbd3
Merge branch 'NOAA-EMC:develop' into feature/combine_vminmon_scripts
KateFriedman-NOAA Dec 1, 2023
92b225f
Address shellcheck SC2126 in ush/radmon_diag_ck.sh
KateFriedman-NOAA Dec 1, 2023
5a7c5f9
Address shellcheck SC2250 and SC2086
KateFriedman-NOAA Dec 1, 2023
57995ff
Merge branch 'NOAA-EMC:develop' into feature/combine_vminmon_scripts
KateFriedman-NOAA Dec 4, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ fix/chem
fix/cice
fix/cpl
fix/datm
fix/gdas
fix/gldas
fix/gsi
fix/lut
Expand All @@ -47,7 +46,7 @@ fix/wafs
#--------------------------
parm/config/config.base
parm/gldas
parm/mon
parm/monitor
parm/post/AEROSOL_LUTS.dat
parm/post/nam_micro_lookup.dat
parm/post/optics_luts_DUST.dat
Expand Down
4 changes: 0 additions & 4 deletions fix/product/gfs_minmon_cost.txt

This file was deleted.

6 changes: 0 additions & 6 deletions fix/product/gfs_minmon_gnorm.txt

This file was deleted.

53 changes: 8 additions & 45 deletions jobs/JGDAS_ATMOS_VERFOZN
Original file line number Diff line number Diff line change
Expand Up @@ -6,69 +6,32 @@
source "${HOMEgfs}/ush/preamble.sh"
source "${HOMEgfs}/ush/jjob_header.sh" -e "verfozn" -c "base verfozn"

export OZNMON_SUFFIX=${OZNMON_SUFFIX:-${NET}}

#---------------------------------------------
# Specify Execution Areas
#
export HOMEgfs_ozn=${HOMEgfs:-${NWROOT}/gfs.${gfs_ver}}
export HOMEgdas_ozn=${HOMEgfs_ozn:-${NWROOT}/gfs.${gfs_ver}}
export PARMgdas_ozn=${PARMgfs_ozn:-${HOMEgfs_ozn}/parm/mon}
export SCRgdas_ozn=${SCRgfs_ozn:-${HOMEgfs_ozn}/scripts}
export FIXgdas_ozn=${FIXgfs_ozn:-${HOMEgfs_ozn}/fix/gdas}

export HOMEoznmon=${HOMEoznmon:-${HOMEgfs_ozn}}
export EXECoznmon=${EXECoznmon:-${HOMEoznmon}/exec}
export FIXoznmon=${FIXoznmon:-${HOMEoznmon}/fix}
export USHoznmon=${USHoznmon:-${HOMEoznmon}/ush}


#-----------------------------------
# source the parm file
#
. ${PARMgdas_ozn}/gdas_oznmon.parm


#############################################
# determine PDY and cyc for previous cycle
#############################################

pdate=$(${NDATE} -6 ${PDY}${cyc})
echo "pdate = ${pdate}"

export P_PDY=${pdate:0:8}
export p_cyc=${pdate:8:2}
GDATE=$(date --utc +%Y%m%d%H -d "${PDY} ${cyc} - ${assim_freq} hours")
github-advanced-security[bot] marked this conversation as resolved.
Fixed
Show resolved Hide resolved
export gPDY=${GDATE:0:8}
export gcyc=${GDATE:8:2}

#---------------------------------------------
# OZN_TANKDIR - WHERE OUTPUT DATA WILL RESIDE
#
YMD=${PDY} HH=${cyc} generate_com -rx COM_ATMOS_ANALYSIS
YMD=${PDY} HH=${cyc} generate_com -rx COM_ATMOS_OZNMON

export oznstat="${COM_ATMOS_ANALYSIS}/gdas.t${cyc}z.oznstat"
export TANKverf_ozn=${TANKverf_ozn:-${COM_ATMOS_OZNMON}}

if [[ ! -d ${TANKverf_ozn} ]]; then mkdir -p -m 775 ${TANKverf_ozn} ; fi

#---------------------------------------
# set up validation file
#
if [[ ${VALIDATE_DATA} -eq 1 ]]; then
export ozn_val_file=${ozn_val_file:-${FIXgdas_ozn}/gdas_oznmon_base.tar}
fi

#---------------------------------------
# Set necessary environment variables
#
export OZN_AREA=${OZN_AREA:-glb}
export oznstat=${oznstat:-${COM_ATMOS_ANALYSIS}/gdas.t${cyc}z.oznstat}


#-------------------------------------------------------
# Execute the script.
#
${OZNMONSH:-${SCRgdas_ozn}/exgdas_atmos_verfozn.sh} ${PDY} ${cyc}
"${SCRgfs}/exgdas_atmos_verfozn.sh"
err=$?
[[ ${err} -ne 0 ]] && exit ${err}
if (( err != 0 )); then
exit "${err}"
fi


################################
Expand Down
64 changes: 9 additions & 55 deletions jobs/JGDAS_ATMOS_VERFRAD
Original file line number Diff line number Diff line change
Expand Up @@ -6,81 +6,35 @@
source "${HOMEgfs}/ush/preamble.sh"
source "${HOMEgfs}/ush/jjob_header.sh" -e "verfrad" -c "base verfrad"

export COMPONENT="atmos"

export RADMON_SUFFIX=${RADMON_SUFFIX:-${RUN}}
export CYCLE_INTERVAL=${CYCLE_INTERVAL:-6}

##############################################
# Specify Execution Areas
##############################################
export HOMEgfs=${HOMEgfs:-${NWROOT}/gfs.${gfs_ver}}
export SCRgfs=${SCRgfs:-${HOMEgfs}/scripts}

export FIXgdas=${FIXgdas:-${HOMEgfs}/fix/gdas}
export PARMmon=${PARMmon:-${HOMEgfs}/parm/mon}

export HOMEradmon=${HOMEradmon:-${HOMEgfs}}
export EXECradmon=${EXECradmon:-${HOMEradmon}/exec}
export FIXradmon=${FIXradmon:-${FIXgfs}}
export USHradmon=${USHradmon:-${HOMEradmon}/ush}


###################################
# source the parm file
###################################
parm_file=${parm_file:-${PARMmon}/da_mon.parm}
. ${parm_file}


#############################################
# determine PDY and cyc for previous cycle
#############################################

pdate=$(${NDATE} -6 ${PDY}${cyc})
echo "pdate = ${pdate}"

export P_PDY=${pdate:0:8}
export p_cyc=${pdate:8:2}
GDATE=$(date --utc +%Y%m%d%H -d "${PDY} ${cyc} - ${assim_freq} hours")
Fixed Show fixed Hide fixed
export gPDY=${GDATE:0:8}
export gcyc=${GDATE:8:2}

#############################################
# COMOUT - WHERE GSI OUTPUT RESIDES
# TANKverf - WHERE OUTPUT DATA WILL RESIDE
#############################################
YMD=${PDY} HH=${cyc} generate_com -rx COM_ATMOS_ANALYSIS
YMD=${PDY} HH=${cyc} generate_com -rx COM_ATMOS_RADMON
YMD=${P_PDY} HH=${p_cyc} generate_com -rx COM_ATMOS_RADMON_PREV:COM_ATMOS_RADMON_TMPL
YMD=${gPDY} HH=${gcyc} generate_com -rx COM_ATMOS_RADMON_PREV:COM_ATMOS_RADMON_TMPL

export biascr="${COM_ATMOS_ANALYSIS}/gdas.t${cyc}z.abias"
export radstat="${COM_ATMOS_ANALYSIS}/gdas.t${cyc}z.radstat"
export TANKverf_rad=${TANKverf_rad:-${COM_ATMOS_RADMON}}
export TANKverf_radM1=${TANKverf_radM1:-${COM_ATMOS_RADMON_PREV}}

if [[ ! -d ${TANKverf_rad} ]]; then mkdir -p -m 775 ${TANKverf_rad} ; fi
if [[ ! -d ${TANKverf_radM1} ]]; then mkdir -p -m 775 ${TANKverf_radM1} ; fi

########################################
# Set necessary environment variables
########################################
export RAD_AREA=${RAD_AREA:-glb}

export biascr=${biascr:-${COM_ATMOS_ANALYSIS}/gdas.t${cyc}z.abias}
export radstat=${radstat:-${COM_ATMOS_ANALYSIS}/gdas.t${cyc}z.radstat}

echo " "
echo "JOB HAS STARTED"
echo " "


########################################################
# Execute the script.
${RADMONSH:-${SCRgfs}/exgdas_atmos_verfrad.sh} ${PDY} ${cyc}
"${SCRgfs}/exgdas_atmos_verfrad.sh"
err=$?

if [[ ${err} -ne 0 ]] ; then
exit ${err}
else
echo " "
echo "JOB HAS COMPLETED NORMALLY"
echo " "
if (( err != 0 )); then
exit "${err}"
fi

################################
Expand Down
73 changes: 0 additions & 73 deletions jobs/JGDAS_ATMOS_VMINMON

This file was deleted.

71 changes: 0 additions & 71 deletions jobs/JGFS_ATMOS_VMINMON

This file was deleted.

43 changes: 43 additions & 0 deletions jobs/JGLOBAL_ATMOS_VMINMON
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
#! /usr/bin/env bash

###########################################################
# Global Minimization Monitor (MinMon) job
###########################################################
source "${HOMEgfs}/ush/preamble.sh"
source "${HOMEgfs}/ush/jjob_header.sh" -e "vminmon" -c "base vminmon"

#############################################
# Determine PDY and cyc for previous cycle
KateFriedman-NOAA marked this conversation as resolved.
Show resolved Hide resolved
#############################################
GDATE=$(date --utc +%Y%m%d%H -d "${PDY} ${cyc} - ${assim_freq} hours")
Fixed Show fixed Hide fixed
export gPDY=${GDATE:0:8}
export gcyc=${GDATE:8:2}

#############################################
# TANKverf - WHERE OUTPUT DATA WILL RESIDE
#############################################
YMD=${PDY} HH=${cyc} generate_com -rx COM_ATMOS_ANALYSIS
YMD=${PDY} HH=${cyc} generate_com -rx COM_ATMOS_MINMON
YMD=${gPDY} HH=${gcyc} generate_com -rx COM_ATMOS_MINMON_PREV:COM_ATMOS_MINMON_TMPL

export gsistat="${COM_ATMOS_ANALYSIS}/${RUN}.t${cyc}z.gsistat"
export M_TANKverf=${M_TANKverf:-${COM_ATMOS_MINMON}}
export M_TANKverfM1=${M_TANKverfM1:-${COM_ATMOS_MINMON_PREV}}

if [[ ! -d ${M_TANKverf} ]]; then mkdir -p -m 775 ${M_TANKverf} ; fi

Check notice

Code scanning / shellcheck

Double quote to prevent globbing and word splitting. Note

Double quote to prevent globbing and word splitting.
if [[ ! -d ${M_TANKverfM1} ]]; then mkdir -p -m 775 ${M_TANKverfM1} ; fi

Check notice

Code scanning / shellcheck

Double quote to prevent globbing and word splitting. Note

Double quote to prevent globbing and word splitting.

########################################################
# Execute the script.
"${SCRgfs}/exglobal_atmos_vminmon.sh"
err=$?
if (( err != 0 )); then
exit "${err}"
fi

################################
# Remove the Working Directory
################################
[[ "${KEEPDATA}" = "NO" ]] && rm -rf "${DATA}"

exit 0
2 changes: 1 addition & 1 deletion jobs/rocoto/vminmon.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ export jobid="${job}.$$"
echo
echo "=============== START TO RUN MINMON ==============="

"${HOMEgfs}/jobs/J${RUN^^}_ATMOS_VMINMON"
"${HOMEgfs}/jobs/JGLOBAL_ATMOS_VMINMON"
status=$?

exit "${status}"
Loading