Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidate VMINMON and cleanup monitor scripts #2088

Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
ef25518
Rename and delete VMINMON scripts
KateFriedman-NOAA Nov 27, 2023
0aabdcf
Monitor fix file updates
KateFriedman-NOAA Nov 27, 2023
0fe3072
Consolidate new GLOBAL VMINMON scripts
KateFriedman-NOAA Nov 27, 2023
be25d08
Export gsistat for ex-script
KateFriedman-NOAA Nov 27, 2023
6f24765
Update fix subfolder to "mon"
KateFriedman-NOAA Nov 27, 2023
7b42fed
Clean up the GSI monitor scripts some more
KateFriedman-NOAA Nov 27, 2023
80e0590
Resolve SC2086 shellcheck warnings for VMINMON
KateFriedman-NOAA Nov 27, 2023
2e76dbc
Remove PDY and cyc arguments for VMINMON scripts
KateFriedman-NOAA Nov 27, 2023
b6cd965
Address reviewer comments with changes
KateFriedman-NOAA Nov 27, 2023
b48772f
Update GDATE setting in JGDAS_ATMOS_VERFOZN
KateFriedman-NOAA Nov 28, 2023
2da4311
Update exit err catch in JGDAS_ATMOS_VERFRAD
KateFriedman-NOAA Nov 28, 2023
3f90e29
Move fix to parm
KateFriedman-NOAA Nov 28, 2023
054cb39
Further cleanup of monitor scripts
KateFriedman-NOAA Nov 28, 2023
43e1fa0
Address shellcheck warnings
KateFriedman-NOAA Nov 28, 2023
80c92c7
Update PDY parsing in ush/radmon_verf_angle.sh
KateFriedman-NOAA Nov 28, 2023
9a3400e
Update PDY parsing in ush/radmon_verf_bcor.sh
KateFriedman-NOAA Nov 28, 2023
3b9c3a9
Merge branch 'NOAA-EMC:develop' into feature/combine_vminmon_scripts
KateFriedman-NOAA Nov 28, 2023
909f7b4
Compress sourcing of parm file in configs
KateFriedman-NOAA Nov 28, 2023
1dc638d
Address SC2086 and SC2250 shellcheck warnings
KateFriedman-NOAA Nov 28, 2023
8f1f6c6
Address SC2086 shellcheck warnings in exgdas_atmos_verfrad.sh
KateFriedman-NOAA Nov 28, 2023
a04fbd2
Cleanup ozn_xtrct.sh a bit
KateFriedman-NOAA Nov 28, 2023
1ae1bce
Remove invocation of script that no longer exists
KateFriedman-NOAA Nov 28, 2023
ba6b2f9
Address SC2086 in exglobal_atmos_vminmon.sh
KateFriedman-NOAA Nov 28, 2023
2da1a9d
A few corrections to radmon scripts
KateFriedman-NOAA Nov 28, 2023
0787ee9
Add {} into scripts/exgdas_atmos_verfrad.sh
KateFriedman-NOAA Nov 29, 2023
9600f0a
Add shellcheck disable to jobs/JGDAS_ATMOS_VERFOZN
KateFriedman-NOAA Nov 29, 2023
d280a02
Add shellcheck disable for SC2153 for monitor jobs
KateFriedman-NOAA Nov 29, 2023
59a5cc4
Merge branch 'NOAA-EMC:develop' into feature/combine_vminmon_scripts
KateFriedman-NOAA Nov 29, 2023
c9f755c
Merge branch 'NOAA-EMC:develop' into feature/combine_vminmon_scripts
KateFriedman-NOAA Nov 29, 2023
c5cef17
Update qdate in ush/radmon_verf_time.sh
KateFriedman-NOAA Dec 1, 2023
3187b30
Apply suggestions from code review
KateFriedman-NOAA Dec 1, 2023
d33bbd3
Merge branch 'NOAA-EMC:develop' into feature/combine_vminmon_scripts
KateFriedman-NOAA Dec 1, 2023
92b225f
Address shellcheck SC2126 in ush/radmon_diag_ck.sh
KateFriedman-NOAA Dec 1, 2023
5a7c5f9
Address shellcheck SC2250 and SC2086
KateFriedman-NOAA Dec 1, 2023
57995ff
Merge branch 'NOAA-EMC:develop' into feature/combine_vminmon_scripts
KateFriedman-NOAA Dec 4, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,11 @@ fix/chem
fix/cice
fix/cpl
fix/datm
fix/gdas
fix/gldas
fix/gsi
fix/lut
fix/mom6
fix/mon
KateFriedman-NOAA marked this conversation as resolved.
Show resolved Hide resolved
fix/orog
fix/reg2grb2
fix/sfc_climo
Expand Down
4 changes: 0 additions & 4 deletions fix/product/gfs_minmon_cost.txt

This file was deleted.

6 changes: 0 additions & 6 deletions fix/product/gfs_minmon_gnorm.txt

This file was deleted.

35 changes: 16 additions & 19 deletions jobs/JGDAS_ATMOS_VERFOZN
Original file line number Diff line number Diff line change
Expand Up @@ -15,27 +15,17 @@ export HOMEgfs_ozn=${HOMEgfs:-${NWROOT}/gfs.${gfs_ver}}
export HOMEgdas_ozn=${HOMEgfs_ozn:-${NWROOT}/gfs.${gfs_ver}}
export PARMgdas_ozn=${PARMgfs_ozn:-${HOMEgfs_ozn}/parm/mon}
export SCRgdas_ozn=${SCRgfs_ozn:-${HOMEgfs_ozn}/scripts}
export FIXgdas_ozn=${FIXgfs_ozn:-${HOMEgfs_ozn}/fix/gdas}
export FIXgdas_ozn=${FIXgfs_ozn:-${HOMEgfs_ozn}/fix/mon}

export HOMEoznmon=${HOMEoznmon:-${HOMEgfs_ozn}}
export EXECoznmon=${EXECoznmon:-${HOMEoznmon}/exec}
export FIXoznmon=${FIXoznmon:-${HOMEoznmon}/fix}
export USHoznmon=${USHoznmon:-${HOMEoznmon}/ush}


#-----------------------------------
# source the parm file
#
. ${PARMgdas_ozn}/gdas_oznmon.parm


#############################################
# determine PDY and cyc for previous cycle
#############################################

pdate=$(${NDATE} -6 ${PDY}${cyc})
echo "pdate = ${pdate}"

export P_PDY=${pdate:0:8}
export p_cyc=${pdate:8:2}

Expand All @@ -49,20 +39,27 @@ export TANKverf_ozn=${TANKverf_ozn:-${COM_ATMOS_OZNMON}}

if [[ ! -d ${TANKverf_ozn} ]]; then mkdir -p -m 775 ${TANKverf_ozn} ; fi

#-----------------------------------
# Other variables
#
export OZN_AREA=${OZN_AREA:-glb}
export oznstat=${oznstat:-${COM_ATMOS_ANALYSIS}/gdas.t${cyc}z.oznstat}
export SATYPE_FILE=${SATYPE_FILE:-${FIXgdas_ozn}/gdas_oznmon_satype.txt}
export PDATE=${PDY}${cyc}
export DO_DATA_RPT=${DO_DATA_RPT:-1}
KateFriedman-NOAA marked this conversation as resolved.
Show resolved Hide resolved

#-----------------------------------
# source the parm file
#
. ${PARMgdas_ozn}/gdas_oznmon.parm

#---------------------------------------
# set up validation file
# Set up validation file
#
if [[ ${VALIDATE_DATA} -eq 1 ]]; then
export ozn_val_file=${ozn_val_file:-${FIXgdas_ozn}/gdas_oznmon_base.tar}
fi

#---------------------------------------
# Set necessary environment variables
#
export OZN_AREA=${OZN_AREA:-glb}
export oznstat=${oznstat:-${COM_ATMOS_ANALYSIS}/gdas.t${cyc}z.oznstat}


#-------------------------------------------------------
# Execute the script.
#
Expand Down
39 changes: 14 additions & 25 deletions jobs/JGDAS_ATMOS_VERFRAD
Original file line number Diff line number Diff line change
Expand Up @@ -6,40 +6,34 @@
source "${HOMEgfs}/ush/preamble.sh"
source "${HOMEgfs}/ush/jjob_header.sh" -e "verfrad" -c "base verfrad"

export COMPONENT="atmos"

export RADMON_SUFFIX=${RADMON_SUFFIX:-${RUN}}
export CYCLE_INTERVAL=${CYCLE_INTERVAL:-6}
export VERBOSE=${VERBOSE:-YES}
KateFriedman-NOAA marked this conversation as resolved.
Show resolved Hide resolved

##############################################
# Specify Execution Areas
##############################################
export HOMEgfs=${HOMEgfs:-${NWROOT}/gfs.${gfs_ver}}
export SCRgfs=${SCRgfs:-${HOMEgfs}/scripts}

export FIXgdas=${FIXgdas:-${HOMEgfs}/fix/gdas}
export FIXgdas=${FIXgdas:-${HOMEgfs}/fix/mon}
export PARMmon=${PARMmon:-${HOMEgfs}/parm/mon}

export HOMEradmon=${HOMEradmon:-${HOMEgfs}}
export EXECradmon=${EXECradmon:-${HOMEradmon}/exec}
export FIXradmon=${FIXradmon:-${FIXgfs}}
export USHradmon=${USHradmon:-${HOMEradmon}/ush}


###################################
# source the parm file
###################################
parm_file=${parm_file:-${PARMmon}/da_mon.parm}
. ${parm_file}


#############################################
# determine PDY and cyc for previous cycle
#############################################

pdate=$(${NDATE} -6 ${PDY}${cyc})
echo "pdate = ${pdate}"

export P_PDY=${pdate:0:8}
export p_cyc=${pdate:8:2}

Expand All @@ -57,31 +51,26 @@ export TANKverf_radM1=${TANKverf_radM1:-${COM_ATMOS_RADMON_PREV}}
if [[ ! -d ${TANKverf_rad} ]]; then mkdir -p -m 775 ${TANKverf_rad} ; fi
if [[ ! -d ${TANKverf_radM1} ]]; then mkdir -p -m 775 ${TANKverf_radM1} ; fi

########################################
# Set necessary environment variables
########################################
export RAD_AREA=${RAD_AREA:-glb}

#############################################
# Filenames
export biascr=${biascr:-${COM_ATMOS_ANALYSIS}/gdas.t${cyc}z.abias}
export radstat=${radstat:-${COM_ATMOS_ANALYSIS}/gdas.t${cyc}z.radstat}
export satype_file=${satype_file:-${FIXgdas}/gdas_radmon_satype.txt}

echo " "
echo "JOB HAS STARTED"
echo " "

# Other variables
export RAD_AREA=${RAD_AREA:-glb}
export MAKE_CTL=${MAKE_CTL:-1}
export MAKE_DATA=${MAKE_DATA:-1}
export USE_ANL=${USE_ANL:-1}
export PDATE=${PDY}${cyc}
export DO_DIAG_RPT=${DO_DIAG_RPT:-1}
export DO_DATA_RPT=${DO_DATA_RPT:-1}
KateFriedman-NOAA marked this conversation as resolved.
Show resolved Hide resolved
KateFriedman-NOAA marked this conversation as resolved.
Show resolved Hide resolved

########################################################
# Execute the script.
${RADMONSH:-${SCRgfs}/exgdas_atmos_verfrad.sh} ${PDY} ${cyc}
err=$?

if [[ ${err} -ne 0 ]] ; then
exit ${err}
else
echo " "
echo "JOB HAS COMPLETED NORMALLY"
echo " "
fi
[[ ${err} -ne 0 ]] && exit ${err}
KateFriedman-NOAA marked this conversation as resolved.
Show resolved Hide resolved

################################
# Remove the Working Directory
Expand Down
71 changes: 0 additions & 71 deletions jobs/JGFS_ATMOS_VMINMON

This file was deleted.

36 changes: 12 additions & 24 deletions jobs/JGDAS_ATMOS_VMINMON → jobs/JGLOBAL_ATMOS_VMINMON
Original file line number Diff line number Diff line change
@@ -1,42 +1,25 @@
#! /usr/bin/env bash

###########################################################
# GDAS Minimization Monitor (MinMon) job
# Global Minimization Monitor (MinMon) job
###########################################################
source "${HOMEgfs}/ush/preamble.sh"
source "${HOMEgfs}/ush/jjob_header.sh" -e "vminmon" -c "base vminmon"

###########################################################
# obtain unique process id (pid) and make temp directories
###########################################################
export MINMON_SUFFIX=${MINMON_SUFFIX:-${NET}}
export m_job=${m_job:-${MINMON_SUFFIX}_mmDE}


##############################################
# Specify Package Areas
##############################################
export HOMEgfs=${HOMEgfs:-${NWROOT}/gfs.${gfs_ver}}
export M_FIX=${M_FIX:-${HOMEgfs}/fix/mon}
export SCRgfs=${SCRgfs:-${HOMEgfs}/scripts}

export M_FIXgdas=${M_FIXgdas:-${HOMEgfs}/fix/gdas}

export HOMEminmon=${HOMEminmon:-${HOMEgfs}}
export EXECminmon=${EXECminmon:-${HOMEminmon}/exec}
export USHminmon=${USHminmon:-${HOMEminmon}/ush}

export USHminmon=${USHminmon:-${HOMEgfs}/ush}
KateFriedman-NOAA marked this conversation as resolved.
Show resolved Hide resolved

#############################################
# determine PDY and cyc for previous cycle
# Determine PDY and cyc for previous cycle
KateFriedman-NOAA marked this conversation as resolved.
Show resolved Hide resolved
#############################################

pdate=$(${NDATE} -6 ${PDY}${cyc})
echo "pdate = ${pdate}"

export P_PDY=${pdate:0:8}
export p_cyc=${pdate:8:2}


#############################################
# TANKverf - WHERE OUTPUT DATA WILL RESIDE
#############################################
Expand All @@ -50,17 +33,22 @@ export M_TANKverfM1=${M_TANKverfM1:-${COM_ATMOS_MINMON_PREV}}
if [[ ! -d ${M_TANKverf} ]]; then mkdir -p -m 775 ${M_TANKverf} ; fi
if [[ ! -d ${M_TANKverfM1} ]]; then mkdir -p -m 775 ${M_TANKverfM1} ; fi


########################################
# Set necessary environment variables
########################################
export MINMON_SUFFIX=${MINMON_SUFFIX:-${NET}}
export CYCLE_INTERVAL=6
export gsistat=${gsistat:-${COM_ATMOS_ANALYSIS}/gdas.t${cyc}z.gsistat}

########################################
# Filenames
########################################
export gsistat=${gsistat:-${COM_ATMOS_ANALYSIS}/${RUN}.t${cyc}z.gsistat}
export mm_gnormfile=${gnormfile:-${M_FIX}/${RUN}_minmon_gnorm.txt}
export mm_costfile=${costfile:-${M_FIX}/${RUN}_minmon_cost.txt}
KateFriedman-NOAA marked this conversation as resolved.
Show resolved Hide resolved

########################################################
# Execute the script.
${GMONSH:-${SCRgfs}/exgdas_atmos_vminmon.sh} ${PDY} ${cyc}
"${GMONSH:-${SCRgfs}/exglobal_atmos_vminmon.sh}"
err=$?
[[ ${err} -ne 0 ]] && exit ${err}

Expand Down
2 changes: 1 addition & 1 deletion jobs/rocoto/vminmon.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ export jobid="${job}.$$"
echo
echo "=============== START TO RUN MINMON ==============="

"${HOMEgfs}/jobs/J${RUN^^}_ATMOS_VMINMON"
"${HOMEgfs}/jobs/JGLOBAL_ATMOS_VMINMON"
status=$?

exit "${status}"
36 changes: 1 addition & 35 deletions scripts/exgdas_atmos_verfozn.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,48 +3,14 @@
source "$HOMEgfs/ush/preamble.sh"

################################################################################
# exgdas_vrfyozn.sh
# exgdas_atmos_verfozn.sh
#
# This script runs the data extract/validation portion of the Ozone Monitor
# (OznMon) DA package.
#
################################################################################
err=0

#-------------------------------------------------------------------------------
# Set environment
#
export NET=${NET:-gfs}
export RUN=${RUN:-gdas}
export envir=${envir:-prod}

# Other variables
export SATYPE_FILE=${SATYPE_FILE:-$FIXgdas_ozn/gdas_oznmon_satype.txt}
export PDATE=${PDY}${cyc}
export DO_DATA_RPT=${DO_DATA_RPT:-1}
export NCP=${NCP:-/bin/cp}


#-----------------------------------------------------------------
# ensure work and TANK dirs exist, verify oznstat is available
#
export OZN_WORK_DIR=${OZN_WORK_DIR:-$(pwd)}

if [[ ! -d ${OZN_WORK_DIR} ]]; then
mkdir $OZN_WORK_DIR
fi
cd $OZN_WORK_DIR

if [[ ! -d ${TANKverf_ozn} ]]; then
mkdir -p $TANKverf_ozn
fi

if [[ -s ${oznstat} ]]; then
echo ${oznstat} is available
fi



data_available=0

if [[ -s ${oznstat} ]]; then
Expand Down
Loading