Skip to content

Commit

Permalink
Fix RadMon file transfers (#95)
Browse files Browse the repository at this point in the history
* Ref #94

First cut.

* Ref #94

Clean up files.

* Ref #94

Unrelated cleanup items recently encountered.
  • Loading branch information
EdwardSafford-NOAA committed Jul 25, 2023
1 parent 835998a commit 360cc51
Show file tree
Hide file tree
Showing 4 changed files with 19 additions and 10 deletions.
2 changes: 1 addition & 1 deletion src/Radiance_Monitor/image_gen/html/Install_html.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ function usage {
echo "Usage: Install_html.sh suffix [-t|--tank]"
echo " Suffix is data source identifier that matches data in "
echo " the $TANKDIR/stats directory."
echo " -t | --tank parent directory to the radnmon data file location. This"
echo " -t | --tank parent directory to the radmon data file location. This"
echo " will be extended by \$RADMON_SUFFIX, \$RUN, and \$PDATE to locate the"
echo " extracted radmon data."
echo ""
Expand Down
10 changes: 6 additions & 4 deletions src/Radiance_Monitor/image_gen/html/README
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,14 @@ The transfer mechanism is a password-free, secure log-in using a public/private
encryption key pair. There is information on the bottom of the wiki page about
how to set that up (see https://svnemc.ncep.noaa.gov/trac/gsi/wiki/RadianceMonitorPackage).

The install script requires two parameters: the data suffix (identifier) and area. The
suffix is used to access data files from $MY_TANKDIR. Area must be either 'glb' or 'rgn'
meaning global or regional.
The install script requires one parameter: the data suffix (identifier). The
suffix is used to access data files from $MY_TANKDIR. If the data source is not in your
$MY_TANKDIR location (which it won't be if you're working from a parallel), then use the
--tank option to specify the location of the data. That directory will then be expanded
using the suffix, run (gdas), and date.

An example calling sequence is thus:

> ./Install_html.sh newd glb
> ./Install_html.sh newd --tank /dir/to/my/data


15 changes: 11 additions & 4 deletions src/Radiance_Monitor/image_gen/ush/RadMon_IG_glb.sh
Original file line number Diff line number Diff line change
Expand Up @@ -344,15 +344,22 @@ if [[ $RUN_TRANSFER -eq 1 ]]; then

transfer_queue=dev_transfer
jobname=transfer_${RADMON_SUFFIX}
export WEBDIR=${WEBDIR}/${RADMON_SUFFIX}/pngs
export WEBDIR=${WEBDIR}/${RADMON_SUFFIX}

cmdfile="${PLOT_WORK_DIR}/transfer_cmd"
echo "${IG_SCRIPTS}/transfer.sh" >$cmdfile
chmod 755 $cmdfile

run_time="$rhr$cmin" # HHMM format for qsub
$SUB -q $transfer_queue -A $ACCOUNT -o ${transfer_log} -e ${transfer_err} \
-V -l select=1:mem=500M -l walltime=45:00 -N ${jobname} -a ${run_time} ${cmdfile}
if [[ ${MY_MACHINE} = "hera" ]]; then
${SUB} --account ${ACCOUNT} --ntasks=1 --mem=500M --time=45:00 -J ${jobname} \
--partition service -o ${transfer_log} --begin=${rhr}:${cmin} ${IG_SCRIPTS}/transfer.sh

elif [[ ${MY_MACHINE} = "wcoss2" ]]; then
run_time="$rhr$cmin" # HHMM format for qsub
$SUB -q $transfer_queue -A $ACCOUNT -o ${transfer_log} -e ${transfer_err} \
-V -l select=1:mem=500M -l walltime=45:00 -N ${jobname} -a ${run_time} ${cmdfile}
fi


fi
fi
Expand Down
2 changes: 1 addition & 1 deletion src/Radiance_Monitor/image_gen/ush/transfer.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ if [[ ${TANKimg} != "/" && -d ${TANKimg} ]]; then
fi

/usr/bin/rsync -ave ssh --exclude *.ctl.${Z} \
--exclude 'horiz' --exclude *.png --delete-during --update ${TANKimg}/ \
--exclude 'horiz' --exclude '*.png' --delete-during --update ${TANKimg}/ \
${WEBUSER}@${WEBSVR}:${WEBDIR}/

else
Expand Down

0 comments on commit 360cc51

Please sign in to comment.