ascii to nc and pointstat for test case

[perthsb]

Hi,
I am trying to use Aeronet ascii to nc and then pointstat to verify model forecast. Using a sample station data I used
/scratch1/NCEPDEV/stmp2/Partha.Bhattacharjee/METplus/Scripts/Scripts/Aeronet_asciinccnvt.conf to convert it netcdf and generating interpolated 550nm field and then using then using /scratch1/NCEPDEV/stmp2/Partha.Bhattacharjee/METplus/Scripts/Scripts/Pointstat_AeronetAOD.conf to do pointstat to verify model forecast.
However, I am getting the follwoing error while running the 2nd part :
ERROR: Cannot set FCST_VAR1_NAME or OBS_VAR1_NAME if BOTH_VAR1_NAME is set.
09/30 15:04:32Z run-METplus-metplus: ERROR: Cannot set FCST_VAR1_NAME or OBS_VAR1_NAME if BOTH_VAR1_NAME is set

Any help on this to resolve this issue and general comment on use of these .conf for this purpose.
Appreciate your help.

[j-opatz]

Hello,

And thank you for your question. I took a look at Pointstat_AeronetAOD.conf and didn't see any issues with the way you've set up the configuration file.

However, I investigated the area you indicated your working in and saw

run_metplus_PointstatAeronet.sh

Is this the Bash script you're using to run Pointstat_AeronetAOD.conf? If so, the error is in the run_metplus command.

The command listed is

run_metplus.py \
${MET_PLUS_CONF}/PointStat_fcstGFS_obsGDAS_UpperAir_MultiField_PrepBufr.conf \
${METPLUS_TUTORIAL_DIR}/Pointstat_AeronetAOD.conf \
config.OUTPUT_BASE=${METPLUS_OUTPUT_DIR}/Aeronet

Which will read in the PointStat_fcstGFS_obsGDAS_UpperAir_MultiField_PrepBufr.conf configuration file first, and overwrite/append anything found in the second configuration file, Pointstat_AeronetAOD.conf.

Taking a look at PointStat_fcstGFS_obsGDAS_UpperAir_MultiField_PrepBufr.conf, specifically the variable fields, we quickly see why there's an error in the METplus run:

BOTH_VAR1_NAME = TMP
BOTH_VAR1_LEVELS = P1000, P925, P850, P700, P500, P400, P300, P250, P200, P150, P100, P50, P20, P10

BOTH_VAR2_NAME = RH
BOTH_VAR2_LEVELS = P1000, P925, P850, P700, P500, P400, P300

BOTH_VAR3_NAME = UGRD
BOTH_VAR3_LEVELS = P1000, P925, P850, P700, P500, P400, P300, P250, P200, P150, P100, P50, P20, P10

BOTH_VAR4_NAME = VGRD
BOTH_VAR4_LEVELS = P1000, P925, P850, P700, P500, P400, P300, P250, P200, P150, P100, P50, P20, P10

BOTH_VAR5_NAME = HGT
BOTH_VAR5_LEVELS = P1000, P950, P925, P850, P700, P500, P400, P300, P250, P200, P150, P100, P50, P20, P10

Because these variables utilize the BOTH prefix, they will only be overridden by the contents of Pointstat_AeronetAOD.conf if it also uses the BOTH prefix and accompanying variable values. But Pointstat_AeronetAOD.conf uses the FCST and OBS prefixes, which means the final METplus configuration file would look something like

BOTH_VAR1_NAME = TMP
BOTH_VAR1_LEVELS = P1000, P925, P850, P700, P500, P400, P300, P250, P200, P150, P100, P50, P20, P10

BOTH_VAR2_NAME = RH
BOTH_VAR2_LEVELS = P1000, P925, P850, P700, P500, P400, P300

BOTH_VAR3_NAME = UGRD
BOTH_VAR3_LEVELS = P1000, P925, P850, P700, P500, P400, P300, P250, P200, P150, P100, P50, P20, P10

BOTH_VAR4_NAME = VGRD
BOTH_VAR4_LEVELS = P1000, P925, P850, P700, P500, P400, P300, P250, P200, P150, P100, P50, P20, P10

BOTH_VAR5_NAME = HGT
BOTH_VAR5_LEVELS = P1000, P950, P925, P850, P700, P500, P400, P300, P250, P200, P150, P100, P50, P20, P10

FCST_VAR1_NAME = AOTK
FCST_VAR1_LEVELS = A0
FCST_VAR1_THRESH =  >=0.1, >=0.2, >=0.4, >=0.6, >=0.8, >=1.0
FCST_VAR1_OPTIONS = GRIB2_process=62000;GRIB2_ipdtmpl_index = [ 2, 10 ]; GRIB2_ipdtmpl_val = [ 62000, 545 ]

OBS_VAR1_NAME = AOD
OBS_VAR1_LEVELS = L550
OBS_VAR1_OPTIONS = GRIB1_ptv  = 141
OBS_VAR1_THRESH = >=0.1, >=0.2, >=0.4, >=0.6, >=0.8, >=1.0

You can confirm this behavior in the metplus_final.conf file, in /scratch1/NCEPDEV/stmp2/Partha.Bhattacharjee/METplus/Scripts/Aeronet.

To get the desired behavior, I would suggest copying the PointStat_fcstGFS_obsGDAS_UpperAir_MultiField_PrepBufr.conf file to a local area, then modifying the copied configuration file to only run the PB2NC section,

PROCESS_LIST = PB2NC

. That should ignore the BOTH variables part (since PB2NC doesn't utilize those variables) and run with that error. If you need some of the settings from PointStat_fcstGFS_obsGDAS_UpperAir_MultiField_PrepBufr.conf in order for Pointstat_AeronetAOD.conf to run, you'll stil want to create a local copy of PointStat_fcstGFS_obsGDAS_UpperAir_MultiField_PrepBufr.conf and simply remove any uses of the BOTH prefix.

[perthsb]

Thanks @j-opatz . Alternatively, I can use what in PointStat_fcstGFS_obsGDAS_UpperAir_MultiField_PrepBufr.conf like
PROCESS_LIST = PB2NC, PointStat
to PROCESS_LIST = ASCII2NC, PointStat

and modify my conf following the way it mentioned in PointStat_fcstGFS_obsGDAS_UpperAir_MultiField_PrepBufr.conf.
One question is, does ascii2nc convert all input files to nc if Station list is not mentioned in configure file ? And similarly if Station list is not mentioned in Pointstat will it do analysis for all stations mentioned in the directory ?
Thanks

[j-opatz]

You are correct in your modification of the PointStat_fcstGFS_obsGDAS_UpperAir_MultiField_PrepBufr.conf PROCESS_LIST option as well, good to point that out.

As for your question on stations, ASCII2NC will read all stations that are in the input file and write them out to the resulting netCDF file. If you'd like to filter them, ASCII2NC_MASK_SID will allow you to pass a file that defines a filter.

PointStat will also read in all of the stations that are in the netCDF input file, assuming that they have valid data and are under the same variable name in the file. You have the ability to be inclusive/exclusive by station ID (sid) with POINT_STAT_MASK_SID. PointStat also allows additional filtering with the sid_inc and sid_exc lists from MET, which can be accessed in METplus by the FCST_VAR<n>_OPTIONS and OBS_VAR<n>_OPTIONS.

[j-opatz]

Hello,

It's been a while since we last touched base about this issue. We like to mark Discussions as closed if they have no activity and/or if the issue has been solved.

Was the provided guidance helpful in solving the configuration settings and how ASCII2NC operates? If so, please select one of the discussions as the answer and I'll close this out. Otherwise, feel free to write back and let us know if there's more in this topic that you need help on.