Calculating FSS (processing obs accordingly) with NEP method

[jwolff-ncar]

I have modified the GridStat_APCP01h_prob.conf file to have:
...
FCST_VAR5_NAME = APCP_01_A01_ENS_NEP_gt0.0_NBRHD25
FCST_VAR5_LEVELS = A01
FCST_VAR5_THRESH = ==0.1

OBS_VAR5_NAME = APCP
OBS_VAR5_LEVELS = A01
OBS_VAR5_THRESH = >0.0
OBS_VAR5_OPTIONS = nbrhd = { field = OBS; shape = SQUARE; width = [ 5 ]; }
...

but when this is run we are getting this error:

ERROR :
ERROR : Dictionary::lookup_double() -> lookup failed for name "vld_thresh"
ERROR :

This conf file work appropriately prior to adding the OBS_VAR_OPTIONS line so I am hoping it is a simple syntax issue!
Thanks for your assistance.
Jamie

[JohnHalleyGotway]

@jwolff-ncar, there's a couple details we need to discuss.

1. My guess about this immediate parsing error is that it's happening because the “nbrhd” dictionary setting does NOT include a setting for vld_thresh as seen in this line of the default config file. I'd try adding vld_thresh to that line and rerunning. All of the details as to when settings from dictionaries are or are not overridden is not entirely clear to me. Probably should figure that out at some point!

2. Based on your existing settings, I am guessing that you'd like to process the APCP_01_A01_ENS_NEP_gt0.0_NBRHD25 data in 2 ways...

• As a field of probability values for which you compute probabilistic stats.
• As a field of scalar values for which you compute neighborhood stats, including FSS.

The key detail here is that neighborhood stats are computed for scalar input data, NOT probability data. And probabilistic stats are computed for probability input data, NOT scalar data.

The punchline is that you'll need to list the same variable twice, once with config settings to process it as probabilities and once with config settings to process it as scalars. @georgemccabe can you please review the following settings and correct any mistakes I've made?

# Process as probabilities
FCST_VAR5_NAME = APCP_01_A01_ENS_NEP_gt0.0_NBRHD25
FCST_VAR5_LEVELS = A01
FCST_VAR5_THRESH = ==0.1
FCST_VAR5_OPTIONS = prob = TRUE;

OBS_VAR5_NAME = APCP
OBS_VAR5_LEVELS = A01
OBS_VAR5_THRESH = >0.0

# Process as scalars for neighborhood methods.
# Note that the number of forecast and obs thresholds must match
# but won’t actually be applied to NBRCNT outputs with "nbrhd.field = OBS;"
FCST_VAR6_NAME = APCP_01_A01_ENS_NEP_gt0.0_NBRHD25
FCST_VAR6_LEVELS = A01
FCST_VAR6_THRESH = >0.0

OBS_VAR6_NAME = APCP
OBS_VAR6_LEVELS = A01
OBS_VAR6_THRESH = >0.0
OBS_VAR6_OPTIONS = nbrhd = { field = OBS; shape = SQUARE; width = [ 5 ]; vld_thresh = 1.0; } output_flag = FALSE; output_flag = { nbrcnt = STAT; }

[georgemccabe]

@JohnHalleyGotway, I don't see anything obviously incorrect with those settings, as long as output_flag can be set that way on a field-by-field basis.

[JohnHalleyGotway]

How to process this type of Gen-Ens-Prod output data using FSS in Grid-Stat:

1. ENS_FREQ

• Between 0 and 1 and indicates the fraction of ensemble members at which the event is occurring.
• Configure nhbrd = { field = OBS; shape = SQUARE; width = 1; vld_thresh = 1.0; } and prob = FALSE;
• Computes FSS using forecast [0, 1] vs observation of 0 or 1.

2. ENS_NEP

• Between 0 and 1 and indicates the fraction of a "cylinder" of points at which the event is occurring. Each slice is a spatial neighborhood for each member.
• Configure nhbrd = { field = OBS; shape = SQUARE; width = N; vld_thresh = 1.0; } and prob = FALSE;
• Computes FSS using forecast [0, 1] vs observation [0, 1].
• N matches how ENS_NEP was defined.

3. ENS_NMEP
   Do not recommend computing FSS for ENS_NMEP outputs since it is difficult or impossible to process the obs for a fair comparison.

[JohnHalleyGotway]

@jwolff-ncar and @michelleharrold, please feel free to select one of these responses as being a good answer to the original question. Once you do, I'll lock conversation on it to prevent future posts.

[jwolff-ncar]

I have marked the first one as the answer given it answered the original question. Though the second response does provide good additional information. One clarification in the second response - the nhbrd setting should be included in the OBS_VAR_OPTIONS while the prob=FALSE should be set in the FCST_VAR_OPTIONS