Climo Time Errors

[craigr3]

When processsing climo data in point_stat, I have been getting error when it tries to process the ncep climo file to interpolate a value to the forecast lead time. The error is attached.

The ncep climo file monthly data every 6 hours. I don't seem to get the error for forecasts divisible by 6 but for other forecast hours I do.

[j-opatz]

Hello,

And thank you for your question. In order to get a better understanding of why this particular error is occurring, would you be able to share the configuration file you're using? That might provide the missing piece to this error puzzle.

[craigr3]

Here is the config fileSent from my Verizon, Samsung Galaxy smartphone -------- Original message --------From: j-opatz ***@***.***> Date: 10/21/22 11:24 AM (GMT-06:00) To: dtcenter/METplus ***@***.***> Cc: craigr3 ***@***.***>, Author ***@***.***> Subject: Re: [dtcenter/METplus] Climo Time Errors (Discussion #1879) Hello, And thank you for your question. In order to get a better understanding of why this particular error is occurring, would you be able to share the configuration file you're using? That might provide the missing piece to this error puzzle. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***> //////////////////////////////////////////////////////////////////////////////// // // Point-Stat configuration file. // // For additional information, see the MET_BASE/config/README file. // //////////////////////////////////////////////////////////////////////////////// // // Output model name to be written // model = "${MODEL}"; // // Output description to be written // May be set separately in each "obs.field" entry // desc = "NA"; //////////////////////////////////////////////////////////////////////////////// // // Verification grid // regrid = { to_grid = NONE; method = NEAREST; width = 1; vld_thresh = 0.5; shape = SQUARE; } //////////////////////////////////////////////////////////////////////////////// // // May be set separately in each "field" entry // censor_thresh = []; censor_val = []; cat_thresh = [ NA ]; cnt_thresh = [ NA ]; cnt_logic = UNION; wind_thresh = [ NA ]; wind_logic = UNION; eclv_points = 0.05; // // Forecast and observation fields to be verified // fcst = { convert(x) = ${FCONVERT}; lead_time = "${FCST}"; censor_thresh = [${MDLFILTER}]; censor_val = [-9999]; field = [ { name = "${FNAME}"; level = [ "${FLEVEL}" ]; cat_thresh = [>15, >25, >35, >50]; } ]; } obs = { convert(x) = ${OCONVERT}; censor_thresh = [${MDLFILTER}]; censor_val = [-9999]; field = [ { name = "${ONAME}"; level = [ "${OLEVEL}" ]; cat_thresh = [>15, >25, >35, >50]; } ]; }; //////////////////////////////////////////////////////////////////////////////// // // Point observation filtering options // May be set separately in each "obs.field" entry // message_type = [ "ADPSFC" ]; sid_exc = []; sid_inc = []; obs_quality_inc= ${OBQTY}; obs_quality_exc= []; duplicate_flag = UNIQUE; obs_summary = NEAREST; obs_perc_value = 50; //////////////////////////////////////////////////////////////////////////////// // // Climatology data // climo_mean = { file_name = "${CLIMO}"; convert(x) = ${CCONVERT}; field = [ { name = "${CNAME}"; level = ["${CLEVEL}"]; } ]; regrid = { method = NEAREST; width = 1; vld_thresh = 0.5; shape = SQUARE; } time_interp_method = DW_MEAN; day_interval = 31; hour_interval = 6; } climo_stdev = climo_mean; climo_stdev = { file_name = []; } climo_cdf_bins = 1; //////////////////////////////////////////////////////////////////////////////// // // Point observation time window // obs_window = { beg = -1800; end = 1800; } //////////////////////////////////////////////////////////////////////////////// // // Verification masking regions // mask = { grid = [ "FULL" ]; poly = ["/lustre/active/nwp306/proj-shared/masks/USC.nc", "/lustre/active/nwp306/proj-shared/masks/USE.nc", "/lustre/active/nwp306/proj-shared/masks/USN.nc", "/lustre/active/nwp306/proj-shared/masks/USS.nc", "/lustre/active/nwp306/proj-shared/masks/USP.nc", "/lustre/active/nwp306/proj-shared/masks/USA.nc", "/lustre/active/nwp306/proj-shared/masks/conus_USN.nc", "/lustre/active/nwp306/proj-shared/masks/alaska_USN.nc", "/lustre/active/nwp306/proj-shared/masks/swa_usc.nc", "/lustre/active/nwp306/proj-shared/masks/kor_USP.nc", "/lustre/active/nwp306/proj-shared/masks/eur_USE.nc", "/lustre/active/nwp306/proj-shared/masks/afgpak_USC.nc"]; sid = ${SID}; } //////////////////////////////////////////////////////////////////////////////// // // Confidence interval settings // ci_alpha = [ 0.05 ]; boot = { interval = PCTILE; rep_prop = 1.0; n_rep = 0; rng = "mt19937"; seed = ""; } //////////////////////////////////////////////////////////////////////////////// // // Interpolation methods // interp = { vld_thresh = 1.0; shape = SQUARE; type = [ { method = ${INTMETHOD}; width = ${INTWIDTH}; } ]; } //////////////////////////////////////////////////////////////////////////////// // // HiRA verification method // hira = { flag = FALSE; width = [ 2, 3, 4, 5 ]; vld_thresh = 1.0; cov_thresh = [ ==0.25 ]; shape = SQUARE; } //////////////////////////////////////////////////////////////////////////////// // // Statistical output types // output_flag = { fho = NONE; ctc = BOTH; cts = BOTH; mctc = NONE; mcts = NONE; cnt = BOTH; sl1l2 = BOTH; sal1l2 = NONE; vl1l2 = NONE; val1l2 = NONE; pct = NONE; pstd = NONE; pjc = NONE; prc = NONE; eclv = NONE; mpr = STAT; } //////////////////////////////////////////////////////////////////////////////// rank_corr_flag = FALSE; tmp_dir = "/lustre/active/nwp306/proj-shared/tmp"; output_prefix = "${MODEL}_${THEATER}_SFC"; version = "V10.1"; ////////////////////////////////////////////////////////////////////////////////

[j-opatz]

After reviewing the configuration file, I believe part of the error is in your chosen interpolation method for the climatology data. According to the configuration file you provided,

time_interp_method = DW_MEAN;

According to the MET user's guide, time_interp_method ultimately determines how the climatology data is interpreted to the valid time of the forecast. Using DW_MEAN will provide a "...linear interpolation in time of data before and after". As noted in the error message, the climatology field times are 20221015_000000 and 20221015_060000, while the forecast valid time is 20221010_030000 which occurs 5 days before your climatology time bracket. And because the DW_MEAN method requires the valid time to be between those climatology times, it errors out. You may want to utilize NEAREST method instead,

time_interp_method = NEAREST; 

but there is another issue to discuss for a climatology setting.

You mentioned in your question,

I don't seem to get the error for forecasts divisible by 6 but for other forecast hours I do.

If you look at the hour_interval variable in your configuration file, you'll notice it's set to 6:

hour_interval = 6;

In the user's guide, it describes hour_interval as "...an integer specifying the spacing in hours of the climatology data for each day." Taking a look at your output screen, I noted that you've got a few climatology fields being rejected due to this conflict:

>DEBUG 3: Skipping the 19591014_180000 "WIND/Z10" climatology field since the time offset (54000 seconds) >= the "hour_interval" entry (21600 seconds).

So although you've said the climatology files are every 6 hours, MET is seeing offsets of 54000 seconds (15 hours), 32400 seconds (9 hours), and 21600 seconds (6 hours). If this is in error and you want all of these climatologies read in, you can disable the hour offset check by setting it equal to NA,

hour_interval = NA;

and it should read them successfully. But as mentioned above, the interpolation method you've chosen will result in errors due to timestamps of the climatology fields and your forecast valid time.

[j-opatz]

Hello,

I was wondering if you had any success with updating your climatology intake. If you did, we would appreciate if you would mark one of the comments as an answer so we can close out the discussion. If you still need some help on this, please let us know where the issue stands and we'll continue to try and troubleshoot.

[craigr3]

Sorry for the delay in response - will give that a try.  Okay, after setting time_interp_method to nearest and hour_interval to na I get the error " can't find climo data before/after date 19700101_172140. Another question - I am using the mpr_column filtering to remove abs(fcst- obs) differences out of range.  I placed the two mpr lines in the fcst area after censor thresh and no filtering seemed to occur (I set the mpr_thresh to <=0 so all pairs should have been filtered but none were.  Did I place these two line in the wrong place in the point_stat config file?

[craigr3]

Okay, after setting time_interp_method to nearest and hour_interval to na I get the error " can't find climo data before/after date 19700101_172140.

[craigr3]

I got the mpr_thresh thing figured out.Bob

[JohnHalleyGotway]

Bob, John Opatz, with whom you've been working, is out of the country for month or so. Just wanted to check in on this discussion to see if you're still having issue with specifying climatology data?

Are you still getting that can't find climo data before/after date 19700101_172140 error message?

[craigr3]

Yes.  It works if my forecasts are every 6 hours but errors if they are every two hours.

[JohnHalleyGotway]

Hmm, OK, so for each valid time evaluated, the MET library code looks for climatology data with timestamps before/after that valid time. And it uses the hour_interval to know how far to search. With hour_interval = 6 and a forecast valid at 17Z, it'll search for climo data between 11Z and 23Z to find data before/after the forecast valid time. Then it'll use the time_interp_method setting to decide what to do next... use data from NEAREST climo time or interpolate to the actual valid time.

The error message you're getting indicates that it isn't finding climo data within the time window prior to that valid time.

At this point, I think I'd need to actually see the data that's causing trouble to help further. If you'd like to package up some fcst/obs/climo data along with your METplus config and log files and post it somewhere, I could take a closer look.

[craigr3]

I will send it to you in DOD Safe.

[craigr3]

What is your email if I can't find it?

[JohnHalleyGotway]

Bob, I received that file via DOD Safe and will take a closer look at debugging the behavior you describe on Monday.

[craigr3]

Cool. Thanks.

[j-opatz]

Hello,

And thank you for your patience during my absence (and as we tried to figure this out!).

I ran some test data against the climatology file you provided and was unable to replicate the issue. Let me post my configuration file for you:

////////////////////////////////////////////////////////////////////////////////
//
// Point-Stat configuration file.
//
// For additional information, please see the MET User's Guide.
//
////////////////////////////////////////////////////////////////////////////////

//
// Output model name to be written
//
model = "GDAS";

//
// Output description to be written
// May be set separately in each "obs.field" entry
//
desc = "NA";

////////////////////////////////////////////////////////////////////////////////

//
// Verification grid
// May be set separately in each "field" entry
//
regrid = {
   to_grid    = NONE;
   method     = NEAREST;
   width      = 1;
   vld_thresh = 0.5;
   shape      = SQUARE;
}
////////////////////////////////////////////////////////////////////////////////

//
// May be set separately in each "field" entry
//
censor_thresh  = [];
censor_val     = [];
mpr_column     = [];
mpr_thresh     = [];
cat_thresh     = [ NA ];
cnt_thresh     = [ NA ];
cnt_logic      = UNION;
wind_thresh    = [ NA ];
wind_logic     = UNION;
eclv_points    = 0.05;
hss_ec_value   = NA;
rank_corr_flag = FALSE;

//
// Forecast and observation fields to be verified
//
fcst = {
   file_type = GRIB2;
   set_attr_valid = "20220516_010000";
   field = [
      {
        name       = "TMP";
        level      = [ "Z0" ];
//        cat_thresh = [ >80.0 ];
      }
   ];

}
obs = {
field = [
   {
     name = "TMP";
     level = ["Z0"];
   }
];
}
////////////////////////////////////////////////////////////////////////////////

//
// Point observation filtering options
// May be set separately in each "obs.field" entry
//
message_type   = [ "ANYSFC" ];
sid_inc        = [];
sid_exc        = [];
obs_quality_inc  = [];
obs_quality_exc  = [];
duplicate_flag = NONE;
obs_summary    = NONE;
obs_perc_value = 50;

//
// Mapping of message type group name to comma-separated list of values.
//
message_type_group_map = [
   { key = "SURFACE"; val = "ADPSFC,SFCSHP,MSONET";               },
   { key = "ANYAIR";  val = "AIRCAR,AIRCFT";                      },
   { key = "ANYSFC";  val = "ADPSFC,SFCSHP,ADPUPA,PROFLR,MSONET"; },
   { key = "ONLYSF";  val = "ADPSFC,SFCSHP";                      },
   { key = "LANDSF";  val = "ADPSFC,MSONET";                      },
   { key = "WATERSF"; val = "SFCSHP";                             }
];

////////////////////////////////////////////////////////////////////////////////

//
// Climatology data
//
climo_mean = {

   file_name = ["/d1/projects/METplus/discussions/1879/pgba_mean.19590515"];
   field     = [
        {
          name = "TMP";
          level = ["R58"];
}
];

   regrid = {
      method     = NEAREST;
      width      = 1;
      vld_thresh = 0.5;
      shape      = SQUARE;
   }

   time_interp_method = NEAREST;
   day_interval       = 31;
   hour_interval      = NA;
}

climo_stdev = climo_mean;
climo_stdev = {
   file_name = [];
}
//
// May be set separately in each "obs.field" entry
//
climo_cdf = {
   cdf_bins    = 1;
   center_bins = FALSE;
   write_bins  = TRUE;
   direct_prob = FALSE;
}

////////////////////////////////////////////////////////////////////////////////

//
// Land/Sea mask
// For LANDSF message types, only use forecast grid points where land = TRUE.
// For WATERSF message types, only use forecast grid points where land = FALSE.
// land_mask.flag may be set separately in each "obs.field" entry.
//
land_mask = {
   flag      = FALSE;
   file_name = [];
   field     = { name = "LAND"; level = "L0"; }
   regrid    = { method = NEAREST; width = 1; }
   thresh    = eq1;
}

//
// Topography
// For SURFACE message types, only use observations where the topo - station
// elevation difference meets the use_obs_thresh threshold.
// For the observations kept, when interpolating forecast data to the
// observation location, only use forecast grid points where the topo - station
// difference meets the interp_fcst_thresh threshold.
// topo_mask.flag may be set separately in each "obs.field" entry.
//
topo_mask = {
   flag               = FALSE;
   file_name          = [];
   field              = { name = "TOPO"; level = "L0"; }
   regrid             = { method = BILIN; width = 2; }
   use_obs_thresh     = ge-100&&le100;
   interp_fcst_thresh = ge-50&&le50;
}

////////////////////////////////////////////////////////////////////////////////

//
// Point observation time window
// May be set separately in each "obs.field" entry
//
obs_window = {
   beg = -540000000;
   end =  540000000;
}

////////////////////////////////////////////////////////////////////////////////

//
// Verification masking regions
// May be set separately in each "obs.field" entry
//
mask = {
   grid  = [ "FULL" ];
   poly  = [];
   sid   = [];
   llpnt = [];
}

////////////////////////////////////////////////////////////////////////////////
//
// Confidence interval settings
// May be set separately in each "obs.field" entry
//
ci_alpha  = [ 0.05 ];

boot = {
   interval = PCTILE;
   rep_prop = 1.0;
   n_rep    = 0;
   rng      = "mt19937";
   seed     = "";
}

////////////////////////////////////////////////////////////////////////////////
//
// Interpolation methods
// May be set separately in each "obs.field" entry
//
interp = {
   vld_thresh = 1.0;
   shape      = SQUARE;

   type = [
      {
         method = NEAREST;
         width  = 1;
      }
   ];
}

////////////////////////////////////////////////////////////////////////////////
//
// HiRA verification method
// May be set separately in each "obs.field" entry
//
hira = {
   flag            = FALSE;
   width           = [ 2, 3, 4, 5 ];
   vld_thresh      = 1.0;
   cov_thresh      = [ ==0.25 ];
   shape           = SQUARE;
   prob_cat_thresh = [];
}

////////////////////////////////////////////////////////////////////////////////
//
// Statistical output types
// May be set separately in each "obs.field" entry
//
output_flag = {
   fho    = BOTH;
   ctc    = NONE;
   cts    = NONE;
   mctc   = NONE;
   mcts   = NONE;
   cnt    = NONE;
   sl1l2  = NONE;
   sal1l2 = NONE;
   vl1l2  = NONE;
   val1l2 = NONE;
   vcnt   = NONE;
   pct    = NONE;
   pstd   = NONE;
   pjc    = NONE;
   prc    = NONE;
   ecnt   = NONE; // Only for HiRA.
   orank  = NONE; // Only for HiRA.
   rps    = NONE; // Only for HiRA.
   eclv   = NONE;
   mpr    = NONE;
   seeps  = NONE;
   seeps_mpr = NONE;
}

////////////////////////////////////////////////////////////////////////////////
// Threshold for SEEPS p1 (Probability of being dry)

seeps_p1_thresh = NA;
////////////////////////////////////////////////////////////////////////////////

tmp_dir        = "/tmp";
output_prefix  = "";
version        = "V10.1.0";

////////////////////////////////////////////////////////////////////////////////

While some of the settings are noticeably different than yours (no conversions, thresholding, etc.), those should have no effect on the ability to read in climatology data.
I utilized the set_attr_valid variable to create a non-standard valid time to see if that affected the outcome. Regardless of the setting (I tried 0z, 1z, 2z, 3z, and 6z), Point-Stat was able to find a match between the forecast valid time and the climatology.
I also tried using different climatology entries (the above example was for Grib record number 58) and they were also accepted without issues.

For that same configuration file, here's the output:

 /d1/projects/MET/MET_releases/met-10.1.0/bin/point_stat gdas.t06z.pgrb2.0p25.f000 /d1/projects/MET/MET_releases/met-10.1.0/out/pb2nc/sample_pb.nc PointStatConfig_default 
DEBUG 1: Default Config File: /d1/projects/MET/MET_releases/met-10.1.0/share/met/config/PointStatConfig_default
DEBUG 1: User Config File: PointStatConfig_default
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=18446744072490621539
DEBUG 1: Forecast File: gdas.t06z.pgrb2.0p25.f000
DEBUG 1: Observation File: /d1/projects/MET/MET_releases/met-10.1.0/out/pb2nc/sample_pb.nc
DEBUG 2: 
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2: 
DEBUG 2: Reading data for TMP/Z0.
DEBUG 2: Processing masking regions.
DEBUG 2: Processing geography data.
DEBUG 2: Regridding the 19590515_000000 "TMP/R58" climatology field to the verification grid.
DEBUG 2: For TMP/Z0 found 1 forecast levels, 1 climatology mean levels, and 0 climatology standard deviation levels.
DEBUG 2: 
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2: 
DEBUG 2: Searching 87753 observations from 9396 messages.
DEBUG 2: 
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2: 
DEBUG 2: Processing TMP/Z0 versus TMP/Z0, for observation type ANYSFC, over region FULL, for interpolation method NEAREST(1), using 4291 matched pairs.
DEBUG 2: Computing Categorical Statistics.
DEBUG 2: 
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2: 
DEBUG 1: Output file: ./point_stat_000000L_20220516_010000V.stat
DEBUG 1: Output file: ./point_stat_000000L_20220516_010000V_fho.txt

Note that the climatology field was found in the file and matched to the valid forecast time. A slightly higher verbosity shows additional information:

DEBUG 3: For forecast valid at 20220516_010000, found 1 climatology field(s) with valid time(s): 20220515_000000
DEBUG 3: Found 1 climatology fields.

So even when I forced the forecast valid time ahead of the climatology, Point-Stat still had no issues.
Can you take a look at your configuration file and see if everything from your environmental variables is being filled in correctly and resembles the example I provided? It's admittedly harder to debug this when we only have the environmental variable-relative configuration.

If you don't see any glaring differences, we'll continue trying to figure this out with you. We might benefit from a copy of your up-to-date configuration file and a copy of the log file that results from your tool run.

[j-opatz]

Hello,

I'm checking in to see if this problem is resolved yet with the configurations that are presented above. It's been quite a while since any activity has happened on this discussion and we'll close this out if there are no further issues.

[craigr3]

I sent you guys data a couple weeks ago but was told you couldn't duplicate the error.  I will get back to that issue next week.  I would suggest closing and i will reopen if the problem reoccursThanks

[JohnHalleyGotway]

@craigr3, I wanted to mention that I just finished up a bugfix for MET version 11.0.0 related to climatology: dtcenter/MET#2412

In fact, @j-opatz is reviewing the corresponding Pull Requests right now.

The issue was with monthly climo data, not correctly interpolating between December 15th and January 15th climo data. It basically didn't handle the difference in year well. So Dec dates only processed Dec climo data, and Jan dates only processed Jan climo data. That wouldn't matter for nearest neighbor time interpolation but would matter for the "distance-weighted" (i.e. time-weighted) mean option.

The resulting logic is simpler and Debug(3) log messages have been improved. So hopefully you'll find using climo data more intuitive in the MET-11.0.1 bugfix release (not released yet) and any future versions.