rejected due to obs_var_name

[perthsb]

Hi,
I am using pointstat and getting 0 matched pair largely because of "rejected obs var name". I think I have put everything required for obs_var in the config file, but not sure if something else is required. Logfile, Input raw data, output netcdf, configure file on Hera here :
/scratch1/NCEPDEV/stmp2/Partha.Bhattacharjee/METplus/Scripts/SurfaceObs/metplus.log.20240430222147

Thanks.

[jprestop]

Hi @perthsb. Thanks for sending your log file location. I will look into this and follow up with you once I have more information.

[jprestop]

Hi @perthsb.

Looking at this case, for example:

/apps/ops/prod/libs/intel/19.1.3.304/met/11.0.2/bin/point_stat -v 3 /lfs/h2/emc/physics/noscrub/partha.bhattacharjee/RRFSSD_Eval/Airnow/pcpcomb/20240429/pm25tot_2024042900.nc /lfs/h2/emc/physics/noscrub/partha.bhatta\
charjee/RRFSSD_Eval/Airnow/Airnowobs/20240428/Airnow_HrlyPM25_2024042823.nc /apps/ops/para/libs/intel/19.1.3.304/metplus/5.0.1/parm/met_config/PointStatConfig_wrapped -outdir /lfs/h2/emc/physics/noscrub/partha.bhattacharjee/RRFSSD_Eval/Airnow/pointstat/202404\
2900.stat

I see:

DEBUG 2: Number of matched pairs   = 0
DEBUG 2: Observations processed    = 3922
DEBUG 2: Rejected: station id      = 0
DEBUG 2: Rejected: obs var name    = 2623
...

I can see that you're looking for the obs name of PM25:

DEBUG 2: Processing PMTF(*,*) versus PM25/A1, for observation type AIRNOW_HOURLY_AQOBS, over region FULL, for interpolation method BILIN(4), using 0 matched pairs.

Running the following command, I can see that PM25 is the second value for obs_var, with an index of 1:

[Julie.Prestopnik@hfe04 Asciitonc]$ ncdump -fc  -v obs_var Airnow_HrlyPM25_2024042823.nc |more
...
data:

 obs_var =
  "OZONE",  // obs_var(0,39)
    "PM25",  // obs_var(1,39)
    "NO2",  // obs_var(2,39)
    "PM10",  // obs_var(3,39)
    "CO",  // obs_var(4,39)
    "SO2";  // obs_var(5,39)
    }

Looking at the obs_vid variable (index of variable names at var_name), I want to determine how many instances there are with a value equal to 1, which shows how many values there are for PM25. So, I created a file called out.txt which will output all of the values in the obs_vid array using the following command:

ncdump -fc -v obs_vid Airnow_HrlyPM25_2024042823.nc >& out.txt

If you'd like, you can see this file at /scratch1/BMC/dtc/Julie.Prestopnik/tmp/scratch1/NCEPDEV/stmp2/Partha.Bhattacharjee/METplus/Scripts/SurfaceObs/Asciitonc/out.txt.

To find a total of only the values with 1, and therefore the total number of PM25 observations, I ran the following command:

[Julie.Prestopnik@hfe04 Asciitonc]$ grep "1," out.txt | wc -l
1299

which gave me 1299. Going back to the log output, we see there were 3922 observations processed and 2623 were rejected due to the obs var name. 3922 observations processed minus 1299 PM25 values gives us 2623 observations that do not have the name PM25 just as we would expect.

The remaining observations were rejected due to the valid time:

DEBUG 2: Rejected: valid time      = 1299

Please take a closer look at the valid times and ensure you're looking for what you expect.

I hope that helps. Please let us know if you have any further questions.

[perthsb]

Thanks @jprestop .Is there any other setting needed in config file to limit the rejection of obs_var_name

[jprestop]

@perthsb I don't think I understand what you are asking. Can you please explain further?

[perthsb]

@jprestop for some reason I could read only part of your answer and I can read the full answer now as I switched browser. Thanks for your explanation, I am closing the ticket.