Point_Stat: MET/12.0.0-beta4 and METplus/6.0.0-beta4 could not find matched pairs

[SamiraArdani-NOAA]

Hi,
I am running PointStat analysis using MET/METplus. I have used NDBC data for the Great Lakes region to perform verifications for GLWU (Great Lakes Waves on Unstructured grid). The ASCII2NC has been used to generate one .nc file from NDBC data for this analysis. I got zero matched pairs when I ran PointStat analysis. The log file does not show any error messages and the METplus generates the stat files but the stat files have no content (they only have headers).

Here is part of the log file that shows the issue:
DEBUG 2: Processing HTSGW/L0 versus WVHT/L0, for observation type NDBC_STANDARD, over region GreatLakes, for interpolation method NEAREST(1), using 0 matched pairs.
DEBUG 2: Number of matched pairs = 0
DEBUG 2: Observations processed = 43484
DEBUG 2: Rejected: station id = 0
DEBUG 2: Rejected: obs var name = 40378
DEBUG 2: Rejected: valid time = 0
DEBUG 2: Rejected: bad obs value = 2329
DEBUG 2: Rejected: off the grid = 0
DEBUG 2: Rejected: topography = 0
DEBUG 2: Rejected: level mismatch = 777
DEBUG 2: Rejected: quality marker = 0
DEBUG 2: Rejected: message type = 0
DEBUG 2: Rejected: masking region = 0
DEBUG 2: Rejected: bad fcst value = 0
DEBUG 2: Rejected: bad climo mean = 0
DEBUG 2: Rejected: bad climo stdev = 0
DEBUG 2: Rejected: mpr filter = 0
DEBUG 2: Rejected: duplicates = 0

To troubleshoot this issue, I followed these steps:
✔️ wgrib2 glwu.grlc_2p5km.20240903.t01z.f000.grib2 (one sample forecast file)
1:0:d=2024090301:WIND:surface:anl:
2:42881:d=2024090301:WDIR:surface:anl:
3:103682:d=2024090301:UGRD:surface:anl:
4:146527:d=2024090301:VGRD:surface:anl:
5:189759:d=2024090301:HTSGW:surface:anl:
6:213980:d=2024090301:PERPW:surface:anl:
7:248662:d=2024090301:DIRPW:surface:anl:
8:316823:d=2024090301:WVHGT:surface:anl:
9:337527:d=2024090301:WVPER:surface:anl:
10:366663:d=2024090301:WVDIR:surface:anl:
✔️ plot_data_plane glwu.grlc_2p5km.20240903.t01z.f000.grib2 test.png 'name="HTSGW"; level="L0";'
✔️ ncdump -v obs_var ndbc.20240903.nc (generated .nc file using ASCII2NC)
data:

obs_var =
"WDIR",
"WSPD",
"GST",
"WVHT",
"DPD",
"APD",
"MWD",
"PRES",
"ATMP",
"WTMP",
"DEWP",
"VIS",
"PTDY",
"TIDE" ;
}
❌ plot_data_plane ndbc.20240903.nc test_obs.png 'name="WVHT"; level="L0";'
DEBUG 1: Opening data file: ndbc.20240903.nc
ERROR :
ERROR : plot_data_plane -> trouble getting field "name="WVHT"; level="L0";" from file "ndbc.20240903.nc"
ERROR :
I tested it with level="Z0" but no success.
I will send the sample files along with log file and the METplus .config file for you to review. Any help is greatly appreciated.
@malloryprow @AliciaBentley-NOAA @PerryShafran-NOAA @AndrewBenjamin-NOAA @sbanihash for the awareness.

[DanielAdriaansen]

Greetings @SamiraArdani-NOAA, thanks for the details and for sharing your files.

Before getting into the matched pair question, I'd like to point out that the GLWU data file you sent is not an unstructured grid. It is on a Lambert Conformal grid:

1:0:grid_template=30:winds(N/S):
        Lambert Conformal: (581 x 361) input WE:SN output WE:SN res 0
        Lat1 41.530708 Lon1 267.363984 LoV 265.000000
        LatD 25.000000 Latin1 25.000000 Latin2 25.000000
        LatSP -90.000000 LonSP 0.000000
        North Pole (581 x 361) Dx 2539.703000 m Dy 2539.703000 m 

Also, since this file ndbc.20240903.nc contains point observations and not gridded data, you'll need to use the plot_point_obs utility rather than plot_data_plane. I used plot_point_obs with this command:

plot_point_obs ncbc.20240903.nc buoy_obs.ps -v 5 

to produce this image:

Then, I used ncdump to more closely inspect your ndbc.20240903.nc file. This command:
ncdump -v obs_lvl ndbc.20240903.nc
Shows that all of the level values for all observations are missing data.

If we look at the height instead:
ncdump -v obs_hgt ndbc.20240903.nc
I see many valid values:

obs_hgt = 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 183, 
    183, 176, 176, 176, 176, 176, 176, 176, 176, 176, 176, 176, 176, 176,

If those are expected height values for your observations, then you can set this:
OBS_VAR1_LEVELS=Z183, or whichever of those levels you wish. When I use Z183, I get 4 matched pairs for WVHGT, 8 matched pairs for WSPD, and 0 matched pairs for APD.

However, if these observations all represent the surface or some level that you'd like to consider the same, you can set:
OBS_VAR1_LEVELS=Z0-500 which will use all observations with a hgt value between 0 and 500.

When I use Z0-500, I get 40 matched pairs for WVHGT, 110 matched pairs for WSPD, and 1 matched pair for APD.

Hopefully this helps!

[SamiraArdani-NOAA]

Hi @DanielAdriaansen,
Thanks for your helpful suggestions! I changed the config file based on your suggestions (OBS_VAR1_LEVELS=Z0-500, OBS_VAR2_LEVELS=Z0-500, OBS_VAR3_LEVELS=Z0-500) and the PointStat is now finding the matched pairs and is generating outputs for WVHT and WSPD which is great, however, I still get zero matched pair for APD.

[DanielAdriaansen]

Hi @SamiraArdani-NOAA,

Can you please share the log file?

I re-did my test, and I am still getting 1 matched pair for APD.

Note that because I did not have your mask and climo files, I removed all of those settings. Is it possible your masks or some other settings I am not using is removing the 1 APD pair I am getting? Here is the observation I am using:

DEBUG 9: For PERPW/L0 versus APD/Z500-0, for observation type NDBC_STANDARD, over region FULL, for interpolation method NEAREST(1), using observation:
DEBUG 9:    NDBC_STANDARD 45194 20240903_220000 45.80400 84.79200 177.42999 APD -9999.00000 177.42999 NA 2.60000

If that site (ID=45194) with lat=45.804 lon=-84.792 is outside your masking regions, then that may explain it.

You can also set:
LOG_POINT_STAT_VERBOSITY=10 to see why point_stat is rejecting all of the APD observations.

Feel free to share the latest log file with that verbosity setting and we can take a closer look if you'd like.

[DanielAdriaansen]

I forgot to also mention, that there were lots of APD variables listed in your obs file, but nearly all of the actual observations were missing data (-9999.). Maybe this is expected but if not I wanted to share what I saw.

[SamiraArdani-NOAA]

@DanielAdriaansen, I just sent the log file and the new config file (after changing the OBS_VAR_LEVELS). I agree with you that having different masks and climo can cause this difference in our results. So, in the new config file that I just sent, I removed one of the masks because I thought it may be the reason for this APD zero matched pairs but it did not resolve this issue.

[DanielAdriaansen]

Hi @SamiraArdani-NOAA thanks for the log file and config file.

You are not using the same time I am. I am using a valid time of:
at valid time: 2024-09-03 01:00

And you are using data from today, so our tests are not the same and I can't tell you if there is anything wrong or not without the new data. My suspicion is that there are no valid APD observations for the times you're running for.

Can you run for more times to see if you encounter times where it does find APD matched pairs? Maybe the APD observations are infrequent?

To really dig in you can run with this setting:

POINT_STAT_LOG_VERBOSITY=10

to see the reasons why the APD obs are being rejected for your current time, which will report each APD observation that point_stat encounters and the reason it rejects it, or whether it accepts it.

[SamiraArdani-NOAA]

Thanks, @DanielAdriaansen. I agree with you and I think that APDs are infrequent but I will run with Verbosity 10 and for more times to have a final conclusion. Thanks again for your time and your suggestions.