issue reading nc4 files

[perthsb]

Hi,
I am trying to read both fcst and obs file which are in nc4 format and having issues reading input fcst files. However using plot_data_plane , I see METplus can read withot any error.
I have put them on Hera : /scratch1/NCEPDEV/global/Partha.Bhattacharjee/Test_Data
Any help on reading fcst file nc4 data.
Thanks

[georgemccabe]

Hi @perthsb,

Thank you for providing the files. It looks like your latest run ran without error and produced output. Were you able to resolve this issue already? If not, could you provide more information on what you are expecting to see compared to what you are seeing?

A side note:
I did notice that your output filenames contain the wildcard * character. Starting in METplus v5.1.0, that character is automatically replaced with all in the output file paths. If you'd like remove this text from the output path altogether, you can remove {CURRENT_FCST_LEVEL} from the output prefix value:

Change:
GRID_STAT_OUTPUT_PREFIX = {MODEL}_{CURRENT_FCST_NAME}_vs_{OBTYPE}_{CURRENT_OBS_NAME}_{CURRENT_FCST_LEVEL}

to
GRID_STAT_OUTPUT_PREFIX = {MODEL}_{CURRENT_FCST_NAME}_vs_{OBTYPE}_{CURRENT_OBS_NAME}

which will generate an output file named:

grid_stat_GEFSAERO_AOD_vs_GEOS5_TOTEXTTAU_000000L_20180103_060000V.stat

instead of:

grid_stat_GEFSAERO_AOD_vs_GEOS5_TOTEXTTAU_(0,*,*)_000000L_20180103_060000V.stat

[perthsb]

Thanks @georgemccabe . Apparently the issue was resolved when I added this line in the configure file
GRID_STAT_MET_CONFIG_OVERRIDES = file_type = NETCDF_NCCF;
If that line is commented, METplus find it difficult to read input nc4 file. Although I explicitly set file type options for FCST and OBS for nc4 files, but that was not enough. In the same directory I have tried a run with commenting that line and you can see the error (in hte latest log file).
Is there something else need to be set in the config file for reading nc4 files for METplus or this one is enough. Thanks

[georgemccabe]

The file_type variable can be set in the fcst and obs dictionaries. However, it cannot be set individually for each fcst.field or obs.field values, which explains why setting it in FCST_VAR1_OPTIONS and OBS_VAR1_OPTIONS did not give you the desired outcome. The plot_data_plane tool only reads a single field, so that is why you were able to successfully set it in the field info for that tool.

I tested setting:

GRID_STAT_FCST_FILE_TYPE = NETCDF_NCCF
GRID_STAT_OBS_FILE_TYPE = NETCDF_NCCF

and removing GRID_STAT_MET_CONFIG_OVERRIDES. I was able to get a successful run using your data with METplus 5.0.1 and MET 11.0.1. Please try that and let me know if that gives you a successful run.

[perthsb]

Thanks @georgemccabe. Able to complete a successful run with changes.
One question, while using Gridstat and with the option of writing netcdf for paired statistics is set as true in the config, is there anyway to add output statistics like RMSE, correlation in output netcdf file, other than difference between Fcst and Obs it writes (over grid).

[JohnHalleyGotway]

@perthsb, no, the contents of the NetCDF matched pairs file written by Grid-Stat is described around here in the MET User's Guide. It contains whichever gridded fcst/obs fields you requested in the nc_pairs_flag Grid-Stat config option. But this is all gridded data, not statistics derived from those matched pairs. The stats are written to the ASCII .stat output file, along with optional _LINE_TYPE.txt output files where the data are sorted by line type.

The Series-Analysis tool DOES write statistics to a NetCDF output file. However the matched pairs are accumulated over a time series of fcst/obs grids. The requested stats are computed separately for each grid point. This gives you a sense of how your model performance varies across the grid.

So Grid-Stat computes stats from a spatial aggregation of points while Series-Analysis computes stats from a temporal aggregation.

Hope that helps clarify the existing functionality.

[perthsb]

Thanks @JohnHalleyGotway. You may close this ticket.