CFS GenEnsProd File Issue

[ShannonShields-NOAA]

Hello, I am having a problem with GenEnsProd output files for the CFS model. When I use plot_data_plane to check the gen_ens_prod.nc files, they look like the following:
For Sea Ice Concentration:

For SST:

There seems to be a regridding issue perhaps? The sea ice map should look something like the following:

Maybe there is an issue in my GenEnsProd config file? This config file can be found on WCOSS2 (along with raw CFS member data and gen_ens_prod files) here: /lfs/h2/emc/vpppg/noscrub/shannon.shields/cfs_issue/files/
The spatial maps shown above can be found here: /lfs/h2/emc/vpppg/noscrub/shannon.shields/cfs_issue/spatial_maps/
Thank you for the assistance!

[JohnHalleyGotway]

Hi Shannon, are you able to post these sample files to our anonymous FTP site following these instructions? Very few staff have WCOSS2 access and posting it to a more accessible place would make it easier to debug.

I'd like to see both the Gen-Ens-Prod output as well as at least one input file. For example, whatever input file you used to create the "weekly_avg_seaice_WeeklyAvg_Concentration_init ... " plot.

Thanks!

[JohnHalleyGotway]

@ShannonShields-NOAA thanks for copying over those sample files to hera. I was able to grab them there. The issue is in reading data from CFS GRIB2 files themselves. Whenever getting started with new datasets in MET, its always a good idea to run them through plot_data_plane to see how MET is reading them.

plot_data_plane cfs.ens01.2023030400.f840 ~/cfs_TMP_Z2.ps 'name="TMP"; level="Z2";'

And here's the result:

I double-checked by plotting this GRIB2 file using Panoply and the result looks fine.

That indicates to me that this is a bug in MET's reading of GRIB2 data on a Gaussian grid.

I did some more testing... converting the GRIB2 file to GRIB1 and plotting it with MET. This result looks a bit more sane:

But it's still problematic, as seen by the clear eastward shift of values in South America.

So this is a bug.

[JohnHalleyGotway]

Shannon, can you tell me more about the origin of these files:

cfs.ens01.2023030400.f840
cfs.ens02.2023030400.f840
cfs.ens03.2023030400.f840
cfs.ens04.2023030400.f840

Running them through wgrib2, I see they're defined on a Gaussian grid:

	Gaussian grid: (384 x 190) units 1e-06 input WE:NS output WE:SN
	number of latitudes between pole-equator=95 #points=72960
	lat 89.277000 to -89.277000
	lon 0.000000 to 359.062000 by 0.938000

And that appears to match NCEP Grid 126.

All the CFS GRIB2 data files I see on NOMADS appear to be on a 360x181 latlon grid:

	lat-lon grid:(360 x 181) units 1e-06 input WE:NS output WE:SN res 48
	lat 90.000000 to -90.000000 by 1.000000
	lon 0.000000 to 359.000000 by 1.000000 #points=65160

Is CFS data produced and/or distributed on both Gaussian and Lat/Lon grids?

[ShannonShields-NOAA]

Thank you John for looking into this. I thought the CFS ensemble member files were on a 1 degree lat lon grid, so I have to look into this. Maybe something is going wrong in my prep data step. Mallory Row is also using these same CFS files for her EVS work, so I'll ask what she is doing with her files.

[JohnHalleyGotway]

Aha, wait. I understand better what's going on here! MET assumes that all records in a single GRIB file are all on the same grid. This CFS file violates that assumption, and that's the source of the problem. The CFS files you sent contain a mixture of records on 2 different grids:

1. A global lat/lon grid:

	lat-lon grid:(360 x 181) units 1e-06 input WE:NS output WE:SN res 48
	lat 90.000000 to -90.000000 by 1.000000
	lon 0.000000 to 359.000000 by 1.000000 #points=65160

2. A gaussian grid:

	Gaussian grid: (384 x 190) units 1e-06 input WE:NS output WE:SN
	number of latitudes between pole-equator=95 #points=72960
	lat 89.277000 to -89.277000
	lon 0.000000 to 359.062000 by 0.938000

You can see their use alternating in this wgrib2 output:

wgrib2 -Sec3 cfs.ens01.2023030400.f840.grib2 
1:0:Sec3 len=72 src gdef=0 npts=65160 Grid Def Template=3.0 opt arg=0
2:56875:Sec3 len=72 src gdef=0 npts=72960 Grid Def Template=3.40 opt arg=0
3:89955:Sec3 len=72 src gdef=0 npts=65160 Grid Def Template=3.0 opt arg=0
4:130313:Sec3 len=72 src gdef=0 npts=72960 Grid Def Template=3.40 opt arg=0
5:145672:Sec3 len=72 src gdef=0 npts=72960 Grid Def Template=3.40 opt arg=0
6:158954:Sec3 len=72 src gdef=0 npts=72960 Grid Def Template=3.40 opt arg=0
7:240656:Sec3 len=72 src gdef=0 npts=72960 Grid Def Template=3.40 opt arg=0
8:320019:Sec3 len=72 src gdef=0 npts=72960 Grid Def Template=3.40 opt arg=0
9:351041:Sec3 len=72 src gdef=0 npts=72960 Grid Def Template=3.40 opt arg=0
10:383591:Sec3 len=72 src gdef=0 npts=65160 Grid Def Template=3.0 opt arg=0
11:440178:Sec3 len=72 src gdef=0 npts=65160 Grid Def Template=3.0 opt arg=0
12:490259:Sec3 len=72 src gdef=0 npts=72960 Grid Def Template=3.40 opt arg=0
13:549876:Sec3 len=72 src gdef=0 npts=65160 Grid Def Template=3.0 opt arg=0
14:606774:Sec3 len=72 src gdef=0 npts=65160 Grid Def Template=3.0 opt arg=0
15:656632:Sec3 len=72 src gdef=0 npts=72960 Grid Def Template=3.40 opt arg=0

MET assumes the grid of the first record applies to all records in the file. So it's using the 2m temperature Guassian data from record number 7 to populate the lat/lon grid from record number 1.

When I use wgrib2 to select out record 7 by itself, MET plots it just fine:

Based on this assumption that MET is making for better or for worse (for worse in this case!), please step back to see how the cfs.ens01.2023030400.f840.grib2 file is being cobbled together. And please segregate the records on different grids into different files.

[AliciaBentley-NOAA]

Hi everyone,

I found the following information on the CPC webpage:
https://www.ftp.cpc.ncep.noaa.gov/wd51we/wgrib2/tricks.wgrib2

(32) How do I separate a grib file by grid types?

   **Sometimes a grib file may have multiple grids and want to split the file
   by grid type.**

   You can see the grid used by the file

       wgrib2 -grid IN.grb

   You can see the number of grids (and the checksum) used by the (linux/unix)

       wgrib2 IN.grb -checksum 3 | cut -f3 -d: | sort -u

            wgrib2 IN.grb -checksum 3   will write the checksum of section 3, the grid definition section
            cut -f3 -d:                 will select field 3 which is the check sum
            sort -u                     sort and only show unique line

        bash-3.2$ wgrib2 grids.grb -checksum 3 | cut -f3 -d: | sort -u
        sec3_cksum=1966855777
        sec3_cksum=2751972329
        sec3_cksum=3045129773
        sec3_cksum=3781314504
        sec3_cksum=4009958728

        You can extract a specific grid from grids.grb by, for example,

        bash-3.2$ wgrib2 grids.grb -checksum 3 | grep "sec3_cksum=1966855777" | wgrib2 grids.grb -i -grib grid1.grb
        2:169368:d=2009060500:HGT:200 mb:330 hour fcst:std dev
        
        Perhaps this would help with saving only the lat/lon grids that we need?

[ShannonShields-NOAA]

Thank you John; you explained the problem exactly. I am combining both pgbf and flxf CFS file types which must be on different grids (unlike GEFS which explains why those files look correct). Combining the pgbf and flxf files together in my prep step is what is causing the grid problem. I'll work on separating those files once I have access to WCOSS2.

[JohnHalleyGotway]

Whew, I'm relieved to hear that we don't need to work on a bugfix for this! Thanks for confirming.

As to the underlying assumption in MET, we could take a step back to revisit the assumption of the same grid being used for all the records. To my recollection, this is the first time it's come up.

Some options to consider are...

1. Fully support a mixture of grids in a single GRIB file. This would introduce some ambiguity though. Grid-Stat presumes that all comparisons in a single run are done on the same grid. And setting regrid.to_grid = FCST; can be set to indicate that the forecast grid should be used. If 2 variables are processed on 2 different grids, that setting becomes ambiguous.

2. We could add more validation checks so that each time we read gridded data, we make sure the grid is what we expect. And if not, print an error message stating that all input GRIB records must be on the same grid.

@AliciaBentley-NOAA and @ShannonShields-NOAA, please let me know if you think either of these approaches should be written up as an actual MET GitHub issue.

[ShannonShields-NOAA]

I don't know at this time @AliciaBentley-NOAA . @JohnHalleyGotway If the flxf file that I am using is on a Gaussian latitude/longitude grid (/scratch2/NCEPDEV/stmp1/Shannon.Shields/scripts/s2s_data/flxf2023041000.01.2023040900.grb2), will that work in METplus? It seems like all records in that file are on the same grid (same npts):
wgrib2 -Sec3 flxf2023041000.01.2023040900.grb2
1:0:Sec3 len=72 src gdef=0 npts=72960 Grid Def Template=3.40 opt arg=0
2:46011:Sec3 len=72 src gdef=0 npts=72960 Grid Def Template=3.40 opt arg=0
3:91314:Sec3 len=72 src gdef=0 npts=72960 Grid Def Template=3.40 opt arg=0
So according to our discussion above, as long as the grid npts are all the same within the one grib2 file, it should not pose a problem. The problem occurs when these npts don't match. Am I understanding this correctly?

[ShannonShields-NOAA]

Thanks John for marking this as answered. I just wanted to close this discussion by confirming that the fixed/separated CFS files do indeed look much better and more similar to the GEFS output: