Problem opening new format ECMWF grib files using CCSDS compression

[simonrp84]

Hello,
I'm trying to open a grib format file from ECMWF. The file is generated using the ECMWF open data python library as follows:

from ecmwf.opendata import Client
client = Client()

result = client.retrieve(
    type="fc",
    param="skt",
    step="0",
    target="data.grb2",
)

This produces a grib file I can read with cfgrib and eccodes, and a zipped example is attached to this issue: data.zip

But, gdal does not successfully open the grib.
I suspect this may be due to changes in their grib format, see: https://confluence.ecmwf.int/display/COPSRV/Implementation+of+IFS+cycle+48r1+for+CAMS#ImplementationofIFScycle48r1forCAMS-DefaultpackingtypesfordatafromDisseminationandMARS

I raised this on the gdal mailing list and was asked to make an issue, so hopefully this provides enough info to reproduce the problem.

Expected behavior and actual behavior.

Expected behaviour: GDAL opens the GRIB file and provides information about its contents.
Actual behaviour, this error message:

      getdrstemplate: DRS Template 5.42 not defined.
      ERROR 4: data.grb2 is a grib file, but no raster dataset was successfully identified.
             gdalinfo failed - unable to open 'data.grb2'.

Steps to reproduce the problem.

With the attached grib:

gdalinfo data.grb2

Operating system

Tested on Windows 11 and Ubuntu 20.04

GDAL version and provenance

GDAL 3.7.0 installed via conda.

[rouault]

implemented per #8108

[simonrp84]

Thanks for your quick fix on this. Right now I can't install the dev gdal but as soon as I'm able to I'll do so and confirm this works on my end.

[rouault]

@simonrp84 Here's the GeoTIFF file produced with #8108 and gdal_translate data.grb2 data.grb2.tif
data.grb2.tif.zip

$ gdalinfo data.grb2 -stats
Driver: GRIB/GRIdded Binary (.grb, .grb2)
Files: data.grb2
Size is 900, 451
Coordinate System is:
GEOGCRS["Coordinate System imported from GRIB file",
    DATUM["unnamed",
        ELLIPSOID["Sphere",6371229,0,
            LENGTHUNIT["metre",1,
                ID["EPSG",9001]]]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433,
            ID["EPSG",9122]]],
    CS[ellipsoidal,2],
        AXIS["latitude",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433,
                ID["EPSG",9122]]],
        AXIS["longitude",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433,
                ID["EPSG",9122]]]]
Data axis to CRS axis mapping: 2,1
Origin = (-180.199999999999989,90.200000000000003)
Pixel Size = (0.400000000000000,-0.400000000000000)
Corner Coordinates:
Upper Left  (-180.2000000,  90.2000000) (180d12' 0.00"W, 90d12' 0.00"N)
Lower Left  (-180.2000000, -90.2000000) (180d12' 0.00"W, 90d12' 0.00"S)
Upper Right ( 179.8000000,  90.2000000) (179d48' 0.00"E, 90d12' 0.00"N)
Lower Right ( 179.8000000, -90.2000000) (179d48' 0.00"E, 90d12' 0.00"S)
Center      (  -0.2000000,   0.0000000) (  0d12' 0.00"W,  0d 0' 0.01"N)
Band 1 Block=900x1 Type=Float64, ColorInterp=Undefined
  Description = 0[-] SFC="Ground or water surface"
  Minimum=-69.028, Maximum=64.784, Mean=9.384, StdDev=22.688
  Metadata:
    GRIB_COMMENT=Skin temperature [C]
    GRIB_DISCIPLINE=0(Meteorological)
    GRIB_ELEMENT=SKINT
    GRIB_FORECAST_SECONDS=10800
    GRIB_IDS=CENTER=98(ECMWF) SUBCENTER=0 MASTER_TABLE=27 LOCAL_TABLE=0 SIGNF_REF_TIME=1(Start_of_Forecast) REF_TIME=2023-07-11T12:00:00Z PROD_STATUS=0(Operational) TYPE=1(Forecast)
    GRIB_PDS_PDTN=0
    GRIB_PDS_TEMPLATE_ASSEMBLED_VALUES=0 17 2 255 154 0 0 1 3 1 -127 -2147483647 255 -127 -2147483647
    GRIB_PDS_TEMPLATE_NUMBERS=0 17 2 255 154 0 0 0 1 0 0 0 3 1 255 255 255 255 255 255 255 255 255 255 255
    GRIB_REF_TIME=1689076800
    GRIB_SHORT_NAME=0-SFC
    GRIB_UNIT=[C]
    GRIB_VALID_TIME=1689087600
    STATISTICS_MAXIMUM=64.784448242188
    STATISTICS_MEAN=9.3835983211709
    STATISTICS_MINIMUM=-69.028036499023
    STATISTICS_STDDEV=22.6883327624
    STATISTICS_VALID_PERCENT=100

[simonrp84]

Oh, so sorry for not getting back to you on this - it slipped my mind. The updates work nicely, thanks for your work on this and for closing the issue.