Bugs in NEXRAD L2 reader

[ghiggi]

• xradar version: 0.5.0
• Python version: 3.11
• Operating System: Linux

Description

I identified two bugs affecting the NEXRAD L2 reader.

• The first bug cause the time coordinates and time_coverage variables to be shifted by 1 day compared to the actual time. This impacts all NEXRAD L2 files.
• The second bug is related to an indexing error arising with some NEXRAD files.

Can someone have a look at it ? Maybe @mgrover1 who implemented the reader?

Here below I provide MCVE and error logs.

---

MCVE - Time is shifted by 1 day

import xradar as xd
import fsspec # s3fs need to be installed 
filepath = 'noaa-nexrad-level2/2023/08/20/KNKX/KNKX20230820_220808_V06' 
file = fsspec.open_local(
    f"simplecache::s3://{filepath}",
    s3={"anon": True},
    filecache={"cache_storage": "."},
dt_gr = xd.io.open_nexradlevel2_datatree(file)
print(dt_gr["time_coverage_start"])  # '2023-08-21T22:08:08Z'
print(dt_gr["time_coverage_end"]) # '2023-08-21T22:13:33Z'
print(dt_gr["sweep_0"]["time"]) # you will see the date is +1 day

---

MCVE - IndexError: list index out of range

import xradar as xd
import fsspec # s3fs need to be installed 
filepath = 'noaa-nexrad-level2/2023/08/19/KNKX/KNKX20230819_220909_V06'
file = fsspec.open_local(
    f"simplecache::s3://{filepath}",
    s3={"anon": True},
    filecache={"cache_storage": "."},
dt_gr = xd.io.open_nexradlevel2_datatree(file)

Error log:

  File ~/micromamba/envs/gpm-api-dev/lib/python3.11/site-packages/xradar/io/backends/nexrad_level2.py:485 in get_sweep
    moments = self.msg_31_data_header[sweep_number]["msg_31_data_header"].keys()

IndexError: list index out of range

[kmuehlbauer]

@ghiggi might be my fault. Could you please create a minimal example without fsspec? Just to have this out of the equation.

[ghiggi]

Just use fsspec to download the file with the following code snippet. And then open the file from disk:

import os
import fsspec
local_tmp_dir = "/tmp"
fs_args = {}
_ = fs_args.setdefault("anon", True)
fs = fsspec.filesystem("s3", **fs_args)
fs.download(filepath, local_tmp_dir)

file = os.path.join(local_tmp_dir, os.path.basename(filepath))
dt_gr = xd.io.open_nexradlevel2_datatree(file)

[kmuehlbauer]

@ghiggi The first one is most likely an issue with not taking the 1-based day count into account here:

xradar/xradar/io/backends/nexrad_level2.py

Line 1336 in 2d266bb

date = np.array([ms["collect_date"] for ms in msg_31_header]) * 86400e3

So we would need to subtract 1 day on that line, before transforming to milliseconds. Feel free to send in a PR to correct this.

For the second issue I'll need a deeper look into next week.

[mgrover1]

Ahhh okay - that makes sense... agreed, I can dig into this a bit more next week as well.