Trying to run the MODE wrapper

[jcorner1]

Hello,

I am trying to run the MODE wrapper with composite radar data as the obs and HRRR as the model input. I have tried a few different things but ultimately have failed get any results. I will send my data through the ftp server in the "corner_data" directory as well as the log file with the error as well the config file to run MODE.

Thanks,
Jeremy

[georgemccabe]

Hi @jcorner1, I pulled your files from FTP, but it looks like one of the files you provided may have been corrupted when it was uploaded. I am not able to read it using ncdump. Here is what I see:

ncdump: Level3_Composite_dhr_1km_20221104_2100.gini.nc4: NetCDF: Unknown file format

Could you upload this file again?

[jcorner1]

Hi @georgemccabe, That is correct, I regridded the file prior to running METplus. I would think the HRRR file is smaller as I eliminated everything in the file that was unneeded to save space, so it should be just the datetime, lats/lons, and reflectivity.

I did try setting the MODE_REGRID_METHOD = FORCE to the config script, but I still got the same error. I'm using "xesmf" to do the regridding and using a hrrr_esrl file to regrid to so that I know the grid is correct as it works in one of the example cases, however I am still getting the error with the lat/lons not being evenly spaced. That is my point of confusion on why the example works but when I do it I get the lat/lon error.

[georgemccabe]

It looks like the hrrr_esrl data from the short_range/MODE_fcstHRRR_obsMRMS_Hail_GRIB2 use case is on a Lambert Conformal grid, so I'm confused as to why the data you regridded is on a lat/lon grid. I'm not sure how xesmf regrids data but this may be causing an issue, as NetCDF files that have unevenly lat/lon grids cannot be read natively by MET.

I noticed that the example use case you are referring to reads in GRIB2 observations and regrids the data to the forecast grid in MODE using MODE_REGRID_TO_GRID = FCST. If the MRMS data is GRIB2, you could try reading it into MODE without preprocessing it and let MET handle the regridding.

Is there another way you can provide us with the input data? It would be easier to see what exactly is going on with access to those files. You could upload them to a Google Drive folder or perhaps try uploading them to the FTP from another computer.

[jcorner1]

Hi @georgemccabe.
I'm not sure what the issue is with the file but I have uploaded them to a Google Drive (https://drive.google.com/drive/folders/1BwWrdUUEcG0xRc8IU0BzZ_hYUxcUC0ib?usp=sharing). If it doesn't work, I will assume that it is an issue with how am creating these data or how it is regridded by "xesmf" (possibly forcing it to change projection also). I wasn't able to work on your other ideas of fixes yesterday, but today I will give them a try as well as look at other test cases to see if we can come to a solution. Sorry for all the trouble with the data thus far, hopefully Google Drive will fix this issue. Thanks!

[georgemccabe]

Hi @jcorner1, I was able to download the files from the Google Drive folder and they are not corrupted. Hooray! It looks like both of the files are NetCDF despite the grib2 extension of the HRRR file. It looks like the HRRR file is lacking metadata required to read it into MET. The MRMS file also cannot be read due to the unevenly spaced lat/lon grids that we discovered earlier.

Could you upload the input files that were used to generate these files and the script you used to regrid them?

[jcorner1]

Hi @georgemccabe
I should have put them in the Google Drive now. let me know if you need anything else!

I did catch the thing with the extension being the wrong format and hopefully that will clear any futures that will cause. I did try it again, but obviously that didn't fix the lat/lon issue.

NOTE:
the MRMS data is the "res.nc" file. That was the name the file was given when I downloaded from the sever manually and I didn't catch that until after I uploaded it to the Google Drive.

[georgemccabe]

Thanks for sending those files. It looks like I am able to read both the HRRR and the MRMS files into MET.

Here are the commands I ran to generate the images:

plot_data_plane ./res.nc out_res.ps 'name="mrms_lcref"; level="(*,*)";'
convert -rotate 90 out_res.ps out_res.png
plot_data_plane ./hrrr.t16z.wrfsfcf05.grib2 out_hrrr.ps 'name="REFC"; level="L0";'
convert -rotate 90 out_hrrr.ps out_hrrr.png

You should be able to use the same name and level information to read the fields into MODE. Just be sure to set MODE_REGRID_TO_GRID = FCST or MODE_REGRID_TO_GRID = OBS so that the data are matched to the same grid.

Is there a reason you need to regrid these data with a script before passing them to MET? It looks like that step may not be necessary.

[jcorner1]

Hi @georgemccabe,

My advisor wants it regridded but I don't have a specific reason at the current moment. We'll be doing some machine learning, so I'm guessing that's his reasoning for now. I don't think it's an issue right now.

I'm having a little problem with setting the level in the .conf file. I will attach my log and conf file as well as put them in the Google Drive as I no longer trust a single way of sending files.
MODE_HRRR_FCST_RADAR_OBS.conf.txt
metplus.log.20221206145041.txt

[georgemccabe]

Hi @jcorner1, you can set MODE_REGRID_TO_GRID to the path to a file that MET can read and it will use its grid. You can also define an explicit grid definition. See Grid Definition Strings in the MET User's Guide.

It looks like you have refc set for the forecast field name, but the name of the field is REFC.

[jcorner1]

That looks to have done it! Thanks @georgemccabe!!!

[georgemccabe]

Great to hear! I am going to lock this discussion to prevent future posts. If you have any other questions, please create a new discussion topic.