Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature #2780 grib tables #3005

Merged
merged 27 commits into from
Nov 7, 2024
Merged

Conversation

JohnHalleyGotway
Copy link
Collaborator

@JohnHalleyGotway JohnHalleyGotway commented Nov 1, 2024

Expected Differences

This PR makes changes to GRIB1 and GRIB2 table files in MET. These changes include:

  • Incorporating new tables and modified entries from wgrib2 version 3.1.4 (hera:/home/Wesley.Ebisuzaki/grib2/wgrib2/gribtables) and from wgrib2 version 3.4.0 from NOAA-EMC/wgrib2.
  • Note that these version numbers appear sequential but were created from different development branches.
  • Eliminating whitespace from the units strings which are written to the FCST_UNITS and OBS_UNITS STAT header columns, where embedded whitespace is not allowed.
  • Modifying units strings from using ** to ^ to consistently indicate exponentiation (e.g. switch from m**2 to m^2).
  • Correct many units strings that lacked ** or ^ for exponentiation (e.g. switch from m2 to m^2).
  • Modifying units strings to replace negative exponents with / (e.g. switch from m s**-1 to m/s).
  • Modifying units strings to eliminate parentheses in the denominator (e.g. switch from kg/(m^2*s) to kg/m^2/s).
  • I note that some GRIB table abbreviations (in the grib1_dwd_*.txt tables) do still include embedded whitespace. MET does check for that when writing the FCST_VAR and OBS_VAR STAT header columns and reports this error message:
ERROR  : check_hdr_str() -> output header column FCST_VAR value ("Temp With Spaces") should contain no embedded whitespace!

Users can use set_attr_name = "TMP" to provide a non-whitespace version of the name.

  • Do these changes introduce new tools, command line arguments, or configuration file options? [No]

    If yes, please describe:

  • Do these changes modify the structure of existing or add new output data types (e.g. statistic line types or NetCDF variables)? [No]

    If yes, please describe:

Pull Request Testing

  • Describe testing already performed for these changes:

    None yet.

  • Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

    Spend no more than 1 hour scanning the differences, looking for obvious problems in changes to the tables, in particular for updated unit strings.
    Please refer to this NCEP GRIB2 documentation and this NCEP GRIB1 documentation only when needed.

  • Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes]
    None needed.

  • Do these changes include sufficient testing updates? [Yes]
    None needed.

  • Will this PR result in changes to the MET test suite? [Yes]

    If yes, describe the new output and/or changes to the existing output:

    The vorticity units reported to the MET differ:

COMPARING grid_stat/grid_stat_GRIB_lvl_typ_val_180000L_20170610_180000V_pairs.nc
file1: /data/output/met_test_truth/grid_stat/grid_stat_GRIB_lvl_typ_val_180000L_20170610_180000V_pairs.nc
file2: /data/output/met_test_output/grid_stat/grid_stat_GRIB_lvl_typ_val_180000L_20170610_180000V_pairs.nc
ERROR: NetCDF headers differ:
71c71
< 		FCST_PVORT_L310_FULL:units = "Km^2/kg/s" ;
---
> 		FCST_PVORT_L310_FULL:units = "K*m^2/kg/s" ;

See the GRIB2 table documentation for:
Screenshot 2024-11-01 at 4 22 42 PM

  • Will this PR result in changes to existing METplus Use Cases? [Probably]

    If yes, create a new Update Truth METplus issue to describe them.
    The units reported to the MET output may differ.

  • Do these changes introduce new SonarQube findings? [No]

    If yes, please describe:
    I expect no impact.

  • Please complete this pull request review by [Tues 11/5/24].

Pull Request Checklist

See the METplus Workflow for details.

  • Review the source issue metadata (required labels, projects, and milestone).
  • Complete the PR definition above.
  • Ensure the PR title matches the feature or bugfix branch name.
  • Define the PR metadata, as permissions allow.
    Select: Reviewer(s) and Development issue
    Select: Milestone as the version that will include these changes
    Select: Coordinated METplus-X.Y Support project for bugfix releases or MET-X.Y.Z Development project for official releases
  • After submitting the PR, select the ⚙️ icon in the Development section of the right hand sidebar. Search for the issue that this PR will close and select it, if it is not already selected.
  • After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
  • Close the linked issue and delete your feature or bugfix branch from GitHub.

…with updates from the 109 defined in the wgrib2 table.
…tring to avoid problems writing them to FCST_UNITS and OBS_UNITS output columns
…f wgrib2: hera:/home/Wesley.Ebisuzaki/grib2/wgrib2/gribtables/ncep/gribtable.dat
…ng them more human-readable by replacing ** with ^ to indicate exponentiation. Also correct bad units for radiation flux in AF tables by listing them as W/m^2.
…h 'mcg' to represent micrograms. Also replace 'mm6' with 'mm^6' by double-checking against the GRIB1 docs.
…' for consistent usage across NCEP GRIB tables.
@JohnHalleyGotway JohnHalleyGotway added this to the MET-12.0.0 milestone Nov 1, 2024
@JohnHalleyGotway JohnHalleyGotway linked an issue Nov 1, 2024 that may be closed by this pull request
21 tasks
…e the use of parantheses in the denominator. For example, replace 'kg/(m*s)' with 'kg/m/s'.
@JohnHalleyGotway JohnHalleyGotway marked this pull request as ready for review November 1, 2024 22:38
Copy link

@CPKalb CPKalb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like many of the * have changed to /. I just wanted to double check that this was intentional...

169 254 46 -1 "vmfl" "VERT. INTEGRATED MOISTURE FLUX CONV." "kg/(m2*s)"
170 254 46 -1 "vadv" "VERTICAL MOISTURE ADVECTION " "kg/(kg*s)"
171 254 46 -1 "nhcm" "NEG. HUM. CORR. MOISTURE SOURCE" "kg/(kg*s)"
169 254 46 -1 "vmfl" "VERT. INTEGRATED MOISTURE FLUX CONV." "kg/m^2/s"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this supposed to change from * to /? Same for the next 2 lines...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CPKalb, yes, it was an intentional change... switching from a/(b*c) to a/b/c which is mathematically equivalent. I found a pretty wide variety of conventions in units strings across GRIB1 and GRIB2 table sources. And I made these changes to move toward more consistency.

But please advise if you'd prefer that those differences be retained.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JohnHalleyGotway, I think the change is fine, I just wanted to be sure. Like you said, there are a variety of ways to express it.

Copy link

@CPKalb CPKalb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went through as many of the changes as I could in an hour and didn't see anything amiss.

@JohnHalleyGotway JohnHalleyGotway merged commit c452010 into develop Nov 7, 2024
39 of 40 checks passed
@JohnHalleyGotway JohnHalleyGotway deleted the feature_2780_grib_tables branch November 7, 2024 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🏁 Done
Development

Successfully merging this pull request may close these issues.

Update GRIB tables in MET based on wgrib2 versions 3.1.4 and 3.4.0
2 participants