Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Metadata class #110

Merged
merged 16 commits into from
Aug 25, 2023
Merged

Improve Metadata class #110

merged 16 commits into from
Aug 25, 2023

Conversation

edsaac
Copy link
Contributor

@edsaac edsaac commented Aug 22, 2023

Implements a base metadata class that can be inherited from each service. This PR follows discussion from #109.

graph TD;
    nwis.NWIS_Metadata-->utils.BaseMetadata;
    wqp.WQP_Metadata-->utils.BaseMetadata;
Loading

Changes in dataretrieval.utils:

  • BaseMetadata class is implemented such that it is initialized from the a request (md = BaseMetadata(request)). However, each service should inherit from BaseMetadata and implement their particular methods.
  • The set_metadata function is removed, this is handled by the BaseMetadata's methods.

Changes in dataretrieval.nwis:

  • NWIS_Metadata class inherits from utils.BaseMetadata. It implements site_info and variable_info as properties and handles the metadata comments.
  • User sees a nicer __repr__ of the Metadata object.

Example:

In [1]: from dataretrieval import nwis

In [2]: df, md = nwis.get_ratings(site="01594440")

In [3]: df
Out[3]: 
    INDEP       DEP STOR
0    2.99     30.00    *
1    4.00    110.00    *
...
9   20.85  16497.75    *
10  27.90  31100.00    *

In [4]: md
Out[4]: NWIS_Metadata(url=https://nwis.waterdata.usgs.gov/nwisweb/get_ratings/?site_no=01594440&file_type=base)

Changes in dataretrieval.wqp:

  • "WQP_Metadata" inhertis from utils.BaseMetadata and implements site_info property.

Example:

In [1]: from dataretrieval import wqp

In [2]: df, md = wqp.get_results(lat='44.2', long='-88.9', within='0.5')

In [3]: df
Out[3]: 
   OrganizationIdentifier                     OrganizationFormalName  ActivityIdentifier  ... DetectionQuantitationLimitMeasure/MeasureUnitCode PreparationStartDate  ProviderName
0               WIDNR_WQX  Wisconsin Department of Natural Resources  WIDNR_WQX-35940585  ...                                               NaN                  NaN        STORET
1               WIDNR_WQX  Wisconsin Department of Natural Resources  WIDNR_WQX-35940585  ...                                               NaN                  NaN        STORET
...
17              WIDNR_WQX  Wisconsin Department of Natural Resources  WIDNR_WQX-35940585  ...                                               NaN                  NaN        STORET

[18 rows x 63 columns]

In [4]: md
Out[4]: WQP_Metadata(url=https://www.waterqualitydata.us/data/Result/Search?lat=44.2&long=-88.9&within=0.5&zip=no&mimeType=csv)

Changes in tests:

  • Added tests for the BaseMetadata class
  • Modified tests where md.site_info and md.variable_info were expected as callables

Breaking changes:

  • md.site_info and md.variable_info are no longer callables but class properties. I am not sure why those were implemented to return lambda functions, but my guess is so the service query was only made if they where explicitly called by the user. If that was the case, the @property decorator should take care of that.

Copy link
Collaborator

@thodson-usgs thodson-usgs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well done, @edsaac
thank you for contributing.

@thodson-usgs thodson-usgs merged commit 6b6ad6a into DOI-USGS:master Aug 25, 2023
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants