Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Forcing builder #365

Merged
merged 50 commits into from
Sep 20, 2023
Merged

Forcing builder #365

merged 50 commits into from
Sep 20, 2023

Commits on Jul 31, 2023

  1. Make generic distributed esmvaltool recipe

    @sverhoeven
    sverhoeven committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    b2d1fb2 View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2023

  1. New API for forcing generator

    @Peter9192
    Peter9192 committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    23074a5 View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2023

  1. Use fluent builder pattern to construct esmvaltool recipes

    @sverhoeven
    sverhoeven committed Aug 15, 2023
    Configuration menu
    Copy the full SHA
    738abd4 View commit details
    Browse the repository at this point in the history

  2. Added builder.add_variables(['pr'])

    @sverhoeven
    sverhoeven committed Aug 15, 2023
    Configuration menu
    Copy the full SHA
    7f55801 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/main' into forcing_builder

    @sverhoeven
    sverhoeven committed Aug 15, 2023
    Configuration menu
    Copy the full SHA
    35a2604 View commit details
    Browse the repository at this point in the history

  4. Working GenericDistributedForcing.generate

    @sverhoeven
    sverhoeven committed Aug 15, 2023
    Configuration menu
    Copy the full SHA
    370bc75 View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2023

  1. Move all esmvaltool specific stuff to own package 

    This make base package much cleaner
    @sverhoeven
    sverhoeven committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    34ec1af View commit details
    Browse the repository at this point in the history

  2. Move builder test

    @sverhoeven
    sverhoeven committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    67a1f26 View commit details
    Browse the repository at this point in the history

  3. Fix mypy errors

    @sverhoeven
    sverhoeven committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    6a7ec07 View commit details
    Browse the repository at this point in the history

  4. Use builder for LisfloodForcing

    @sverhoeven
    sverhoeven committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    0a76884 View commit details
    Browse the repository at this point in the history

  5. Dont use name of Python built-in package as module name 

    Got
Traceback (most recent call last):
  File "/home/verhoes/git/eWaterCycle/ewatercycle/src/ewatercycle/esmvaltool/diagnostic/copy.py", line 7, in <module>
    from esmvaltool.diag_scripts.shared import (
  File "/home/verhoes/mambaforge/envs/ewatercycle/lib/python3.10/site-packages/esmvaltool/diag_scripts/shared/__init__.py", line 2, in <module>
    from . import io, iris_helpers, names, plot
  File "/home/verhoes/mambaforge/envs/ewatercycle/lib/python3.10/site-packages/esmvaltool/diag_scripts/shared/io.py", line 5, in <module>
    from pprint import pformat
  File "/home/verhoes/mambaforge/envs/ewatercycle/lib/python3.10/pprint.py", line 38, in <module>
    import dataclasses as _dataclasses
  File "/home/verhoes/mambaforge/envs/ewatercycle/lib/python3.10/dataclasses.py", line 3, in <module>
    import copy
  File "/home/verhoes/git/eWaterCycle/ewatercycle/src/ewatercycle/esmvaltool/diagnostic/copy.py", line 7, in <module>
    from esmvaltool.diag_scripts.shared import (
ImportError: cannot import name 'ProvenanceLogger' from partially initialized module 'esmvaltool.diag_scripts.shared' (most likely due to a circular import) (/home/verhoes/mambaforge/envs/ewatercycle/lib/python3.10/site-packages/esmvaltool/diag_scripts/shared/__init__.py)
    @sverhoeven
    sverhoeven committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    1f107c2 View commit details
    Browse the repository at this point in the history

  6. Added GenericLumpedForcing, untested as dcache is curently down

    @sverhoeven
    sverhoeven committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    f0f08be View commit details
    Browse the repository at this point in the history

  7. Use recipe builder for MarrmotForcing

    @sverhoeven
    sverhoeven committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    23be08c View commit details
    Browse the repository at this point in the history

  8. Use builder in WflowForcing

    @sverhoeven
    sverhoeven committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    6b56e3b View commit details
    Browse the repository at this point in the history

  9. Use builder for ForcingHype

    @sverhoeven
    sverhoeven committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    905866e View commit details
    Browse the repository at this point in the history

  10. Merge remote-tracking branch 'origin/main' into default-gridded-forcing

    @sverhoeven
    sverhoeven committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    c519cd2 View commit details
    Browse the repository at this point in the history

  11. Merge branch 'default-gridded-forcing' into forcing_builder

    @sverhoeven
    sverhoeven committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    38fb0fd View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2023

  1. Make mip required in dataset

    @sverhoeven
    sverhoeven committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    6d6909e View commit details
    Browse the repository at this point in the history

  2. More docs

    @sverhoeven
    sverhoeven committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    f81fa90 View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2023

  1. Make expected recipe string consistent

    @sverhoeven
    sverhoeven committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    4c4edcc View commit details
    Browse the repository at this point in the history

  2. Parse esmvaltool recipe output when non netcdf files are produced 

    + more docs + fix tests
    @sverhoeven
    sverhoeven committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    8b9f15a View commit details
    Browse the repository at this point in the history

  3. Got LisfloodForcing.generate(lisvap=None) working 

    Had to make Dataset.mip optional otherwise tdps var with mip=day throws esmvaltool error
    @sverhoeven
    sverhoeven committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    afe524d View commit details
    Browse the repository at this point in the history

  4. Make MarrmotForcing.generate work

    @sverhoeven
    sverhoeven committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    182ce48 View commit details
    Browse the repository at this point in the history

  5. Fix tests for wflow and pcrglobwb

    @sverhoeven
    sverhoeven committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    24f0c18 View commit details
    Browse the repository at this point in the history

  6. Add support for variable without temporal selection

    @sverhoeven
    sverhoeven committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    94d1f03 View commit details
    Browse the repository at this point in the history

  7. give right dataset to lisvap

    @sverhoeven
    sverhoeven committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    959ca88 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2023

  1. Correct expected orog var

    @sverhoeven
    sverhoeven committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    596023f View commit details
    Browse the repository at this point in the history

  2. Make mypy happy

    @sverhoeven
    sverhoeven committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    ebae7bb View commit details
    Browse the repository at this point in the history

  3. Dont copy line number, >>> and ... from code blocks

    @sverhoeven
    sverhoeven committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    6950399 View commit details
    Browse the repository at this point in the history

  4. More tests, flake8 fixes and docs

    @sverhoeven
    sverhoeven committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    0f38a97 View commit details
    Browse the repository at this point in the history

  5. Tests for DefaultForcing + Remove discriminator from forcing classes 

    As you now have to specify which class you want to use to load a forcing.
    @sverhoeven
    sverhoeven committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    0d52380 View commit details
    Browse the repository at this point in the history

  6. Replaced with tests/src/base/test_forcing.py

    @sverhoeven
    sverhoeven committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    5080074 View commit details
    Browse the repository at this point in the history

  7. Remove discriminator from forcing classes from tests

    @sverhoeven
    sverhoeven committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    884ac67 View commit details
    Browse the repository at this point in the history

  8. Make flake8 happier

    @sverhoeven
    sverhoeven committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    1788661 View commit details
    Browse the repository at this point in the history

  9. Model validator should return otherwise object is set to None

    @sverhoeven
    sverhoeven committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    f6de70d View commit details
    Browse the repository at this point in the history

  10. Add test for MarrmotForcing.to_xarray()

    @sverhoeven
    sverhoeven committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    01862c3 View commit details
    Browse the repository at this point in the history

  11. Add generic forcing to user guide + custom forcing instructions + 

    prevent circular deps with generic forcings in entry points
    @sverhoeven
    sverhoeven committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    d0d4845 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2023

  1. More docs

    @sverhoeven
    sverhoeven committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    586cce7 View commit details
    Browse the repository at this point in the history

  2. Use Dataset object instead of dict in pre defined datasets

    @sverhoeven
    sverhoeven committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    5b66ba9 View commit details
    Browse the repository at this point in the history

  3. Tested GenericDistributedForcing with non-era dataset

    @sverhoeven
    sverhoeven committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    51b0e3e View commit details
    Browse the repository at this point in the history

  4. Allow extra attributes in Dataset 

    '*' in forcing files was caused by version attribute in Dataset being silently forgotten.
By allowing extra attributes the generate file has version value instead of `*`.
    @sverhoeven
    sverhoeven committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    2731d9d View commit details
    Browse the repository at this point in the history

  5. Tested CMIP6

    @sverhoeven
    sverhoeven committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    e9778dc View commit details
    Browse the repository at this point in the history

  6. Add dataset.version to expected recipes

    @sverhoeven
    sverhoeven committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    198bc5d View commit details
    Browse the repository at this point in the history

  7. Add changes to CHANGELOG + more docs + more todos

    @sverhoeven
    sverhoeven committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    a9ef890 View commit details
    Browse the repository at this point in the history

  8. Fix tests with updated model names

    @sverhoeven
    sverhoeven committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    11fe44b View commit details
    Browse the repository at this point in the history

  9. Disable lisflood model tests as downloading parameter set too slow

    @sverhoeven
    sverhoeven committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    aa38d0d View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2023

  1. Implement PR suggestions

    @sverhoeven
    sverhoeven committed Sep 18, 2023
    Configuration menu
    Copy the full SHA
    df523e9 View commit details
    Browse the repository at this point in the history

  2. Rename build_recipe() to build_<modelname>_recipe()

    @sverhoeven
    sverhoeven committed Sep 18, 2023
    Configuration menu
    Copy the full SHA
    589515d View commit details
    Browse the repository at this point in the history

  3. Centralize esmvaltool config for download

    @sverhoeven
    sverhoeven committed Sep 18, 2023
    Configuration menu
    Copy the full SHA
    04fb155 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2023

  1. Rename esmvaltool.models to esmvaltool.schema 

    When we talk about models we mean hydrological models.
    @sverhoeven
    sverhoeven committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    783a9c9 View commit details
    Browse the repository at this point in the history