Implement regridding during Dataset.load()

[BSchilperoort]

After a lot of messing around I've settled upon the following implementation:

• A regridding implementation with flox and xarray (both available for all systems & on pypi).
• An optional regridding method that uses xesmf (which requires esmpy/ESMF: only on linux, only on conda).

For most cases the two methods give basically the same result, although the custom regridding routine doesn't handle the following too well:

• There is no weighted averaging performed: areas near the poles will incorrectly contribute more to the total average.
• When the new resolution is close to the old resolution, an interpolation step is run first (to avoid anti-aliassing),and then the data is regridded to the intended resolution. This will lead to small inconsistencies with NaN thresholds.

---

How to review:

• Download & ingest the eth canopy dataset.
• Try out the .load method's resolution and regrid_method kwargs.

[BSchilperoort]

Hi @geek-yang , this PR is ready for review!

You can ignore the failing windows tests for now, it's due to a pytest argument apparently not working (correctly) on Windows 🤪

[geek-yang]

Hi @geek-yang , this PR is ready for review!

You can ignore the failing windows tests for now, it's due to a pytest argument apparently not working (correctly) on Windows 🤪

So for windows --ignore-glob='*xesmf*.py' this argument doesn't seem to work....(quite annoying...). A slightly nasty solution to bypass it, perhaps we can add the following to xesmf_regrid.py:

try:
    import xesmf
except ImportError:
    pass

This will not trigger the dependency check failure for pytest. Later when we need to add some unit tests for xesmf_regrid.py, we can put:

@pytest.mark.skipif('xesmf' not in sys.modules,
                    reason="requires the xesmf library")
def test_xesmf_function():
    ...

[geek-yang]

@BSchilperoort Super! Awesome solution for regridding! I've tested both methods for regridding (I need to get conda/mamba back for it.... 😒) and they all work perfectly. It would be nice to update the notebook as well (since now resolution and regrid_method are mandatory). But overall everything looks fantastic! You get a go 😜.

PS: if you plan to update the notebook, please also add equal_nan=True to the assertion of conversion (last cell), otherwise it might fail.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[BSchilperoort]

Thanks for the review, @geek-yang ! 😄

I fixed the pytest issue with --doctest-ignore-import-errors. I couldn't find that by searching the documentation, but by calling pytest --help. It seems that it's not very well known about in general, returning only 31 results on google.

[geek-yang]

Thanks for the review, @geek-yang ! 😄

I fixed the pytest issue with --doctest-ignore-import-errors. I couldn't find that by searching the documentation, but by calling pytest --help. It seems that it's not very well known about in general, returning only 31 results on google.

Awesome! Yesterday I also did some googling but didn't find anything like this. It is quite nice that you checked --help 👍. I never expect --help to be helpful. Lesson for today haha 🤣 .