ESMPy fails when creating a Grid for a high-resolution dataset #263
-
Requirements
Affiliation(s)University of Leicester ESMF Versionv8.6.1 (ESMF and ESMPy) IssueI originally came across this problem when using xESMF (see here), but thought I might get a better reply from this group. I'm trying to regrid some rectilinear data from Sentinel-3 (~300 m resolution, global coverage) onto the EURO-CORDEX curvilinear grid (~12.5 km resolution, European coverage) using conservative regridding. However, even after clipping my Sentinel data to the 12 km grid extent, the size of the dataset is still very large (approx. 17000 lat x 39000 lon cells), which causes ESMPy to fail when creating a Grid:
Is there a way around this in ESMPy, or would I need to use ESMF explicitly? I'm not sure how to write a netCDF file for a curvilinear grid including cell boundaries that ESMF would recognise (I couldn't find any examples online that worked with Autotag |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
Thanks for reporting this. I was able to reproduce it. The problem arises because the memory used to represent your grid slightly exceeds the 32-bit integer limit (20000 * 15000 * 8 bytes = 2.4 billion, exceeding the 32-bit limit of ~ 2.1 billion). If you were able to clip your grid just a bit more, you could avoid this issue. If you're comfortable installing ESMPy from source (https://earthsystemmodeling.org/esmpy_doc/release/latest/html/install.html#installing-esmpy-from-source), you can patch the code with this fix that I have just submitted: We will try to get this fix into the upcoming ESMF 8.7 release. |
Beta Was this translation helpful? Give feedback.
Thanks for reporting this. I was able to reproduce it. The problem arises because the memory used to represent your grid slightly exceeds the 32-bit integer limit (20000 * 15000 * 8 bytes = 2.4 billion, exceeding the 32-bit limit of ~ 2.1 billion). If you were able to clip your grid just a bit more, you could avoid this issue.
If you're comfortable installing ESMPy from source (https://earthsystemmodeling.org/esmpy_doc/release/latest/html/install.html#installing-esmpy-from-source), you can patch the code with this fix that I have just submitted:
#267
We will try to get this fix into the upcoming ESMF 8.7 release.