Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Makkink forcing will use excessive memory with default ESMValTool dask.yml #425

Open
BSchilperoort opened this issue Jun 5, 2024 · 0 comments
Labels
bug Something isn't working forcing Issue related to forcing (generation, loading, etc.)

Comments

@BSchilperoort
Copy link
Member

When running the Makkink forcing generation for ERA5 for a range of a couple of years, it can use excessive memory causing the kernel to crash.

However, when defining ~/.esmvaltool/dask.yml as the following:

cluster:
  type: distributed.LocalCluster
  n_workers: 2
  threads_per_worker: 1
  memory_limit: 5 GiB

It runs fine with a memory use <2GB.

The issue probably occurs in the post-processing function where the potential evaporation is computed. However, we do run into conflicting advice:

@BSchilperoort BSchilperoort added bug Something isn't working forcing Issue related to forcing (generation, loading, etc.) labels Jun 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working forcing Issue related to forcing (generation, loading, etc.)
Projects
None yet
Development

No branches or pull requests

1 participant