You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Turns out the memory issue fix in #387 is problematic in some setups. It never happens in our CI, but the grid.destroy() calls sometimes result in segmentation faults when used in parallel with dask. I haven't been able to find the exact reason for these errors, but I believe the reason is that the Regridder object gets sent to the dask worker where the ESMpy grid object is not in the memory ?
Errors look like :
Fatal Python error: Segmentation fault
Thread 0x00007f57ecc006c0 (most recent call first):
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/concurrent/futures/thread.py", line 89 in _worker
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1012 in run
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1032 in _bootstrap
Thread 0x00007f57ed6006c0 (most recent call first):
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/concurrent/futures/thread.py", line 89 in _worker
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1012 in run
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1032 in _bootstrap
Thread 0x00007f57ee0006c0 (most recent call first):
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/concurrent/futures/thread.py", line 89 in _worker
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1012 in run
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1032 in _bootstrap
Thread 0x00007f57eea006c0 (most recent call first):
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/concurrent/futures/thread.py", line 89 in _worker
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1012 in run
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1032 in _bootstrap
Thread 0x00007f5800a006c0 (most recent call first):
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 355 in wait
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/queue.py", line 171 in get
File "/home/pbourg/Projets/xscen/src/xscen/catutils.py", line 372 in parse_worker
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1012 in run
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1032 in _bootstrap
Thread 0x00007f58014006c0 (most recent call first):
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 355 in wait
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/queue.py", line 171 in get
File "/home/pbourg/Projets/xscen/src/xscen/catutils.py", line 349 in check_worker
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1012 in run
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1032 in _bootstrap
Thread 0x00007f5801e006c0 (most recent call first):
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/concurrent/futures/thread.py", line 89 in _worker
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1012 in run
File
"/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1075tests/test_ensembles.py::TestGenerateWeights::test_attribute_weight[coefs0-weights0] in _bootstrap_inner
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1032 in _bootstrap
Thread 0x00007f58028006c0 (most recent call first):
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/concurrent/futures/thread.py", line 89 in _worker
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1012 in run
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1032 in _bootstrap
Current thread 0x00007f58032006c0 (most recent call first):
Garbage-collecting
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/esmpy/interface/cbindings.py", line 754 in ESMP_GridDestroy
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/esmpy/api/grid.py", line 925 in destroy
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/xesmf/frontend.py", line 1116 in __del__
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/llvmlite/ir/_utils.py", line 13 in is_used
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/llvmlite/ir/_utils.py", line 26 in deduplicate
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/llvmlite/ir/_utils.py", line 18 in register
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/llvmlite/ir/values.py", line 558 in _set_name
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/llvmlite/ir/values.py", line 542 in __init__
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/llvmlite/ir/instructions.py", line 14 in __init__
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/llvmlite/ir/instructions.py", line 675 in __init__
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/llvmlite/ir/builder.py", line 973 in extract_value
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/datamodel/models.py", line 648 in get
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/datamodel/models.py", line 714 in getter
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/runtime/context.py", line 355 in get_meminfos
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/runtime/context.py", line 367 in _call_incref_decref
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/runtime/context.py", line 388 in decref
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/lowering.py", line 1570 in decref
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/lowering.py", line 1527 in delvar
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/lowering.py", line 547 in lower_inst
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/lowering.py", line 270 in lower_block
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/lowering.py", line 256 in lower_function_body
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/lowering.py", line 226 in lower_normal_function
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/lowering.py", line 187 in lower
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/typed_passes.py", line 468 in run_pass
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/compiler_machinery.py", line 273 in check
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/compiler_machinery.py", line 311 in _runPass
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/compiler_lock.py", line 35 in _acquire_compile_lock
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/compiler_machinery.py", line 356 in run
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/compiler.py", line 472 in _compile_core
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/compiler.py", line 506 in _compile_bytecode
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/compiler.py", line 438 in compile_extra
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/compiler.py", line 744 in compile_extra
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/dispatcher.py", line 107 in _compile_core
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/dispatcher.py", line 94 in _compile_cached
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/dispatcher.py", line 80 in compile
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/dispatcher.py", line 905 in compile
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/numba/core/dispatcher.py", line 375 in _compile_for_args
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/sparse/_common.py", line 454 in _dot
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/sparse/_common.py", line 177 in tensordot
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/array/routines.py", line 276 in _tensordot
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/utils.py", line 78 in apply
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/core.py", line 133 in _execute_task
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/core.py", line 127 in _execute_task
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/core.py", line 133 in <genexpr>
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/core.py", line 133 in _execute_task
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/core.py", line 163 in get
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/optimization.py", line 1028 in __call__
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/core.py", line 133 in _execute_task
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/core.py", line 127 in _execute_task
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/core.py", line 127 in _execute_task
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/core.py", line 133 in <genexpr>
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/core.py", line 133 in _execute_task
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/core.py", line 133 in <genexpr>
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/core.py", line 133 in _execute_task
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/core.py", line 133 in <genexpr>
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/core.py", line 133 in _execute_task
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/local.py", line 226 in execute_task
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/local.py", line 240 in batch_execute_tasks
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/concurrent/futures/thread.py", line 58 in run
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/concurrent/futures/thread.py", line 92 in _worker
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1012 in run
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1032 in _bootstrap
Thread 0x00007f5803c006c0 (most recent call first):
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/concurrent/futures/thread.py", line 89 in _worker
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1012 in run
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 1032 in _bootstrap
Thread 0x00007f583e2006c0 (most recent call first):
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/execnet/gateway_base.py", line 534 in read
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/execnet/gateway_base.py", line 567 in from_io
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/execnet/gateway_base.py", line 1160 in _thread_receiver
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/execnet/gateway_base.py", line 341 in run
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/execnet/gateway_base.py", line 411 in _perform_spawn
Thread 0x00007f584ca96580 (most recent call first):
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/threading.py", line 355 in wait
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/queue.py", line 171 in get
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/local.py", line 139 in queue_get
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/local.py", line 502 in get_async
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/threaded.py", line 91 in get
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/dask/base.py", line 660 in compute
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/xarray/namedarray/daskmanager.py", line 86 in compute
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/xarray/namedarray/pycompat.py", line 130 in to_duck_array
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/xarray/core/variable.py", line 981 in load
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/xarray/core/variable.py", line 1003 in compute
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/xarray/core/merge.py", line 137 in unique_variable
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/xarray/core/merge.py", line 290 in merge_collected
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/xarray/core/merge.py", line 701 in merge_core
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/xarray/core/merge.py", line 976 in merge
File "/home/pbourg/Projets/xscen/src/xscen/ensembles.py", line 711 in build_partition_data
File "/home/pbourg/Projets/xscen/tests/test_ensembles.py", line 1070 in test_build_partition_data
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/_pytest/python.py", line 159 in pytest_pyfunc_call
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/_pytest/python.py", line 1627 in runtest
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/_pytest/runner.py", line 174 in pytest_runtest_call
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/_pytest/runner.py", line 242 in <lambda>
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/_pytest/runner.py", line 341 in from_call
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/_pytest/runner.py", line 241 in call_and_report
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/_pytest/runner.py", line 132 in runtestprotocol
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/_pytest/runner.py", line 113 in pytest_runtest_protocol
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/xdist/remote.py", line 195 in run_one_test
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/xdist/remote.py", line 174 in pytest_runtestloop
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/_pytest/main.py", line 337 in _main
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/_pytest/main.py", line 283 in wrap_session
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/_pytest/main.py", line 330 in pytest_cmdline_main
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/xdist/remote.py", line 393 in <module>
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/execnet/gateway_base.py", line 1291 in executetask
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/execnet/gateway_base.py", line 341 in run
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/execnet/gateway_base.py", line 411 in _perform_spawn
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/execnet/gateway_base.py", line 389 in integrate_as_primary_thread
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/execnet/gateway_base.py", line 1273 in serve
File "/home/pbourg/miniforge3/envs/xscen-dev/lib/python3.12/site-packages/execnet/gateway_base.py", line 1806 in serve
File "<string>", line 8 in <module>
File "<string>", line 1 in <module>
Extension modules: zmq.backend.cython._zmq, tornado.speedups, numpy._core._multiarray_umath, numpy._core._multiarray_tests, numpy.linalg._umath_linalg, pyarrow.lib, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, pandas._libs.tslibs.ccalendar, pandas._libs.tslibs.np_datetime, pandas._libs.tslibs.dtypes, pandas._libs.tslibs.base, pandas._libs.tslibs.nattype, pandas._libs.tslibs.timezones, pandas._libs.tslibs.fields, pandas._libs.tslibs.timedeltas, pandas._libs.tslibs.tzconversion, pandas._libs.tslibs.timestamps, pandas._libs.properties, pandas._libs.tslibs.offsets, pandas._libs.tslibs.strptime, pandas._libs.tslibs.parsing, pandas._libs.tslibs.conversion, pandas._libs.tslibs.period, pandas._libs.tslibs.vectorized, pandas._libs.ops_dispatch, pandas._libs.missing, pandas._libs.hashtable, pandas._libs.algos, pandas._libs.interval, pandas._libs.lib, pyarrow._compute, pandas._libs.ops, bottleneck.move, bottleneck.nonreduce, bottleneck.nonreduce_axis, bottleneck.reduce, pandas._libs.hashing, pandas._libs.arrays, pandas._libs.tslib, pandas._libs.sparse, pandas._libs.internals, pandas._libs.indexing, pandas._libs.index, pandas._libs.writers, pandas._libs.join, pandas._libs.window.aggregations, pandas._libs.window.indexers, pandas._libs.reshape, pandas._libs.groupby, pandas._libs.json, pandas._libs.parsers, pandas._libs.testing, shapely.lib, shapely._geos, shapely._geometry_helpers, pyproj._compat, pyproj._context, pyproj._network, pyproj._version, pyproj._geod, pyproj.list, pyproj._crs, pyproj.database, pyproj._transformer, pyproj._sync, scipy._lib._ccallback_c, cftime._cftime, yaml._yaml, numba.core.typeconv._typeconv, numba._helperlib, numba._dynfunc, numba._dispatcher, numba.core.runtime._nrt_python, numba.np.ufunc._internal, numba.experimental.jitclass._box, cytoolz.utils, cytoolz.itertoolz, cytoolz.functoolz, cytoolz.dicttoolz, cytoolz.recipes, psutil._psutil_linux, psutil._psutil_posix, markupsafe._speedups, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.linalg._fblas, scipy.linalg._flapack, scipy.linalg.cython_lapack, scipy.linalg._cythonized_array_utils, scipy.linalg._solve_toeplitz, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_sqrtm_triu, scipy.linalg.cython_blas, scipy.linalg._matfuncs_expm, scipy.linalg._decomp_update, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.linalg._propack._spropack, scipy.sparse.linalg._propack._dpropack, scipy.sparse.linalg._propack._cpropack, scipy.sparse.linalg._propack._zpropack, scipy.sparse.csgraph._tools, scipy.sparse.csgraph._shortest_path, scipy.sparse.csgraph._traversal, scipy.sparse.csgraph._min_spanning_tree, scipy.sparse.csgraph._flow, scipy.sparse.csgraph._matching, scipy.sparse.csgraph._reordering, scipy._lib._uarray._uarray, scipy.special._ufuncs_cxx, scipy.special._ufuncs, scipy.special._specfun, scipy.special._comb, scipy.special._ellip_harm_2, scipy.fftpack.convolve, PIL._imaging, kiwisolver._cext, scipy.spatial._ckdtree, scipy._lib.messagestream, scipy.spatial._qhull, scipy.spatial._voronoi, scipy.spatial._distance_wrap, scipy.spatial._hausdorff, scipy.spatial.transform._rotation, scipy.optimize._group_columns, scipy.optimize._trlib._trlib, scipy.optimize._lbfgsb, _moduleTNC, scipy.optimize._moduleTNC, scipy.optimize._cobyla, scipy.optimize._slsqp, scipy.optimize._minpack, scipy.optimize._lsq.givens_elimination, scipy.optimize._zeros, scipy.optimize._highs.cython.src._highs_wrapper, scipy.optimize._highs._highs_wrapper, scipy.optimize._highs.cython.src._highs_constants, scipy.optimize._highs._highs_constants, scipy.linalg._interpolative, scipy.optimize._bglu_dense, scipy.optimize._lsap, scipy.optimize._direct, scipy.integrate._odepack, scipy.integrate._quadpack, scipy.integrate._vode, scipy.integrate._dop, scipy.integrate._lsoda, scipy.interpolate._fitpack, scipy.interpolate._dfitpack, scipy.interpolate._bspl, scipy.interpolate._ppoly, scipy.interpolate.interpnd, scipy.interpolate._rbfinterp_pythran, scipy.interpolate._rgi_cython, scipy.special.cython_special, scipy.stats._stats, scipy.stats._biasedurn, scipy.stats._levy_stable.levyst, scipy.stats._stats_pythran, scipy.stats._ansari_swilk_statistics, scipy.stats._sobol, scipy.stats._qmc_cy, scipy.stats._mvn, scipy.stats._rcont.rcont, scipy.stats._unuran.unuran_wrapper, scipy.ndimage._nd_image, _ni_label, scipy.ndimage._ni_label, _cffi_backend, lz4._version, lz4.frame._frame, zstandard.backend_c, netCDF4._netCDF4, numcodecs.compat_ext, numcodecs.blosc, numcodecs.zstd, numcodecs.lz4, numcodecs._shuffle, msgpack._cmsgpack, numcodecs.jenkins, numcodecs.vlen, numcodecs.fletcher32, h5py._errors, h5py.defs, h5py._objects, h5py.h5, h5py.utils, h5py.h5t, h5py.h5s, h5py.h5ac, h5py.h5p, h5py.h5r, h5py._proxy, h5py._conv, h5py.h5z, h5py.h5a, h5py.h5d, h5py.h5ds, h5py.h5g, h5py.h5i, h5py.h5o, h5py.h5f, h5py.h5fd, h5py.h5pl, h5py.h5l, h5py._selector, scipy.signal._sigtools, scipy.signal._max_len_seq_inner, scipy.signal._upfirdn_apply, scipy.signal._spline, scipy.signal._sosfilt, scipy.signal._spectral, scipy.signal._peak_finding_utils, statsmodels.tsa._innovations, sklearn.__check_build._check_build, sklearn.utils._isfinite, sklearn.utils.sparsefuncs_fast, sklearn.utils.murmurhash, sklearn.utils._openmp_helpers, sklearn.metrics.cluster._expected_mutual_info_fast, sklearn.preprocessing._csr_polynomial_expansion, sklearn.preprocessing._target_encoder_fast, sklearn.metrics._dist_metrics, sklearn.metrics._pairwise_distances_reduction._datasets_pair, sklearn.utils._cython_blas, sklearn.metrics._pairwise_distances_reduction._base, sklearn.metrics._pairwise_distances_reduction._middle_term_computer, sklearn.utils._heap, sklearn.utils._sorting, sklearn.metrics._pairwise_distances_reduction._argkmin, sklearn.metrics._pairwise_distances_reduction._argkmin_classmode, sklearn.utils._vector_sentinel, sklearn.metrics._pairwise_distances_reduction._radius_neighbors, sklearn.metrics._pairwise_distances_reduction._radius_neighbors_classmode, sklearn.metrics._pairwise_fast, sklearn.utils._fast_dict, sklearn.cluster._hierarchical_fast, sklearn.cluster._k_means_common, sklearn.cluster._k_means_elkan, sklearn.cluster._k_means_lloyd, sklearn.cluster._k_means_minibatch, sklearn.neighbors._partition_nodes, sklearn.neighbors._ball_tree, sklearn.neighbors._kd_tree, sklearn.utils.arrayfuncs, sklearn.utils._random, sklearn.utils._seq_dataset, sklearn.linear_model._cd_fast, _loss, sklearn._loss._loss, sklearn.svm._liblinear, sklearn.svm._libsvm, sklearn.svm._libsvm_sparse, sklearn.utils._weight_vector, sklearn.linear_model._sgd_fast, sklearn.linear_model._sag_fast, sklearn.decomposition._online_lda_fast, sklearn.decomposition._cdnmf_fast, sklearn.cluster._dbscan_inner, sklearn.cluster._hdbscan._tree, sklearn.cluster._hdbscan._linkage, sklearn.cluster._hdbscan._reachability, sklearn._isotonic, sklearn.tree._utils, sklearn.tree._tree, sklearn.tree._splitter, sklearn.tree._criterion, sklearn.neighbors._quad_tree, sklearn.manifold._barnes_hut_tsne, sklearn.manifold._utils, cartopy.trace, lz4.block._block (total: 281)
I have a solution, but it is a breaking change, PR coming.
The text was updated successfully, but these errors were encountered:
Turns out the memory issue fix in #387 is problematic in some setups. It never happens in our CI, but the
grid.destroy()
calls sometimes result in segmentation faults when used in parallel with dask. I haven't been able to find the exact reason for these errors, but I believe the reason is that theRegridder
object gets sent to the dask worker where the ESMpy grid object is not in the memory ?Errors look like :
I have a solution, but it is a breaking change, PR coming.
The text was updated successfully, but these errors were encountered: