Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Halo Update Fails Quietly on Buffer Size Mismatch #15

Open
oelbert opened this issue Feb 15, 2024 · 0 comments
Open

Halo Update Fails Quietly on Buffer Size Mismatch #15

oelbert opened this issue Feb 15, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@oelbert
Copy link
Collaborator

oelbert commented Feb 15, 2024

Is your feature request related to a problem? Please describe.
If there is a buffer mismatch in a halo update it will crash, but not describe the cause in the stack trace. For example, running a scalar halo update on a Quantity with dimensions X_DIM, Y_INTERFACE_DIM crashes but the stack trace goes into mpi4y:

`File /usr/local/lib/python3.8/site-packages/ndsl/comm/communicator.py:362, in Communicator.halo_update(self, quantity, n_points)
359 quantities = quantity
361 halo_updater = self.start_halo_update(quantities, n_points)
--> 362 halo_updater.wait()

File /usr/local/lib/python3.8/site-packages/ndsl/halo/updater.py:284, in HaloUpdater.wait(self)
282 send_req.wait()
283 for recv_req in self._recv_requests:
--> 284 recv_req.wait()
286 # Unpack buffers (updated by MPI with neighbouring halos)
287 # to proper quantities
288 with self._timer.clock("unpack"):

File mpi4py/MPI/Request.pyx:266, in mpi4py.MPI.Request.wait()

File mpi4py/MPI/msgpickle.pxi:450, in mpi4py.MPI.PyMPI_wait()`

Describe the solution you'd like
It would be nice to have something in the halo update code that raises this immediately, at least when the x and y dimensions are mismatched on a scalar halo update

@oelbert oelbert added the enhancement New feature or request label Feb 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant