Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiprocessing pool is breaking long shot scripts #22

Open
Jean-Reinhold opened this issue Dec 29, 2022 · 2 comments
Open

Multiprocessing pool is breaking long shot scripts #22

Jean-Reinhold opened this issue Dec 29, 2022 · 2 comments

Comments

@Jean-Reinhold
Copy link

On a Fedora system, after about two hours of operation, the pool used to extract features starts hanging open processes, resulting in a significant increase in memory usage.

The solution was to force the pool to close with pool.close(); pool.join():

Screenshot from 2022-12-29 09-21-22

I know it is super strange because the context manager should already do this...

@david-waterworth
Copy link

david-waterworth commented Feb 16, 2023

My understanding is the context manager calls terminate() so close() and join() are still required. See https://stackoverflow.com/a/55035444/2981639

@mjanson
Copy link

mjanson commented Dec 19, 2023

we also ran into an issue with multiprocessing from within a flask web service.

Would the project be open to a PR that allows the threading/multiprocessing to be disabled? Our current workaround is to directly invoke _get_feats which is undesirable for several reasons.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants