Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow users to set number of process pool workers/threads #203

Open
windowshopr opened this issue Nov 24, 2022 · 0 comments
Open

Allow users to set number of process pool workers/threads #203

windowshopr opened this issue Nov 24, 2022 · 0 comments

Comments

@windowshopr
Copy link

Would be nice to see the users be able to control the number of processors/threads used when defining 'multiprocessing' or 'multithreading'.

I would think you could pass in a worker argument (default can be None) to set_run_mode(), something like:

set_run_mode(obj_func, 'multiprocessing', n_workers=3)

..and then further down in the func_transformer() function:

    elif mode == 'multiprocessing':
        from multiprocessing import Pool
        
        pool = Pool()
        if n_workers != None:
            pool = Pool(n_workers)

        def func_transformed(X):
            return np.array(pool.map(func, X))

I'm not sure if this would be the appropriate or working syntax, but just an idea I had. Sometimes we don't want to just default to using ALL CPU cores when using multiprocessing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant