Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Endless warning message when use imap #269

Open
BeiYaaa opened this issue Jul 18, 2023 · 0 comments
Open

Endless warning message when use imap #269

BeiYaaa opened this issue Jul 18, 2023 · 0 comments

Comments

@BeiYaaa
Copy link

BeiYaaa commented Jul 18, 2023

Hi there,

I'm running below script on Google Vertex AI Jupyter Notebook.

import pathos as pa
with PPool(pa.helpers.cpu_count()) as p:
                    # make sure the pool is started
                    try:
                        p.restart()
                    except:
                        pass
                    
                    record_df_ls = []
                    record_df_ls = p.imap(self._fit_n_evaluate, param_grid)
                    
                    # close the process pool
                    p.close()
                    # wait for all tasks to complete
                    p.join()

I was able to run it successfully before. However, today I notice that there are warning messages printed on the screen and I couldn't kill the kernel. Below is part of the recurring and endless output.
File "/opt/conda/lib/python3.7/site-packages/dill/_dill.py", line 286, in loads return load(file, ignore, **kwds) Process ForkPoolWorker-9: Traceback (most recent call last): File "/opt/conda/lib/python3.7/site-packages/multiprocess/process.py", line 297, in _bootstrap self.run() File "/opt/conda/lib/python3.7/site-packages/multiprocess/process.py", line 99, in run self._target(*self._args, **self._kwargs) File "/opt/conda/lib/python3.7/site-packages/multiprocess/pool.py", line 110, in worker task = get() File "/opt/conda/lib/python3.7/site-packages/multiprocess/queues.py", line 357, in get return _ForkingPickler.loads(res) File "/opt/conda/lib/python3.7/site-packages/dill/_dill.py", line 272, in load return Unpickler(file, ignore=ignore, **kwds).load() File "/opt/conda/lib/python3.7/site-packages/dill/_dill.py", line 419, in load obj = StockUnpickler.load(self) File "/opt/conda/lib/python3.7/site-packages/dill/_dill.py", line 942, in _create_namedtuple t = collections.namedtuple(name, fieldnames, defaults=defaults, module=modulename) Process ForkPoolWorker-10: File "/opt/conda/lib/python3.7/collections/__init__.py", line 370, in namedtuple raise ValueError('Field names cannot start with an underscore: ' Traceback (most recent call last): ValueError: Field names cannot start with an underscore: '_1' File "/opt/conda/lib/python3.7/site-packages/multiprocess/process.py", line 297, in _bootstrap self.run() File "/opt/conda/lib/python3.7/site-packages/multiprocess/process.py", line 99, in run self._target(*self._args, **self._kwargs) Process ForkPoolWorker-11: File "/opt/conda/lib/python3.7/site-packages/multiprocess/pool.py", line 110, in worker task = get() File "/opt/conda/lib/python3.7/site-packages/multiprocess/queues.py", line 357, in get return _ForkingPickler.loads(res)

I noticed that it says ValueError can't start with an underscore. But I don't have file starting with an underscore in the fit function. This confuses me a lot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant