Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Async training does not work with macOS and Python 3.8 #572

Open
muupan opened this issue Nov 6, 2019 · 0 comments · May be fixed by #574
Open

Async training does not work with macOS and Python 3.8 #572

muupan opened this issue Nov 6, 2019 · 0 comments · May be fixed by #574
Labels

Comments

@muupan
Copy link
Member

muupan commented Nov 6, 2019

% python examples/gym/train_a3c_gym.py 2 --env CartPole-v0
/Users/fujita-rental/.local/share/virtualenvs/chainerrl-TtJB_mwx/lib/python3.8/site-packages/chainer/_environment_check.py:33: UserWarning: Accelerate has been detected as a NumPy backend library.
vecLib, which is a part of Accelerate, is known not to work correctly with Chainer.
We recommend using other BLAS libraries such as OpenBLAS.
For details of the issue, please see
https://docs.chainer.org/en/stable/tips.html#mnist-example-does-not-converge-in-cpu-mode-on-mac-os-x.

Please be aware that Mac OS X is not an officially supported OS.

  warnings.warn('''\
Traceback (most recent call last):
  File "examples/gym/train_a3c_gym.py", line 210, in <module>
    main()
  File "examples/gym/train_a3c_gym.py", line 196, in main
    experiments.train_agent_async(
  File "/Users/fujita-rental/dev/chainerrl/chainerrl/experiments/train_agent_async.py", line 251, in train_agent_async
    async_.run_async(processes, run_func)
  File "/Users/fujita-rental/dev/chainerrl/chainerrl/misc/async_.py", line 138, in run_async
    p.start()
  File "/Users/fujita-rental/.pyenv/versions/3.8.0/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/fujita-rental/.pyenv/versions/3.8.0/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/Users/fujita-rental/.pyenv/versions/3.8.0/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
    return Popen(process_obj)
  File "/Users/fujita-rental/.pyenv/versions/3.8.0/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/fujita-rental/.pyenv/versions/3.8.0/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/fujita-rental/.pyenv/versions/3.8.0/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Users/fujita-rental/.pyenv/versions/3.8.0/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'train_agent_async.<locals>.run_func'

I suspect this is due to the change made by Python 3.8 of the default mode of multiprocessing.

https://docs.python.org/3/library/multiprocessing.html

Changed in version 3.8: On macOS, the spawn start method is now the default. The fork start method should be considered unsafe as it can lead to crashes of the subprocess. See bpo-33725.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant