Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to use @trace_and_save with multiprocessing #385

Open
adamrobcarter opened this issue Dec 20, 2023 · 1 comment
Open

Unable to use @trace_and_save with multiprocessing #385

adamrobcarter opened this issue Dec 20, 2023 · 1 comment

Comments

@adamrobcarter
Copy link

When trying to use @trace_and_save on a function that runs in a separate thread (using multiprocessing.Pool), I get the following error

Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/home/acarter/.local/lib/python3.10/site-packages/viztracer/decorator.py", line 49, in wrapper
    tracer.fork_save(file_name)
  File "/home/acarter/.local/lib/python3.10/site-packages/viztracer/viztracer.py", line 254, in fork_save
    p.start()
  File "/usr/lib/python3.10/multiprocessing/process.py", line 118, in start
    assert not _current_process._config.get('daemon'), \
AssertionError: daemonic processes are not allowed to have children

I think this is because VizTracer tries to do it's thing in a new process in a new process to minimize performance impacts (docs), but you can't start a new process pool from within one (or something like that). Perhaps there could be an option for @trace_and_save to do the VizTracer work in the same process? I'd be happy to work on this

@gaogaotiantian
Copy link
Owner

Sure, if you want to work on this, you can add an extra argument fork_save=True to@trace_and_save. The user can set it to False if they want to save the trace on the same process.

Just make sure you write the test for it to keep the 100% coverage rate :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants