skip setproctitle in task_runner on Mac OS
#45124
Conversation
On some newer versions of Mac OS setproctitle can cause segfault benoitc/gunicorn#3021
There was a problem hiding this comment.
Yeah I've started noticed this causing tests on OSX to fail on occassion, but hadn't noticed any runtime issues. Makes sense though.
There are probably a few other cases where we setproctitle (in the dag parser code I just landed inside airflow/dag_processor/ could you update those too?)
| else: | ||
| from setproctitle import setproctitle | ||
| setproctitle("airflow scheduler -- DagFileProcessorManager") |
There was a problem hiding this comment.
| else: | |
| from setproctitle import setproctitle | |
| setproctitle("airflow scheduler -- DagFileProcessorManager") | |
| else: | |
| from setproctitle import setproctitle | |
| setproctitle("airflow scheduler -- DagFileProcessorManager") |
There was a problem hiding this comment.
this may actually be a GitHub bug, in the "Files changed" tab it shows the indentation being off, but in the conversation / timeline it shows the indentation as being correct (and my suggested change is unneeded indentation)
There was a problem hiding this comment.
I am in contact with setproctitle maintainer during the "Airflow Beach Cleaning" project. I can ask him to comment.
There was a problem hiding this comment.
After short discussion with @dvarrazzo - it's likely this dvarrazzo/py-setproctitle#144 is going to fix it (unreleased yet).
It would be great though to get some more details about those segfaults @jaketf @ashb when you see them happening again ?
| # setproctitle causes issue on Mac OS: https://github.com/benoitc/gunicorn/issues/3021 | ||
| os_type = sys.platform | ||
| if os_type == "darwin": | ||
| log.info("Mac OS detected, skipping setproctitle") |
There was a problem hiding this comment.
| log.info("Mac OS detected, skipping setproctitle") | |
| log.debug("Mac OS detected, skipping setproctitle") |
On some newer versions of Mac OS setproctitle can cause segfault benoitc/gunicorn#3021
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.