Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AIP-72: Handling AirflowException in task sdk #45308

Merged
merged 1 commit into from
Dec 31, 2024

Conversation

amoghrajesh
Copy link
Contributor

PR: #45106 provides a machinery to handle "failure" / "retry" states in the execution API server itself.

This PR handles the AirflowException if thrown by tasks.

Testing results

DAG:

from airflow import DAG
from airflow.exceptions import AirflowException
from airflow.providers.standard.operators.python import PythonOperator


def print_hello():
    raise AirflowException("hi i am AirflowException!")

with DAG(
    dag_id="afexception",
    schedule=None,
    catchup=False,
) as dag:
    hello_task = PythonOperator(
        task_id="afexception_task",
        retries=2,
        python_callable=print_hello,
    )

Legacy

  1. With retries
    image (12)

  2. Without retries
    image (13)

Task SDK

  1. With retries
image
  1. Without retries
image

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@amoghrajesh
Copy link
Contributor Author

@jscheffl this is a simple one, would like an approval if you are around.

Copy link
Contributor

@jscheffl jscheffl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am around... and I thought I had viewed and approved? Anyway... here you are!

@amoghrajesh
Copy link
Contributor Author

I am around... and I thought I had viewed and approved? Anyway... here you are!

Thanks Jens. Had to check, cos holiday season :)

@amoghrajesh amoghrajesh merged commit 24caa2b into apache:main Dec 31, 2024
45 checks passed
@amoghrajesh amoghrajesh deleted the AIP72-afexception branch December 31, 2024 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants