Skip to content

Narrow AirflowException to specific exceptions in DataSync operator#70152

Open
haseebmalik18 wants to merge 1 commit into
apache:mainfrom
haseebmalik18:narrow-airflowexception-datasync
Open

Narrow AirflowException to specific exceptions in DataSync operator#70152
haseebmalik18 wants to merge 1 commit into
apache:mainfrom
haseebmalik18:narrow-airflowexception-datasync

Conversation

@haseebmalik18

Copy link
Copy Markdown
Contributor

The AWS DataSync operator raised the catch-all AirflowException for ten unrelated conditions, so a DAG author could not distinguish an invalid constructor call from a failed task execution when handling errors.

This narrows each usage to a specific type:

  • Usage/validation errors now raise ValueError.
  • Runtime domain failures raise dedicated exceptions added to
    aws/exceptions.py:
    • DataSyncTaskNotFoundError
    • DataSyncMultipleTasksError / DataSyncMultipleLocationsError
    • DataSyncLocationNotFoundError
    • DataSyncTaskCreationError
    • DataSyncTaskExecutionFailedError

The new classes subclass AirflowException, so existing except AirflowException handlers keep working, with no breaking change.

Tests are updated to assert the specific types, with added coverage for the raise sites that were previously untested.

Part of the ongoing effort to reduce direct AirflowException usage. The check-no-new-airflow-exceptions allowlist entry for this file is tightened from 10 to 0.

@haseebmalik18
haseebmalik18 requested a review from o-nikolas as a code owner July 21, 2026 00:55
@boring-cyborg boring-cyborg Bot added area:providers provider:amazon AWS/Amazon - related issues labels Jul 21, 2026
valid = True
if not valid:
raise AirflowException(
raise ValueError(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

switching this from AirflowException to ValueError is a behavior change for anyone catching AirflowException around operator construction, since ValueError isn't in that hierarchy. the raises in execute() all stay AirflowException subclasses, so this one's now the odd one out. intentional?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, intentional. The constructor check is argument validation, so ValueError fits it better, while the execute failures stay AirflowException subclasses. It's also at parse time where the DAG loader catches broad Exception, so a bad config still surfaces as a normal import error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants