Keep S3DeleteObjectsOperator validation in __init__, narrow to ValueError - #70359
Conversation
boring-cyborg
Bot
added
area:dev-tools
area:providers
backport-to-v3-3-test
|
Sorry, the issue has been linked to the PR and should be unlinked to prevent the issue from closing. My apologies. |
shahar1
changed the title
No worries, it's a matter of changing the "resolves:" into "related:" :) |
|
Thanks for updating the title. Will commit the code to fix the failure in the static checks after they all finish running. The Tests (AMD) check was failing due to an incorrect number in the code. |
42f5c65 to
08f503f
Compare
|
@shahar1 please let me know if there's anything you want me to change in this PR. I fixed all the static bugs and checks from a few days ago. |
shahar1
left a comment
shahar1
left a comment
There was a problem hiding this comment.
Could you please focus the PR on a single provider? (I know that GCP compute operators are being handled in another PR, so maybe just focus on S3DeleteObjectsOperator)
|
Reverted changes! This PR is only focused on S3 now. @shahar1 |
|
Checks just passed. Please let me know what you think! |
AmoghAtreya
changed the title
|
The checks will pass once #70505 gets merged due to the condition being introduced there. |
Please merge from/rebase onto main, and push to your branch. It will retrigger the CI with latest changes (I'll approve if necessary). |
43f0541 to
7a1b525
Compare
|
Since all checks are passing, please let me know next steps @shahar1! |
shahar1
changed the title
shahar1
changed the title
|
@AmoghAtreya |
|
Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions. |
Thank you so much for working with me through this and reviewing my code. Really appreciate that you took the time to help me out and provide feedback! Looking forward to contributing more. |
Addresses review feedback on apache#70723. The check asks whether `match_glob` was passed rather than inspecting its rendered value, so per the false positives section of apache#70296 it is a provision check and belongs in the constructor. Moving it to `execute()` was wrong on two counts: with `render_template_as_native_obj=True` a provided field can render to None, and raising per task instance hides a static authoring mistake that a Dag import error would surface immediately. Rewrite it in place instead, using `is not None` polarity rather than a truthiness test on the un-rendered string, and narrow `AirflowException` to `ValueError` to match apache#70359. Since apache#70505 narrowed the hook to sanction provision checks written this way, the exemption entry still goes. Narrowing removes the file's only `raise AirflowException`, so drop its entry from generated/known_airflow_exceptions.txt to keep the check-no-new-airflow-exceptions allowlist in sync. related: apache#70296 Signed-off-by: atikulmunna <atikul.munna@northsouth.edu>
2 participants
Description
related: #70296 — burn-down of
__init__-time validation exemptions.Changes Made (
S3DeleteObjectsOperator,amazon):__init__argument-combination guard with explicitis not Nonechecks so the
validate-operators-inithook recognizes it (the oldcomprehension-based check read as a false positive). The logic is unchanged —
equivalent on every input combination.
AirflowExceptiontoValueError(parse-time only; the
execute()-time guard is deliberately retainedunchanged as a safety net for templated
keysrendering toNone).S3DeleteObjectsOperatorentry fromscripts/ci/prek/validate_operators_init_exemptions.txtand updated thegenerated/known_airflow_exceptions.txtcount (5 → 4).ValueError.works on: #70296
Was generative AI tooling used to co-author this PR?
Generated-by: Claude following the guidelines