Skip to content

Emit GCSToGCSOperator deprecation warnings after rendering - #70449

Merged
potiuk merged 1 commit into
apache:mainfrom
haseebmalik18:fix-gcs-to-gcs-template-validation
Jul 31, 2026
Merged

Emit GCSToGCSOperator deprecation warnings after rendering#70449
potiuk merged 1 commit into
apache:mainfrom
haseebmalik18:fix-gcs-to-gcs-template-validation

Conversation

@haseebmalik18

Copy link
Copy Markdown
Contributor

source_object, source_objects, and delimiter on GCSToGCSOperator are template fields, so Jinja renders them after __init__ runs. The wildcard/delimiter deprecation warnings were emitted in __init__, so they checked the raw {{ ... }} expressions instead of the rendered values. This moves them into a helper that runs at the start of execute().

Part of #70296.


@boring-cyborg boring-cyborg Bot added area:dev-tools area:providers backport-to-v3-3-test Backport to v3-3-test provider:google Google (including GCP) related issues labels Jul 25, 2026
@shahar1 shahar1 removed the backport-to-v3-3-test Backport to v3-3-test label Jul 25, 2026
@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label Jul 28, 2026
@haseebmalik18
haseebmalik18 force-pushed the fix-gcs-to-gcs-template-validation branch from 2930317 to 67dcc24 Compare July 29, 2026 07:00

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks — correct fix, and the best-tested version of this pattern I've seen in the current batch. source_object, source_objects and delimiter are all template fields, so emitting these deprecation warnings from __init__ checked the raw "{{ ... }}" strings: a templated wildcard never warned, and a literal one warned at parse time for every Dag processor loop rather than once per task run.

test_wildcard_deprecation_warning_uses_rendered_source_object is the part that matters — it builds a real DAG and asserts the warning fires on the rendered value. That's the actual contract, and it's what distinguishes this from a mechanical move.

Pulling the three warnings into _warn_on_deprecated_template_fields() rather than inlining them in execute also keeps execute readable.

Note there's a near-identical PR in #70621, opened three days after this one. I've suggested there that it be closed in favour of this one — this came first and has the stronger test. No action needed from you.

Two small things inline.


Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting

Comment thread providers/google/tests/unit/google/cloud/transfers/test_gcs_to_gcs.py Outdated
@haseebmalik18
haseebmalik18 force-pushed the fix-gcs-to-gcs-template-validation branch from 67dcc24 to ab227d3 Compare July 30, 2026 20:57
@haseebmalik18

Copy link
Copy Markdown
Contributor Author

@potiuk Made requested changes

@potiuk
potiuk merged commit 25c100c into apache:main Jul 31, 2026
86 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:dev-tools area:providers provider:google Google (including GCP) related issues ready for maintainer review Set after triaging when all criteria pass.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants