Add static check ensuring trigger `__init__()` and `serialize()` stay in sync by shahar1 · Pull Request #66960 · apache/airflow

shahar1 · 2026-05-14T18:11:27Z

A trigger's __init__ and serialize() are written as a pair: any __init__ parameter that serialize() does not return is silently dropped when the triggerer re-instantiates the trigger, falling back to the parameter's default. This kind of mismatch is easy to introduce and hard to spot in review.

This adds an AST-based prek static check over provider trigger modules (*/src/airflow/providers/*/triggers/*.py) that flags such mismatches.

Inspiration for this PR: #66917

Details

Resolves the __init__/serialize() pair through in-file base classes, so a class that inherits one and overrides the other is still checked. Also follows **super().serialize()[1] spreads recursively.
When neither the signature nor the serialize dict can be resolved statically (dynamic dict construction, .update(), base class in another file), it skips rather than guessing — no false positives.
8 pre-existing failures were found and reviewed:
- 5 genuine violations are listed in KNOWN_VIOLATIONS and excluded for now; they will be fixed in a follow-up PR (DatabricksExecutionTrigger, DatabricksSQLStatementExecutionTrigger, BigQueryIntervalCheckTrigger, DataFusionStartPipelineTrigger, LivyTrigger).
- 3 by-design cases are permanently excluded in BY_DESIGN_EXCLUSIONS with documented reasons — a deprecated/aliased parameter (delta → moment, pod_name → pod_names) is folded into its replacement at construction time, so it does not need to round-trip.

Was generative AI tooling used to co-author this PR?

Yes — Claude Code (Sonnet 4.6)

Generated-by: Claude Code (Sonnet 4.6) following the guidelines

Trigger __init__ and serialize() are written as a pair: any __init__ parameter that serialize() does not return is silently dropped when the triggerer re-instantiates the trigger, falling back to the parameter's default. This adds an AST-based prek static check over provider triggers that flags such mismatches, resolving __init__/serialize() pairs through in-file base classes (including **super().serialize() spreads). Five existing violations are excluded as KNOWN_VIOLATIONS pending a follow-up fix; three by-design cases (deprecated/aliased params folded into their replacement at construction time) are permanently excluded.

Co-authored-by: Wei Lee <weilee.rx@gmail.com>

Rename based on Lee-W's review feedback: - _init_param_names → _get_init_param_names - _base_simple_names → _get_base_simple_names - _method → _get_method - _serialize_keys → _get_serialize_keys - _super_serialize_keys → _get_super_serialize_keys - violations → get_violations

potiuk

Super nice! we needed it :)

github-actions · 2026-05-16T20:59:20Z

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test

…lize()` stay in sync (apache#66960) * Add static check ensuring trigger __init__ and serialize() stay in sync Trigger __init__ and serialize() are written as a pair: any __init__ parameter that serialize() does not return is silently dropped when the triggerer re-instantiates the trigger, falling back to the parameter's default. This adds an AST-based prek static check over provider triggers that flags such mismatches, resolving __init__/serialize() pairs through in-file base classes (including **super().serialize() spreads). Five existing violations are excluded as KNOWN_VIOLATIONS pending a follow-up fix; three by-design cases (deprecated/aliased params folded into their replacement at construction time) are permanently excluded. * Update scripts/ci/prek/check_trigger_serialize_init.py Co-authored-by: Wei Lee <weilee.rx@gmail.com> * Rename noun-style helper functions to _get_/get_ form Rename based on Lee-W's review feedback: - _init_param_names → _get_init_param_names - _base_simple_names → _get_base_simple_names - _method → _get_method - _serialize_keys → _get_serialize_keys - _super_serialize_keys → _get_super_serialize_keys - violations → get_violations --------- (cherry picked from commit 49958a5) Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com> Co-authored-by: Wei Lee <weilee.rx@gmail.com>

…lize()` stay in sync (#66960) * Add static check ensuring trigger __init__ and serialize() stay in sync Trigger __init__ and serialize() are written as a pair: any __init__ parameter that serialize() does not return is silently dropped when the triggerer re-instantiates the trigger, falling back to the parameter's default. This adds an AST-based prek static check over provider triggers that flags such mismatches, resolving __init__/serialize() pairs through in-file base classes (including **super().serialize() spreads). Five existing violations are excluded as KNOWN_VIOLATIONS pending a follow-up fix; three by-design cases (deprecated/aliased params folded into their replacement at construction time) are permanently excluded. * Update scripts/ci/prek/check_trigger_serialize_init.py Co-authored-by: Wei Lee <weilee.rx@gmail.com> * Rename noun-style helper functions to _get_/get_ form Rename based on Lee-W's review feedback: - _init_param_names → _get_init_param_names - _base_simple_names → _get_base_simple_names - _method → _get_method - _serialize_keys → _get_serialize_keys - _super_serialize_keys → _get_super_serialize_keys - violations → get_violations --------- (cherry picked from commit 49958a5) Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com> Co-authored-by: Wei Lee <weilee.rx@gmail.com>

…lize()` stay in sync (#66960) (#67048) * [v3-2-test] Skip test_schedule_tis_start_trigger pending #55068 backport decision (#66315) The `start_from_trigger` deferred-at-schedule path is commented out on v3-2-test (disabled in 91e1029 as a TODO) and was only re-enabled on main by #55068, which landed one day after the v3-2 branch was cut. The test asserts the feature works and fails on v3-2-test CI. Skip the test on the 3.2 line and link the tracking issue from both the test skip-reason and the disabled production-code site so the follow-up isn't lost. Decision (backport #55068 or formally drop the feature on 3.2) is tracked in #66307. * [v3-2-test] Add static check ensuring trigger `__init__()` and `serialize()` stay in sync (#66960) * Add static check ensuring trigger __init__ and serialize() stay in sync Trigger __init__ and serialize() are written as a pair: any __init__ parameter that serialize() does not return is silently dropped when the triggerer re-instantiates the trigger, falling back to the parameter's default. This adds an AST-based prek static check over provider triggers that flags such mismatches, resolving __init__/serialize() pairs through in-file base classes (including **super().serialize() spreads). Five existing violations are excluded as KNOWN_VIOLATIONS pending a follow-up fix; three by-design cases (deprecated/aliased params folded into their replacement at construction time) are permanently excluded. * Update scripts/ci/prek/check_trigger_serialize_init.py Co-authored-by: Wei Lee <weilee.rx@gmail.com> * Rename noun-style helper functions to _get_/get_ form Rename based on Lee-W's review feedback: - _init_param_names → _get_init_param_names - _base_simple_names → _get_base_simple_names - _method → _get_method - _serialize_keys → _get_serialize_keys - _super_serialize_keys → _get_super_serialize_keys - violations → get_violations --------- (cherry picked from commit 49958a5) Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com> Co-authored-by: Wei Lee <weilee.rx@gmail.com> * Fix SageMaker/MLEngine triggers dropping params on triggerer restart The new trigger __init__/serialize() sync check surfaced two pre-existing violations on v3-2-test (the triggers do not exist or are already correct on main, so they were not in the cherry-picked KNOWN_VIOLATIONS list): - SageMakerNotebookJobTrigger.serialize() returned a non-existent self.poll_interval (an AttributeError on triggerer round-trip) and dropped waiter_delay / waiter_max_attempts. Serialize the stored waiter_* params instead, matching the fixed version on main. - MLEngineStartTrainingJobTrigger.serialize() dropped gcp_conn_id and impersonation_chain, silently resetting them to defaults when the triggerer re-instantiates the trigger. Serialize both. This makes the backported check pass without adding new KNOWN_VIOLATIONS. --------- Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com> Co-authored-by: Wei Lee <weilee.rx@gmail.com>

shahar1 requested review from amoghrajesh, ashb, bugraoz93, gopidesupavan, jason810496, jscheffl and potiuk as code owners May 14, 2026 18:11

boring-cyborg Bot added area:dev-tools area:providers backport-to-v3-2-test labels May 14, 2026

shahar1 mentioned this pull request May 14, 2026

Fix provider triggers whose serialize() drops __init__ parameters #66961

Closed

6 tasks

This was referenced May 14, 2026

Include endpoint_prefix in LivyTrigger serialization #66966

Merged

Serialize poll_interval and impersonation_chain on DataFusionStartPipelineTrigger #66968

Merged

shahar1 requested a review from Lee-W May 15, 2026 07:04

Lee-W reviewed May 15, 2026

View reviewed changes

Comment thread scripts/ci/prek/check_trigger_serialize_init.py Outdated

Comment thread scripts/ci/prek/check_trigger_serialize_init.py Outdated

abhid-007 mentioned this pull request May 15, 2026

Include dataset_id, table_id, poll_interval in BigQueryIntervalCheckTrigger serialization #66990

Closed

1 task

shahar1 force-pushed the check-trigger-serialize-init-compat branch from 3730302 to 36f07ba Compare May 15, 2026 15:41

shahar1 requested a review from Lee-W May 15, 2026 15:43

shahar1 changed the title ~~Add static check ensuring trigger __init__ and serialize() stay in sync~~ Add static check ensuring trigger __init__() and serialize() stay in sync May 15, 2026

shahar1 mentioned this pull request May 15, 2026

AGENTS.md: add function naming convention #66997

Merged

1 task

shahar1 and others added 3 commits May 16, 2026 18:19

Update scripts/ci/prek/check_trigger_serialize_init.py

f49a4e8

Co-authored-by: Wei Lee <weilee.rx@gmail.com>

shahar1 force-pushed the check-trigger-serialize-init-compat branch from 36f07ba to ac6ed56 Compare May 16, 2026 15:19

potiuk reviewed May 16, 2026

View reviewed changes

potiuk approved these changes May 16, 2026

View reviewed changes

potiuk merged commit 49958a5 into apache:main May 16, 2026
275 of 276 checks passed

shahar1 deleted the check-trigger-serialize-init-compat branch May 17, 2026 19:47

dkranchii mentioned this pull request May 21, 2026

Resolve dict-via-variable trigger serialize() in static check #67300

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add static check ensuring trigger `init()` and `serialize()` stay in sync#66960

Add static check ensuring trigger `init()` and `serialize()` stay in sync#66960
potiuk merged 3 commits into
apache:mainfrom
shahar1:check-trigger-serialize-init-compat

shahar1 commented May 14, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

potiuk left a comment

Uh oh!

Uh oh!

github-actions Bot commented May 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

shahar1 commented May 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details

Was generative AI tooling used to co-author this PR?

Uh oh!

Uh oh!

Uh oh!

potiuk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented May 16, 2026

Backport successfully created: v3-2-test

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

shahar1 commented May 14, 2026 •

edited

Loading