Skip to content

allow deadline alert UUID references in serialized Dag schema - #70148

Open
JH-A-Kim wants to merge 7 commits into
apache:mainfrom
JH-A-Kim:deadline-schema-uuid
Open

allow deadline alert UUID references in serialized Dag schema#70148
JH-A-Kim wants to merge 7 commits into
apache:mainfrom
JH-A-Kim:deadline-schema-uuid

Conversation

@JH-A-Kim

@JH-A-Kim JH-A-Kim commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Description

This pr is a continuation of PR #68195
Related: PR #67449

What happens: When a Dag defines a deadline alert, SerializedDagModel._generate_deadline_uuids rewrites the Dag's deadline field from a list of dicts into a list of UUID strings referencing rows in the deadline_alert table. This has a few different problems.

  1. schema.json only allowed deadline to be a dict, an array of dicts, or null so this would throw a validation error.

  2. _generate_deadline_uuids rewrote the passed-in dag.data in place. Any of the other code that still used the reference would then get something completely different from what they expected, that being a dict of deadlines as opposed to a list of UUIDs

  3. One of the possible fixes to the above using a deepcopy would then create a separate issue wherein test_dagrun_success_deadline and test_dagrun_deadline_variable_interval_stable would be using the original version of the dag data that has deadline as an actual dict instead of the changed version with deadlines as a list of strings that are UUIDs.

The fix

  1. Add an array of string option to the deadline schema, so it can accept the UUID.

  2. deepcopy dag.data inside write_dag before writing any of the UUID information so we prevent mutation of the caller object.

  3. In _write_dag, do session.flush() to get the updated information to the db and then from there just call down from the db as the single source of truth. Important thing to note is there is no commit so only that session sees the changes and no other sessions see it. (tldr of this fix is to make it so that it acts like a client server contract with video games wherein all the important information needs to be gathered from the source of truth and not locally.)

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Sonnet 5 following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@JH-A-Kim
JH-A-Kim marked this pull request as ready for review July 21, 2026 23:10
@JH-A-Kim

Copy link
Copy Markdown
Contributor Author

@ferruzzi may I have a review on this? Thank you!

@ferruzzi ferruzzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a couple small comments.

Comment thread airflow-core/tests/unit/models/test_serialized_dag.py Outdated
Comment thread airflow-core/src/airflow/models/serialized_dag.py Outdated
Comment thread airflow-core/src/airflow/models/serialized_dag.py Outdated
@ferruzzi

Copy link
Copy Markdown
Contributor

Pretty sure those failing CI tests were just fixed in main, give it a rebase and they should go green.

sjyangkevin and others added 7 commits July 29, 2026 11:13
resolve these failing test cases that were failing because deepcopy would make it so that the returned value does not have the uuids because it is a copy. And the test expected the uuids. Therefore the fix became editing _write_dag by flushing the data to the db so you can look at the data. And then after we can just grab the version that is now in the db and not the deepcopy that did not have the uuids. Basically this just means that whenever you want to check you reread it from the db instead of hoping that the old copy is still correct.
@JH-A-Kim
JH-A-Kim force-pushed the deadline-schema-uuid branch from c24004f to f9d8393 Compare July 29, 2026 18:13
@JH-A-Kim

Copy link
Copy Markdown
Contributor Author

@ferruzzi i believe this is ready for review again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants