Release TI lock before asset listener callbacks - #70951
Merged
vatsrahul1001 merged 4 commits intoAug 4, 2026
Merged
Conversation
uranusjr
requested review from
Lee-W,
XD-DENG,
amoghrajesh,
ashb and
kaxil
as code owners
August 2, 2026 21:35
jason810496
self-requested a review
August 3, 2026 01:52
jason810496
reviewed
Aug 3, 2026
jason810496
left a comment
jason810496
left a comment
Member
There was a problem hiding this comment.
No comments from my end, LGTM with this direction, thanks.
jason810496
reviewed
Aug 3, 2026
jason810496
left a comment
jason810496
left a comment
Member
There was a problem hiding this comment.
Could we also address the loading issue that #66854 (comment) comment pointed out in this one? Since this PR is small enough. Thanks.
Lee-W
approved these changes
Aug 3, 2026
Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock.
uranusjr
force-pushed
the
fix/asset-registration-lock-contention
branch
from
August 3, 2026 08:59
1815493 to
1abae3e
Compare
kaxil
reviewed
Aug 3, 2026
kaxil
approved these changes
Aug 3, 2026
kaxil
left a comment
kaxil
left a comment
Member
There was a problem hiding this comment.
2 minor comments lgtm otherwise
Andrushika
reviewed
Aug 3, 2026
eladkal
approved these changes
Aug 3, 2026
eladkal
added
type:bug-fix
Contributor
Backport successfully created: v3-3-testNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
|
1 task
vatsrahul1001
pushed a commit
that referenced
this pull request
Aug 4, 2026
…71065) * [v3-3-test] Release TI lock before asset listener callbacks (#70951) * Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com> * [v3-3-test] Explicitly rollback on task state update exception (#71076) --------- Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>
vatsrahul1001
pushed a commit
that referenced
this pull request
Aug 5, 2026
…71065) * [v3-3-test] Release TI lock before asset listener callbacks (#70951) * Release TI lock before asset listener callbacks Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts. The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock. * Optimize asset alias assoc insert * Fix exhausted iterator reuse bug * Test asset reg callback cases (cherry picked from commit 79db995) Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com> * [v3-3-test] Explicitly rollback on task state update exception (#71076) --------- Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Asset registration on the task-success path (ti_update_state) ran the listener hooks synchronously inside the transaction holding a row lock on the task_instance table. A slow listener, multiplied across a large fan-out of asset events, could hold that lock for minutes, causing statement timeouts.
The listener hooks are now deferred until the end of the endpoint instead of executed inline during asset event creation. Registration writes to the database still happen under the caller's transaction, so durability is unchanged; this only moves the best-effort listener hooks off the lock.
Close #66853.