[v3-3-test] Make airflow dags test wait for Human-in-the-loop input instead of hanging (#68492)#69104
Merged
Merged
Conversation
…nstead of hanging (#68492) * Make airflow dags test wait for Human-in-the-loop input instead of hanging Previously a HITL task that parked in awaiting_input made `airflow dags test` loop "No tasks to run. unrunnable tasks: ..." once per second forever, with no way to make progress (the in-process runner also swallows SIGTERM, so even `timeout` could not stop it). dag.test() now treats parked HITL tasks as waiting rather than unrunnable, and never resolves them itself: the run stays alive, logging which tasks await input, until a response recorded from outside flips them back to SCHEDULED -- at which point the existing loop resumes them. This matches how a parked task behaves on a real deployment, and the existing response channels work unchanged: the Required Actions UI or the HITL REST API (PATCH .../hitlDetails) of an api-server sharing the metadata database (e.g. airflow standalone). Humans and AI agents can drive HITL pipelines locally by running dags test and submitting the response through that API. Interactive console prompting in the dags test CLI is left for a follow-up. * Document the HITL REST API calls for responding during dags test (cherry picked from commit 63cee22) Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
1 task
vatsrahul1001
approved these changes
Jun 28, 2026
Contributor
Author
|
Hi maintainer, this PR was merged without a milestone set.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Previously a HITL task that parked in awaiting_input made
airflow dags testloop "No tasks to run. unrunnable tasks: ..." once per secondforever, with no way to make progress (the in-process runner also
swallows SIGTERM, so even
timeoutcould not stop it).dag.test() now treats parked HITL tasks as waiting rather than
unrunnable, and never resolves them itself: the run stays alive, logging
which tasks await input, until a response recorded from outside flips
them back to SCHEDULED -- at which point the existing loop resumes them.
This matches how a parked task behaves on a real deployment, and the
existing response channels work unchanged: the Required Actions UI or
the HITL REST API (PATCH .../hitlDetails) of an api-server sharing the
metadata database (e.g. airflow standalone). Humans and AI agents can
drive HITL pipelines locally by running dags test and submitting the
response through that API.
Interactive console prompting in the dags test CLI is left for a
follow-up.
(cherry picked from commit 63cee22)
Co-authored-by: Kaxil Naik kaxilnaik@gmail.com