Prevent start trigger initialization in scheduler#39585
Conversation
d621388 to
657a1cf
Compare
657a1cf to
926b3d3
Compare
|
maybe we should mark this experimental. trigger_kwargs being None vs {} does not seem like an explicit / obvious enough way to decide whether to start from trigger. why wouldn't we just look at the |
926b3d3 to
ec0a768
Compare
I'm ok with it. maybe wait for others' comment? if we're to mark it as experimental, where should I do so?
In the previous PR, @uranusjr and I discussed whether we could infer |
529ecc1 to
12fb69d
Compare
445274d to
dac10f5
Compare
4b0ac4a to
5f99ce7
Compare
…art_trigger_kwargs
…ss StartTriggerArgs
…her to start task execution from triggerer
8dbe9ff to
e1805e5
Compare
…n scheduler (apache#39585) * fix(baseoperator): change `start_trigger` into `start_trigger_args` * feat(baseoperator): add `start_from_trigger` as the flag to decide whether to start task execution from triggerer * fix(dagrun): set start_date before deferring task from scheduler
Why
During #38674, I introduced a logic that might run user code in the scheduler here. This PR intent to propose a new logic that could avoid this.
What
StartTriggerArgsclass that should containtrigger_cls(or maybe it could be renamedtrigger_cls_path?),tirgger_kwargs,timeout, andnext_method, which are what we need when we run an operator in deferrable mode.suggestion needed:move it toairflow.models.abstractoperatormight not be the best module to put this data class but might need some suggestionairflow.triggers.base(Thanks @jedcunningham !)start_from_triggerflag to allow an operator to be executed as normal or from the triggererOperator authors will now need to set the
start_trigger_args,start_from_trigger(or maybestart_from_triggerer?) this one to start execution directly from the triggerer.Note that the mapped operator is still not supported, but a draft PR #39912 has been created.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.