Align Dag Runs docs with Airflow 3 trigger timetable defaults#70167
Align Dag Runs docs with Airflow 3 trigger timetable defaults#70167Tegh25 wants to merge 2 commits into
Conversation
|
Tagging @ferruzzi for visibility |
| :ref:`CronTriggerTimetable` for ``@daily`` and ``catchup=False``, the scheduler | ||
| does **not** create a run for the past midnight. The next Dag run is created at | ||
| midnight on the morning of 2016-01-03, with ``data_interval_start`` and | ||
| ``data_interval_end`` both equal to that trigger time. | ||
|
|
There was a problem hiding this comment.
Are you sure about this one? Based on my understanding and what is already said above, I would expect the previous midnight to get run.
There was a problem hiding this comment.
When writing this section, I was referencing this example on the timetable doc.
For example, suppose there is a cron expression @daily or 0 0 * * *, which is scheduled to run at 12AM every day. If you enable Dags using the two timetables at 3PM on January 31st, CronTriggerTimetable creates a new Dag run at 12AM on February 1st. The run_id timestamp is midnight, on February 1st.
In this case after enablement, the next Dag run is created on the following midnight, not the previous one.
There was a problem hiding this comment.
Something isn't adding up. If you check out the unit test test_daily_cron_trigger_no_catchup_first_starts_at_next_schedule (specifically the first-run case) here it seems to be testing this and asserting the opposite. Any chance you are using out of date docs to update out of date docs? LOL Have a look at the test, maybe I'm just mis-reading it.
| of a Dag run, for example, denotes the start of the data interval, not when the | ||
| Dag is actually executed. | ||
| of a Dag run is defined by the timetable: for a trigger timetable it is the | ||
| trigger time; for a data-interval timetable it is the start of the data |
There was a problem hiding this comment.
That's only true for zero-width intervals, otherwise it's trigger_time-interval, right?
There was a problem hiding this comment.
I see what you mean. You're referring to CronTriggerTimetable or DeltaTriggerTimetable with a non-zero interval= argument, right?
You're right: DagRunInfo.logical_date is always data_interval.start. For the default zero-width case (interval=timedelta()), that equals the trigger time. With a non-zero interval, the run still fires at the cron tick T, but the data interval is [T - interval, T], so logical_date is T - interval (i.e. data_interval_start), not T.
I can adjust that sentence so it doesn't exclude the non-default case.
e.g. that for a trigger timetable logical_date is data_interval_start, which equals the trigger time when the interval is zero-width (the default).
| scheduled one interval after ``start_date``. | ||
| the same logical date, it marks the start of scheduling for the Dag (the first | ||
| possible logical date), not when tasks in the Dag will start running. In other | ||
| words, a Dag run will only be scheduled once the timetable advances past |
There was a problem hiding this comment.
Nipick, it looks like it's a >=, so "reaches" might be technically more accurate than "advances past"?
There was a problem hiding this comment.
Agreed. I can make the change to "reaches".
Tegh25
changed the title
Prior version assumed that all trigger timetable dags had a zero-width interval, which is not true. Also adjusted "advances past" to "reaches" for more correctness.
2 participants
The Data Interval and Catchup sections in
dag-run.rststill described@dailyand catchup as ifCronDataIntervalTimetablewere the default. That matches Airflow 2 /create_cron_data_intervals=True, not Airflow 3, where bare cron strings resolve toCronTriggerTimetableby default.Update the Data Interval section to explain that interval semantics depend on the timetable: trigger timetables use zero-width intervals at the trigger time; data-interval timetables use contiguous windows and schedule after the
window ends. Link to the timetable comparison docs for
logical_dateandrun_iddifferences.Update the Catchup section and tutorial example to describe behavior under the Airflow 3 default (
CronTriggerTimetable,catchup=False), and keep the prior data-interval story as the explicit opt-in case (CronDataIntervalTimetableorcreate_cron_data_intervals=True). Use timetable-neutral wording ("scheduled run times") in the catchup intro so itapplies to both timetable kinds.
No newsfragment: documentation-only clarification aligned with existing
authoring-and-scheduling/timetable.rstcontent.Was generative AI tooling used to co-author this PR?
Generated-by: Cursor (Auto) following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.