Skip to content

feat: Add lazy filtering support for inlet events by time range, ordering, and limit#54891

Merged
Lee-W merged 7 commits into
apache:mainfrom
hussein-awala:inlet_events_filers
Oct 20, 2025
Merged

feat: Add lazy filtering support for inlet events by time range, ordering, and limit#54891
Lee-W merged 7 commits into
apache:mainfrom
hussein-awala:inlet_events_filers

Conversation

@hussein-awala

Copy link
Copy Markdown
Member

This PR extends the asset events API client and inlet events accessor to support additional query options while keeping the accessor lazy, meaning events are only fetched when accessed.

Changes

  • Updated InletEventsAccessors.__getitem__ to accept optional filtering arguments (after, before, ascending, limit) without breaking existing usage (new key type is tuple[<existing key type>, dict])
  • The accessor continues to lazily fetch events from SUPERVISOR_COMMS only when accessed, but it will only fetch the events that match the filters when they are provided.

Impact

  • Existing calls like accessors[asset] continue to work as before.
  • New functionality allows lazy, filtered queries by time range, limit, and sort order.
  • Enables more efficient and flexible retrieval of asset event history without eager fetching.

Use Cases

  • Useful when the asset is used as input and not as a trigger source.
  • Easily retrieve the latest available value (ascending=False and limit=1).
  • Filter by date range for backfill scenarios without fetching the full history and filtering locally.

@boring-cyborg boring-cyborg Bot added area:API Airflow's REST/HTTP API area:task-sdk labels Aug 24, 2025
@hussein-awala hussein-awala added the type:improvement Changelog: Improvements label Aug 25, 2025
@hussein-awala
hussein-awala marked this pull request as ready for review August 26, 2025 22:46
Comment thread task-sdk/src/airflow/sdk/execution_time/context.py Outdated
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions Bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Oct 20, 2025

@uranusjr uranusjr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some optimisation to be had (specifically how we calculate _asset_events for verious things, somethings not needed), but I’m happy with the interface and we can just improve incrementally when we feel like doing it.

@uranusjr uranusjr removed the stale Stale PRs per the .github/workflows/stale.yml policy file label Oct 20, 2025
@Lee-W
Lee-W merged commit bb8a7b9 into apache:main Oct 20, 2025
108 checks passed
nailo2c pushed a commit to nailo2c/airflow that referenced this pull request Oct 20, 2025
TyrellHaywood pushed a commit to TyrellHaywood/airflow that referenced this pull request Oct 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API area:task-sdk type:improvement Changelog: Improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants