Skip to content

Add asset partition sensor#67941

Open
Leondon9 wants to merge 2 commits into
apache:mainfrom
Leondon9:codex/asset-partition-sensor
Open

Add asset partition sensor#67941
Leondon9 wants to merge 2 commits into
apache:mainfrom
Leondon9:codex/asset-partition-sensor

Conversation

@Leondon9

@Leondon9 Leondon9 commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Add a small server-side partition lookup for asset events and a standard provider sensor that can wait for a specific asset partition from a time-scheduled DAG.

What changed:

  • Added partition_key exact filtering to Execution API asset event lookups by asset and by asset alias.
  • Threaded partition_key through the Task SDK client, comms models, task supervisor, triggerer supervisor, and inlet event accessor.
  • Added AssetPartitionSensor and AssetPartitionTrigger in the standard provider.
  • Added focused tests for the API filter, SDK forwarding path, sensor, and trigger behavior.

This is intended as a narrow draft for the idea in #67375. It deliberately avoids broader pattern/regex filters, Core API expansion, DB migrations/indexes, UI changes, and generated OpenAPI churn. It overlaps with part of #64610 only in the exact partition lookup surface; if #64610 lands first, this branch should be rebased and the lookup commit can likely be dropped or reduced.

Validation:

  • prek run ruff-format --files ...
  • prek run ruff --files ...

Pytest was attempted with Breeze, but the local test bootstrap failed before collecting/running the changed tests due provider discovery/import setup issues (airflow.providers.akeyless) and task_sdk import setup in the container.

related: #67375


Was generative AI tooling used to co-author this PR?
  • Yes — Codex (GPT-5)

Generated-by: Codex (GPT-5) following https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions


Important

🛠️ Maintainer triage note for @Leondon9 · by @potiuk · 2026-07-18 16:36 UTC

I've removed the ready for maintainer review label — the next step here is yours:

  • Merge conflicts with main. See docs.

It'll return to the maintainer queue automatically once resolved — no need to re-add the label by hand. See the Pull Request quality criteria for details.

The ball is in your court — you've been assigned. Rebase onto main, push, then mark it Ready for review.

Automated triage — may be imperfect; a maintainer takes the next look.

@Leondon9
Leondon9 force-pushed the codex/asset-partition-sensor branch from e0b2c41 to 5b85614 Compare June 3, 2026 12:33
@Leondon9
Leondon9 marked this pull request as ready for review June 12, 2026 23:30
@potiuk
potiuk marked this pull request as draft June 18, 2026 21:47
@Leondon9
Leondon9 marked this pull request as ready for review June 21, 2026 16:53
@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label Jun 22, 2026
@potiuk potiuk removed the ready for maintainer review Set after triaging when all criteria pass. label Jul 18, 2026
@potiuk
potiuk marked this pull request as draft July 20, 2026 11:54
@potiuk

potiuk commented Jul 20, 2026

Copy link
Copy Markdown
Member

@Leondon9 This PR has been converted to draft because it does not yet meet our Pull Request quality criteria.

Issues found:

  • Merge conflicts: This PR has merge conflicts with main and cannot be merged as-is. Rebase locally and resolve them: git fetch upstream main && git rebase upstream/main, resolve the conflicts, then force-push.

Note: Your branch is 648 commits behind main. Some check failures may be caused by changes in the base branch rather than by your PR. Please rebase your branch and push again to get up-to-date CI results.

What to do next:

  • Fix each issue listed above.
  • Make sure static checks pass locally (prek run --from-ref main --stage pre-commit).
  • Mark the PR as "Ready for review" when you're done.

Converting a PR to draft is not a rejection — it is an invitation to bring the PR up to the project's standards so that maintainer review time is spent productively. There is no rush — take your time and work at your own pace. We appreciate your contribution and are happy to wait for updates. If you have questions, feel free to ask on the Airflow Slack.


Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

@Leondon9
Leondon9 force-pushed the codex/asset-partition-sensor branch from f048261 to 8f86a15 Compare July 20, 2026 16:43
@Leondon9
Leondon9 marked this pull request as ready for review July 20, 2026 16:43
@Leondon9
Leondon9 force-pushed the codex/asset-partition-sensor branch from 8f86a15 to 03b434d Compare July 21, 2026 16:35
@hussein-awala

Copy link
Copy Markdown
Member

Hi @Leondon9,

After merging #64610 and #64611, my intention was to create a generic sensor for asset events rather than one based only on partitions. It would support filtering by partition, event metadata (extra), or event time.

I've opened #70225 and am testing it now. Since it is more generic and does not require changes to Airflow core, I think it has a better chance of being merged than this PR.

Could you check whether it addresses the needs you described in this discussion?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants