Skip to content

Scope XCom Execution API to teams in multi-team mode#68850

Merged
o-nikolas merged 1 commit into
apache:mainfrom
aws-mwaa:onikolas/pr1/xcom-execution-api-team-scoping
Jun 22, 2026
Merged

Scope XCom Execution API to teams in multi-team mode#68850
o-nikolas merged 1 commit into
apache:mainfrom
aws-mwaa:onikolas/pr1/xcom-execution-api-team-scoping

Conversation

@o-nikolas

Copy link
Copy Markdown
Contributor

XCom was the only shared resource without team isolation at the task api level. Any task could read, overwrite, or delete another team's XCom. Enforce team ownership so reads are allowed for the task's own team or global dags, and writes/deletes only for its own team, matching team-scoped variables/connections.
No cross-team sharing at this time. Gated on the multi_team setting (no-op when disabled).


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

XCom was the only shared resource without team isolation at the
task api level. Any task could read, overwrite, or delete another
team's XCom. Enforce team ownership so reads are allowed for the
task's own team or global dags, and writes/deletes only for its own
team, matching team-scoped variables/connections.
No cross-team sharing at this time. Gated on the multi_team setting
(no-op when disabled).
@boring-cyborg boring-cyborg Bot added area:API Airflow's REST/HTTP API area:task-sdk labels Jun 22, 2026

@vincbeck vincbeck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM. The only question I have is, is this a multi-team issue? By that I mean, should a Dag access xcoms from another Dag (regardless of teams)? Should not we ensure any task from a given Dag can only access xcoms from this exact same Dag?

@o-nikolas

Copy link
Copy Markdown
Contributor Author

Code LGTM. The only question I have is, is this a multi-team issue? By that I mean, should a Dag access xcoms from another Dag (regardless of teams)? Should not we ensure any task from a given Dag can only access xcoms from this exact same Dag?

I think cross dag xcom is a pattern that people make use of today. We could consider reversing that, but we'd have to reckon with the level of breaking change that would be and is a much larger discussion. I think the best short term path forward is to block the access cross teams for now to maintain the same behaviour that we have today.

@ramitkataria ramitkataria left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me and I also agree with this being the best short term solution

@seanghaeli seanghaeli left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noting that this will force a rebase in #66611 since they touch the same paths (or a rebase here if #66611 merges first).

Comment thread airflow-core/src/airflow/api_fastapi/execution_api/routes/xcoms.py
@o-nikolas o-nikolas merged commit 3b9d2a2 into apache:main Jun 22, 2026
77 checks passed
@o-nikolas o-nikolas deleted the onikolas/pr1/xcom-execution-api-team-scoping branch June 22, 2026 23:53
@o-nikolas o-nikolas added this to the Airflow 3.3.0 milestone Jun 22, 2026
cetingokhan pushed a commit to cetingokhan/airflow that referenced this pull request Jun 24, 2026
XCom was the only shared resource without team isolation at the
task api level. Any task could read, overwrite, or delete another
team's XCom. Enforce team ownership so reads are allowed for the
task's own team or global dags, and writes/deletes only for its own
team, matching team-scoped variables/connections.
No cross-team sharing at this time. Gated on the multi_team setting
(no-op when disabled).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API area:task-sdk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants