Add team_name tag to OpenLineage metrics - #69109
Merged
ferruzzi merged 5 commits intoJul 10, 2026
Merged
Conversation
SameerMesiah97
force-pushed
the
68996-OpenLineage-Team-Name-Metrics
branch
4 times, most recently
from
June 29, 2026 20:21
588db94 to
e3f59ba
Compare
SameerMesiah97
marked this pull request as ready for review
June 29, 2026 22:28
shivaam
reviewed
Jun 30, 2026
shivaam
reviewed
Jul 1, 2026
potiuk
added
the
ready for maintainer review
Contributor
Author
|
This is just a reminder for review. |
ferruzzi
reviewed
Jul 6, 2026
SameerMesiah97
force-pushed
the
68996-OpenLineage-Team-Name-Metrics
branch
from
July 6, 2026 21:44
e3f59ba to
240ab79
Compare
ferruzzi
approved these changes
Jul 6, 2026
SameerMesiah97
force-pushed
the
68996-OpenLineage-Team-Name-Metrics
branch
from
July 8, 2026 20:03
240ab79 to
3de74a8
Compare
kacpermuda
suggested changes
Jul 9, 2026
kacpermuda
left a comment
kacpermuda
left a comment
Collaborator
There was a problem hiding this comment.
One small but important change needed.
SameerMesiah97
force-pushed
the
68996-OpenLineage-Team-Name-Metrics
branch
from
July 9, 2026 20:54
3de74a8 to
90cfb6f
Compare
kacpermuda
reviewed
Jul 10, 2026
ferruzzi
approved these changes
Jul 10, 2026
ferruzzi
left a comment
ferruzzi
left a comment
Contributor
There was a problem hiding this comment.
With @kacpermuda approving the OL side, I'm happy to approve this. Thanks to both of you.
vincbeck
approved these changes
Jul 10, 2026
added 3 commits
July 10, 2026 21:40
when multi-team is enabled on Airflow 3.1+. The team name is propagated through the Airflow run facet and included in ol.emit.attempts, ol.emit.failed, ol.extract, and ol.event.size. Tests are updated accordingly.
…nd reuse the existing team_name value when emitting metrics.
…ssociated unit tests by removing unnecessary database markers and simplifying the test cases.
added 2 commits
July 10, 2026 21:40
…or the fallback path, and restore the required db_test markers for the DagRunInfo tests.
SameerMesiah97
force-pushed
the
68996-OpenLineage-Team-Name-Metrics
branch
from
July 10, 2026 20:41
90cfb6f to
c338d5b
Compare
joshuabvarghese
pushed a commit
to joshuabvarghese/airflow
that referenced
this pull request
Jul 16, 2026
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This change adds a
team_nametag to OpenLineage metrics to improve observability in multi-team deployments.The following metrics now include the
team_nametag when available:ol.emit.attemptsol.emit.failedol.extractol.event.sizeA new
DagRunInfo.team_name()helper has been added to resolve the team associated with aDagRunfrom its DAG bundle.For listener metrics (
ol.extractandol.event.size), theteam_nameis resolved directly from theDagRun. For adapter metrics (ol.emit.attemptsandol.emit.failed), the value is propagated through the Airflow run facet attached to the emitted OpenLineage event.Rationale
The OpenLineage integration emits metrics for lineage extraction and event publication, but these metrics could not previously be attributed to individual teams in multi-team deployments.
Adding the
team_nametag enables per-team dashboards, alerting, and troubleshooting while remaining backwards compatible for deployments that do not use multi-team scheduling.Tests
Added unit tests for the new
DagRunInfo.team_name()helper, including multi-team configuration handling.Updated the existing OpenLineage listener and adapter unit tests to verify metric emission both with and without the
team_nametag. Test cases covering the presence ofteam_nameare skipped when running against Airflow versions earlier than 3.3.0.Notes
A typographical error ('deg') in the docstring for
dag_version_infohas been corrected.Backwards Compatibility
This change is additive only. Existing metric names remain unchanged, and the
team_nametag is emitted only when running on Airflow 3.3+ with multi-team enabled. Existing dashboards and integrations that do not use the new tag continue to function unchanged.Related: #68996
Was generative AI tooling used to co-author this PR?
Generated-by: [GPT 5.5] following the guidelines