Skip to content

Introduce RUN-level OpenLineage events for dbt Cloud jobs#70169

Closed
br413 wants to merge 3 commits into
apache:mainfrom
br413:feat/dbt-cloud-openlineage-run-level
Closed

Introduce RUN-level OpenLineage events for dbt Cloud jobs#70169
br413 wants to merge 3 commits into
apache:mainfrom
br413:feat/dbt-cloud-openlineage-run-level

Conversation

@br413

@br413 br413 commented Jul 21, 2026

Copy link
Copy Markdown

Summary

  • Emit START and COMPLETE OpenLineage run events for dbt Cloud job executions
  • Reparent dbt model/test/source lineage events to the dbt Cloud job run instead of the Airflow task run
  • Preserve Airflow task run as the parent of the dbt Cloud job run event

Hierarchy

Airflow Task Run
└── dbt Cloud Job Run
    ├── dbt Model
    ├── dbt Test
    └── dbt Snapshot

Test plan

  • pytest providers/dbt/cloud/tests/unit/dbt/cloud/utils/test_openlineage.py
  • CI provider tests for dbt-cloud and openlineage pass

Closes #68661

Emit START and COMPLETE events for dbt Cloud job runs and reparent dbt
resource lineage events to the job run instead of the Airflow task run.

Closes apache#68661
@br413

br413 commented Jul 21, 2026

Copy link
Copy Markdown
Author

Converting approach based on @kacpermuda's feedback on #68661: the RUN-level layer belongs upstream in OpenLineage (OpenLineage/OpenLineage#4720) first. Closing this in favor of the short-term OperatorLineage facet approach in the follow-up PR, then upstream invocation events.

@br413

br413 commented Jul 21, 2026

Copy link
Copy Markdown
Author

Closing in favor of the short-term facet approach (#70185) and upstream OpenLineage#4720 work.

@br413 br413 closed this Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Introduce RUN-level OpenLineage Events for dbt Cloud Jobs in Airflow Integration

1 participant