Skip to content

acc: Added test for new Lakeflow pipeline deployment#3578

Merged
andrewnester merged 2 commits intomainfrom
feat/deploy-lakeflow-pipelines
Sep 9, 2025
Merged

acc: Added test for new Lakeflow pipeline deployment#3578
andrewnester merged 2 commits intomainfrom
feat/deploy-lakeflow-pipelines

Conversation

@andrewnester
Copy link
Copy Markdown
Contributor

Changes

Added test for new Lakeflow pipeline deployment

Why

To confirm that new Lakeflow pipelines are deployed correctly

@@ -0,0 +1,5 @@
Local = true
Cloud = false
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious, what makes this pipeline "lakeflow pipeline"?

And should we test it against cloud?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using root_path and libraries glob https://docs.databricks.com/api/workspace/pipelines/create#root_path

Testing against cloud makes sense, will add it

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using root_path and libraries glob https://docs.databricks.com/api/workspace/pipelines/create#root_path

Thanks, I'd add this info as a comment to databricks.yml

@eng-dev-ecosystem-bot
Copy link
Copy Markdown
Collaborator

eng-dev-ecosystem-bot commented Sep 9, 2025

Run: 17583111990

Env ✅​pass 🔄​flaky 🙈​skip
✅​ aws linux 311 523
✅​ aws windows 312 522
✅​ aws-ucws linux 423 421
✅​ aws-ucws windows 424 420
✅​ azure linux 311 522
🔄​ azure windows 311 1 521
✅​ azure-ucws linux 423 420
✅​ azure-ucws windows 424 419
✅​ gcp linux 310 524
🔄​ gcp windows 308 3 523
Test Name azure windows gcp windows
TestAccept ✅​pass 🔄​flaky
TestAccept/bundle/deploy/files/no-snapshot-sync ✅​pass 🔄​flaky
TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_CLI_DEPLOYMENT=terraform ✅​pass 🔄​flaky
TestFetchRepositoryInfoAPI_FromRepo 🔄​flaky ✅​pass

@andrewnester andrewnester added this pull request to the merge queue Sep 9, 2025
github-merge-queue bot pushed a commit that referenced this pull request Sep 9, 2025
## Changes
Added test for new Lakeflow pipeline deployment

## Why
To confirm that new Lakeflow pipelines are deployed correctly

<!-- If your PR needs to be included in the release notes for next
release,
add a separate entry in NEXT_CHANGELOG.md as part of your PR. -->
Merged via the queue into main with commit 9429f2a Sep 9, 2025
13 checks passed
@andrewnester andrewnester deleted the feat/deploy-lakeflow-pipelines branch September 9, 2025 13:54
github-merge-queue bot pushed a commit that referenced this pull request Sep 17, 2025
## Changes
Added new Lakeflow Pipelines support for `bundle generate`

## Why
Now if the pipeline refers to `root_path` we download the whole content
of it and rewrite any related `libraries.glob` paths pointing to the
same remote paths to the local ones

## Tests
Added an acceptance test

Confirmed that they can be deployed here:
#3578

<!-- If your PR needs to be included in the release notes for next
release,
add a separate entry in NEXT_CHANGELOG.md as part of your PR. -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants