Skip to content

pipelines: update deploy.go and add fail-on-active-runs flag #3235

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
alyssa-db merged 4 commits into from
Jul 15, 2025
Merged

pipelines: update deploy.go and add fail-on-active-runs flag #3235

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e9d9eee
update deploy.go and add new flag
alyssa-db Jul 11, 2025
3fb0ffa
Merge branch 'main' into fail-on-active-runs-0
alyssa-db Jul 11, 2025
5704891
reverted deploy.go metrics
alyssa-db Jul 11, 2025
00f022f
acceptance test for pipelines
alyssa-db Jul 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions acceptance/pipelines/deploy/fail-on-active-runs/databricks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
bundle:
name: pipeline-fail-on-active-runs
resources:
pipelines:
my_pipeline:
name: pipeline-fail-on-active-runs
5 changes: 5 additions & 0 deletions acceptance/pipelines/deploy/fail-on-active-runs/out.test.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Local = true
Cloud = false

[EnvMatrix]
DATABRICKS_CLI_DEPLOYMENT = ["terraform"]
12 changes: 12 additions & 0 deletions acceptance/pipelines/deploy/fail-on-active-runs/output.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@

>>> [PIPELINES] deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/pipeline-fail-on-active-runs/default/files...
Deploying resources...
Updating deployment state...
Deployment complete!

>>> errcode [PIPELINES] deploy --fail-on-active-runs
Error: pipeline [UUID] is running


Exit code: 1
4 changes: 4 additions & 0 deletions acceptance/pipelines/deploy/fail-on-active-runs/script
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
trace $PIPELINES deploy

# We deploy the bundle again to check that the deploy is failing if the job is running
trace errcode $PIPELINES deploy --fail-on-active-runs
14 changes: 14 additions & 0 deletions acceptance/pipelines/deploy/fail-on-active-runs/test.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Deploy relies on terraform.CheckRunningResource()
EnvMatrix.DATABRICKS_CLI_DEPLOYMENT = ["terraform"]

# Cycling between states not implemented yet
# spec to avoid "pipeline spec is nil" error

[[Server]]
Pattern = "GET /api/2.0/pipelines/{pipeline_id}"
Response.Body = '''
{
"state": "RUNNING",
"spec": {}
}
'''
2 changes: 2 additions & 0 deletions cmd/bundle/deploy.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
// Copied to cmd/pipelines/deploy.go and adapted for pipelines use.
Copy link
Copy Markdown
Contributor

@andrewnester andrewnester Jul 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we need the comment here though

Copy link
Copy Markdown
Contributor Author

@alyssa-db alyssa-db Jul 11, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there another way you would suggest? @jefferycheng1 and I were discussing how to standardize these comments in the future, as we do want changes made to deploy.go to be considered for pipelines as well.

Copy link
Copy Markdown
Collaborator

@jefferycheng1 jefferycheng1 Jul 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andrewnester I thought to keep this in for posterity. tbh, I'm not sure how helpful this will be though, given how few people work on this, but it at least reminds PR authors to consider it. When we move pipelines out, we can remove this altogether. thoughts?

// Consider if changes made here should be made to the pipelines counterpart as well.
package bundle

import (
Expand Down
11 changes: 9 additions & 2 deletions cmd/pipelines/deploy.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// Copied from cmd/bundle/deploy.go and adapted for pipelines use.
// Consider if changes made here should be made to the bundle counterpart as well.
package pipelines

import (
Expand All @@ -22,9 +23,11 @@ func deployCommand() *cobra.Command {
}

var forceLock bool
var failOnActiveRuns bool
var autoApprove bool
var verbose bool
cmd.Flags().BoolVar(&forceLock, "force-lock", false, "Force acquisition of deployment lock.")
cmd.Flags().BoolVar(&failOnActiveRuns, "fail-on-active-runs", false, "Fail if there are running pipelines in the deployment.")
cmd.Flags().BoolVar(&autoApprove, "auto-approve", false, "Skip interactive approvals that might be required for deployment.")
cmd.Flags().BoolVar(&verbose, "verbose", false, "Enable verbose output.")
// Verbose flag currently only affects file sync output, it's used by the vscode extension
Expand All @@ -33,15 +36,19 @@ func deployCommand() *cobra.Command {
cmd.RunE = func(cmd *cobra.Command, args []string) error {
ctx := logdiag.InitContext(cmd.Context())
cmd.SetContext(ctx)
b := utils.ConfigureBundleWithVariables(cmd)

if logdiag.HasError(ctx) {
b := utils.ConfigureBundleWithVariables(cmd)
if b == nil || logdiag.HasError(ctx) {
return root.ErrAlreadyPrinted
}

bundle.ApplyFuncContext(ctx, b, func(context.Context, *bundle.Bundle) {
b.Config.Bundle.Deployment.Lock.Force = forceLock
b.AutoApprove = autoApprove

if cmd.Flag("fail-on-active-runs").Changed {
b.Config.Bundle.Deployment.FailOnActiveRuns = failOnActiveRuns
}
})

var outputHandler sync.OutputHandler
Expand Down
Loading