Skip to content

Integrate deployment metadata service for locking and state#4856

Draft
shreyas-goenka wants to merge 3 commits intomainfrom
shreyas-goenka/deployment-metadata-service
Draft

Integrate deployment metadata service for locking and state#4856
shreyas-goenka wants to merge 3 commits intomainfrom
shreyas-goenka/deployment-metadata-service

Conversation

@shreyas-goenka
Copy link
Copy Markdown
Contributor

Summary

  • Add client integration with the deployment metadata service API (/api/2.0/bundle/) for server-side deployment locking and resource state tracking
  • Replace file-based workspace locks with CreateVersion (lock acquire) / CompleteVersion (lock release) when the feature is enabled
  • Report resource operations to the metadata service after each deploy (using INITIAL_REGISTER for first-time tracking)
  • Background heartbeat goroutine keeps the lock alive during long-running deployments

Gated behind DATABRICKS_BUNDLE_DEPLOYMENT_SERVICE=true environment variable. Zero behavior change when the flag is off.

New files

  • bundle/env/deployment_metadata.go — env var definition
  • bundle/deploy/metadata/service/types.go — Go structs matching the proto API
  • bundle/deploy/metadata/service/client.go — HTTP client for all deployment metadata endpoints
  • bundle/deploy/metadata/service/heartbeat.go — background lock renewal
  • bundle/phases/deploy_metadata.go — new deploy flow with metadata service
  • bundle/phases/destroy_metadata.go — new destroy flow with metadata service

Modified files

  • bundle/deploy/state_update.go — export LoadState() function
  • bundle/phases/deploy.go — feature flag check
  • bundle/phases/destroy.go — feature flag check

Test plan

  • Unit tests for metadata service client
  • Acceptance tests with [[Server]] stubs for deploy/destroy flows
  • E2E test against dev workspace with service deployed
  • Verify zero behavior change with flag OFF (existing acceptance tests pass)

This pull request was AI-assisted by Isaac.

Add client integration with the deployment metadata service API for
server-side deployment locking and resource state tracking. Gated behind
DATABRICKS_BUNDLE_DEPLOYMENT_SERVICE=true environment variable.

Co-authored-by: Isaac
@eng-dev-ecosystem-bot
Copy link
Copy Markdown
Collaborator

eng-dev-ecosystem-bot commented Mar 26, 2026

Commit: 342fef8

Run: 23764962616

Env ✅​pass 🙈​skip Time
✅​ aws linux 73 14 0:46
✅​ aws windows 73 14 0:32
✅​ aws-ucws linux 76 11 0:41
✅​ aws-ucws windows 76 11 0:30
✅​ azure linux 72 14 0:49
✅​ azure windows 72 14 0:39
✅​ azure-ucws linux 75 11 0:43
✅​ azure-ucws windows 75 11 0:36
✅​ gcp linux 73 14 0:42
✅​ gcp windows 73 14 0:31

- Use background context with timeout for CompleteVersion in defer blocks,
  so the lock is released even if the parent context is cancelled (e.g. Ctrl+C)
- Add nil state.ID guard in destroy to avoid querying with zero UUID
- Fix misleading --force-lock error message to explain lock expiry behavior
- Fix import ordering

Co-authored-by: Isaac
Move the deployment metadata service client from bundle/deploy/metadata/service
to libs/tempdms with SDK-style method signatures (single request struct param).
When the protos land in the Go SDK, migration is just an import path change.

Unify deploy and destroy flows: instead of separate *WithMetadataService
functions that duplicated all mutator calls, the core logic stays in Deploy()
and Destroy() with conditional lock management based on the env var.

Co-authored-by: Isaac
@shreyas-goenka shreyas-goenka deployed to test-trigger-is March 30, 2026 20:00 — with GitHub Actions Active
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants