postgres: Support purge_on_delete on postgres_projects#5414
Open
pietern wants to merge 1 commit into
Open
Conversation
Contributor
Approval status: pending
|
Collaborator
|
Commit: 36bd5cc |
Adds support for `purge_on_delete` on Lakebase `postgres_projects` so bundles can hard-delete a project on destroy. The flag is passed to the DeleteProject API call as `?purge=true`; when unset, the backend performs a soft delete that can be undone via `databricks postgres undelete-project` within the project's retention window. The field is input-only — it is not modeled by the backend resource for projects, and the GET API never returns it. We store it in state purely so DoDelete can apply it on destroy: by that point the configuration for the resource is gone, so state is the only place it can live. PrepareState preserves `input.ForceSendFields` so the structdiff comparison correctly distinguishes "explicit false" from the fictional remote zero — otherwise toggling `true -> false` would be classified as no change, state would stay `true`, and the next destroy would still emit `?purge=true`. DoUpdate strips `purge_on_delete` from the API field mask so a state-only flip doesn't fire an unnecessary remote write. Acceptance tests under `acceptance/bundle/resources/postgres_projects/`: - `purge_on_delete/`: deploys a `hard_delete` and a `soft_delete` project side by side and asserts the destroy emits `?purge=true` and a plain DELETE respectively, on both engines. - `purge_on_delete_transitions/`: direct-engine only. Walks `purge_on_delete` through unset -> true -> false -> unset and records the persisted value at each step; final destroy is a plain DELETE. Regression coverage for the FSF-preservation fix. Manually verified against dogfood with ephemeral projects on both engines (deploy -> flip -> destroy; GET on project/branches/endpoints in soft vs hard cases; native `postgres undelete-project` restoration semantics — captured in DECO-27233). Co-authored-by: Isaac
denik
reviewed
Jun 2, 2026
| This action will result in the deletion of the following Lakebase projects along with | ||
| all their branches, databases, and endpoints. All data stored in them will be permanently lost: | ||
| delete resources.postgres_projects.hard_delete | ||
| delete resources.postgres_projects.soft_delete |
Contributor
There was a problem hiding this comment.
unrelated, but why do we have duplicate message here?
| cleanup() { | ||
| # Belt-and-braces in case bundle destroy was skipped or partially failed. | ||
| # The soft-delete case leaves a record in the trash; --purge clears it. | ||
| $CLI postgres delete-project --purge "projects/test-pg-proj-hard-${UNIQUE_NAME}" 2>/dev/null || true |
Contributor
There was a problem hiding this comment.
what is 2>/dev/null for? is there noise? you can also sent to LOG.delete-project, it'll show up in go test -v output.
| title "bundle destroy" | ||
| $CLI bundle destroy --auto-approve > out.destroy.txt 2>&1 || true | ||
|
|
||
| trace print_requests.py --keep --sort --get '//postgres' '^//workspace-files/' '^//workspace/' '^//telemetry-ext' '^//operations/' > out.requests.destroy.json |
| trace $CLI postgres get-project "projects/test-pg-proj-hard-${UNIQUE_NAME}" | jq 'del(.create_time, .update_time)' | ||
| trace $CLI postgres get-project "projects/test-pg-proj-soft-${UNIQUE_NAME}" | jq 'del(.create_time, .update_time)' | ||
|
|
||
| trace print_requests.py --keep --sort --get '//postgres' '^//workspace-files/' '^//workspace/' '^//telemetry-ext' '^//operations/' > out.requests.deploy.json |
Contributor
There was a problem hiding this comment.
why --keep? it'll be double printed by next print_requests.py
| # resources.json. The field is omitted from JSON when unset (omitempty + | ||
| # ForceSendFields tracking), so absent => null, explicit true/false => bool. | ||
| get_purge() { | ||
| jq -r '.state["resources.postgres_projects.proj"].state.purge_on_delete' .databricks/bundle/default/resources.json |
Contributor
There was a problem hiding this comment.
nit: gron.py purge_on_delete < .databricks/bundle/default/resources.json could be more self-descriptive.
| @@ -0,0 +1,4 @@ | |||
| # Direct engine only: this test exercises the FSF preservation in | |||
| # PrepareState and direct's plan/diff classification when the user flips | |||
| # purge_on_delete. Terraform has its own provider-managed state lifecycle. | |||
Contributor
There was a problem hiding this comment.
but destroy requests should match between TF and direct, right? so could still make sense to run part of of these tests on TF to record destroy requests and compare.
denik
reviewed
Jun 2, 2026
| // not a spec field. Strip it from the mask so toggling it between deploys | ||
| // becomes a state-only refresh (the framework saves newState when this | ||
| // returns nil error). | ||
| fieldPaths = slices.DeleteFunc(fieldPaths, func(p string) bool { |
Contributor
There was a problem hiding this comment.
can we instead do a static list of the paths above? We should not use entry.Changes to populate update-mask.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds support for
purge_on_deleteon Lakebasepostgres_projectsso bundles can hard-delete a project on destroy. The flag is passed to the DeleteProject API call as?purge=true; when unset, the backend performs a soft delete that can be undone viadatabricks postgres undelete-projectwithin the project's retention window.The field is input-only — it is not modeled by the backend resource for projects, and the GET API never returns it. We store it in state purely so
DoDeletecan apply it on destroy: by that point the configuration for the resource is gone, so state is the only place it can live.PrepareStatepreservesinput.ForceSendFieldsso the structdiff comparison correctly distinguishes "explicit false" from the fictional remote zero — otherwise togglingtrue → falsewould be classified as no change, state would staytrue, and the next destroy would still emit?purge=true.DoUpdatestripspurge_on_deletefrom the API field mask so a state-only flip doesn't fire an unnecessary remote write.Acceptance tests under
acceptance/bundle/resources/postgres_projects/:purge_on_delete/: deploys ahard_deleteand asoft_deleteproject side by side and asserts the destroy emits?purge=trueand a plainDELETErespectively, on both engines.purge_on_delete_transitions/: direct-engine only. Walkspurge_on_deletethrough unset → true → false → unset and records the persisted value at each step; final destroy is a plainDELETE. Regression coverage for the FSF-preservation fix.Test plan
databricks postgres undelete-projectrestoration semantics confirmed end-to-end (project + implicit production branch + primary endpoint return; user-created sub-resources are cascade-deleted and not restored — captured in DECO-27233).This pull request and its description were written by Isaac.