Add post-deploy smoke tests to production and preview deploys - #177
Merged
Conversation
The #142 outage deployed green and then hung every request: CI cannot catch failures that only manifest against live infrastructure, so probe the deployed backend's /version after each modal deploy and fail the run on persistent non-200s (12 attempts x 10s, covering cold starts). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Beta preview has been cleaned up because this PR was closed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
After every
modal deploy(productiondeploy.yml, both preview deploy steps inpr-beta-deploy.yml), probe the deployed backend's/versionendpoint: 12 attempts × 10s with a 15s request timeout, failing the workflow run on persistent non-200s.Why
The #142 outage shipped through a fully green pipeline — tests correctly mock network seams, so a failure that only manifests against live infrastructure (the WIF token exchange hanging the request path) was structurally invisible to CI, and production served 500s until a human noticed. This probe turns that class of failure into a red workflow run within ~2 minutes of the deploy.
Independent of the GCL re-land: this is protection for every future deploy regardless of what it carries.
Verification
Both workflow files parse. The probe exercises the same URL contract the frontend uses (
policyengine--<app>-web.modal.run); the preview steps reuse the existingsteps.names.outputs.modal_urloutput.🤖 Generated with Claude Code