ci: regression guard for 5 known production bugs#1312
Conversation
Adds a portable, repo-only CI guard (regression-guard.sh + workflow) that greps for the fixed code paths of 5 real production regressions and fails the build if any disappears. No SSH, no secrets. Passes green on main today. See REGRESSION-GUARD.md for the bug behind each check.
📝 WalkthroughWalkthroughThis PR introduces a regression guard system to prevent re-introduction of five documented production bugs affecting the Ultimate Multisite plugin. The system includes comprehensive documentation of bug root causes, a portable shell script that auto-detects plugin variant and runs grep-based checks on fixed code paths, and a GitHub Actions workflow that executes the guard on push to main and pull requests. ChangesRegression Guard System
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🔨 Build Complete - Ready for Testing!📦 Download Build Artifact (Recommended)Download the zip build, upload to WordPress and test:
🌐 Test in WordPress Playground (Very Experimental)Click the link below to instantly test this PR in your browser - no installation needed! Login credentials: |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/regression-guard.yml:
- Line 22: Update the checkout step in the regression-guard workflow to pin the
actions/checkout action to an immutable commit SHA instead of the mutable tag
and add persist-credentials: false to the step configuration; specifically
modify the step that currently uses "actions/checkout@v4" so it references a
specific commit SHA and includes persist-credentials: false to ensure the job
token is not written to the local git config for subsequent steps.
In `@REGRESSION-GUARD.md`:
- Around line 84-88: The documentation text references a different guide
filename than the actual file REGRESSION-GUARD.md; update all usage text and any
script or workflow output that mentions the guide to instead reference
REGRESSION-GUARD.md so operators see the correct file. Search for occurrences in
the regression-guard.sh usage/help messages and
.github/workflows/regression-guard.yml comments or step descriptions, and
replace the incorrect filename with REGRESSION-GUARD.md while keeping the rest
of the message intact.
In `@regression-guard.sh`:
- Line 20: The script references a non-existent help file name
"PARA-DAVID-regression-guard.md"; update all occurrences in regression-guard.sh
(e.g., the comment at the top and the occurrence around line 138) to reference
the actual file name "REGRESSION-GUARD.md" so CI failure guidance points to the
real remediation doc; search for the string "PARA-DAVID-regression-guard.md" in
regression-guard.sh and replace it with "REGRESSION-GUARD.md" (preserve
surrounding messaging and formatting).
- Around line 115-116: The current guard uses an OR which allows a false pass;
change the conditional so both anchors are required: call has with "$MIN"
"e&&e.token" AND has with the exact second minified anchor string (currently
"Captcha verification is taking longer than expected") before calling ok, i.e.
replace the `if has "$MIN" "e&&e.token" || has "$MIN" "Captcha verification is
taking longer than expected"; then` check with a combined (logical AND) check so
both has checks must succeed for ok "minified resolver carries the guard /
resolver string" to run.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: b5a6eadf-153a-485d-9531-bbfdf559c7d3
📒 Files selected for processing (3)
.github/workflows/regression-guard.ymlREGRESSION-GUARD.mdregression-guard.sh
| guard: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Show the workflow file with line numbers and surrounding context
file=".github/workflows/regression-guard.yml"
echo "=== $file (with line numbers) ==="
nl -ba "$file" | sed -n '1,120p'
# Pull out any checkout usage and relevant settings
echo
echo "=== checkout occurrences ==="
rg -n "uses:\s*actions/checkout@" .github/workflows || true
echo
echo "=== persist-credentials occurrences ==="
rg -n "persist-credentials" .github/workflows || true
echo
echo "=== job/workflow permissions occurrences ==="
rg -n "permissions:" .github/workflows || trueRepository: Ultimate-Multisite/ultimate-multisite
Length of output: 209
🏁 Script executed:
#!/bin/bash
set -euo pipefail
file=".github/workflows/regression-guard.yml"
echo "=== $file (first 160 lines with numbers via cat -n) ==="
cat -n "$file" | sed -n '1,160p'
echo
echo "=== checkout occurrences ==="
rg -n "uses:\s*actions/checkout@" .github/workflows || true
echo
echo "=== persist-credentials occurrences ==="
rg -n "persist-credentials" .github/workflows || true
echo
echo "=== permissions occurrences ==="
rg -n "permissions:" .github/workflows || trueRepository: Ultimate-Multisite/ultimate-multisite
Length of output: 3143
🌐 Web query:
actions/checkout persist-credentials default value and what it does (writes credentials to .git/config)
💡 Result:
actions/checkout persist-credentials default value: true (the input’s default is “true”; opting out requires persist-credentials: false). [1][2] What it does (writes credentials to.git/config): when enabled, the auth token (HTTPS) or SSH key credentials used by the action are persisted by configuring them in the local git config so later git commands in the job can run authenticated; specifically, “The auth token is persisted in the local git config.” [1][2] GitHub notes this token/credential setup is removed during post-job cleanup, and you can disable it with persist-credentials: false. [1][2] Therefore, yes: with the default, actions/checkout persists the credentials by writing them into the repository’s local git configuration (e.g., .git/config), and later steps can use them until cleanup. [1][2]
Citations:
- 1: https://github.com/actions/checkout/?tab=readme-ov-file
- 2: https://github.com/actions/checkout
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# find the script path referenced by workflow
rg -n "regression-guard\.sh" -S .github/workflows/regression-guard.yml || true
# locate regression-guard.sh in repo root (likely) and search for git usage
ls -la regression-guard.sh .github/workflows/regression-guard.yml 2>/dev/null || true
if [ -f "regression-guard.sh" ]; then
echo "=== git usage in regression-guard.sh ==="
rg -n "\bgit\b" regression-guard.sh || true
echo
echo "=== first 200 lines of regression-guard.sh (for context) ==="
sed -n '1,200p' regression-guard.sh | cat -n
fiRepository: Ultimate-Multisite/ultimate-multisite
Length of output: 7623
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# locate the script
ls -la regression-guard.sh .github/workflows/regression-guard.yml 2>/dev/null || true
if [ -f "regression-guard.sh" ]; then
echo "=== git usage in regression-guard.sh ==="
rg -n "\bgit\b" regression-guard.sh || true
echo
echo "=== first 220 lines of regression-guard.sh ==="
sed -n '1,220p' regression-guard.sh | cat -n
else
echo "regression-guard.sh not found at repo root"
fiRepository: Ultimate-Multisite/ultimate-multisite
Length of output: 7487
🏁 Script executed:
#!/bin/bash
set -euo pipefail
ls -la regression-guard.sh .github/workflows/regression-guard.yml 2>/dev/null || true
if [ -f "regression-guard.sh" ]; then
echo "=== git usage in regression-guard.sh ==="
rg -n "\bgit\b" regression-guard.sh || true
echo
echo "=== first 220 lines of regression-guard.sh ==="
sed -n '1,220p' regression-guard.sh | cat -n
else
echo "regression-guard.sh not found at repo root"
fiRepository: Ultimate-Multisite/ultimate-multisite
Length of output: 7487
🏁 Script executed:
#!/bin/bash
set -euo pipefail
ls -la regression-guard.sh .github/workflows/regression-guard.yml 2>/dev/null || true
if [ -f "regression-guard.sh" ]; then
echo "=== git usage in regression-guard.sh ==="
rg -n "\bgit\b" regression-guard.sh || true
echo
echo "=== first 220 lines of regression-guard.sh ==="
sed -n '1,220p' regression-guard.sh | cat -n
else
echo "regression-guard.sh not found at repo root"
fiRepository: Ultimate-Multisite/ultimate-multisite
Length of output: 7487
Harden regression-guard checkout step (pin SHA + disable credential persistence).
regression-guard.yml uses actions/checkout@v4 (mutable tag, not pinned) and does not set persist-credentials: false (default is true, which persists the auth token in the job’s local git config). Even though regression-guard.sh only greps the repo (no git commands), this step still needlessly leaves credentials available to subsequent steps.
Suggested hardening
- - uses: actions/checkout@v4
+ - uses: actions/checkout@<full_commit_sha_for_v4>
+ with:
+ persist-credentials: false🧰 Tools
🪛 zizmor (1.25.2)
[warning] 22-22: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
[error] 22-22: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/regression-guard.yml at line 22, Update the checkout step
in the regression-guard workflow to pin the actions/checkout action to an
immutable commit SHA instead of the mutable tag and add persist-credentials:
false to the step configuration; specifically modify the step that currently
uses "actions/checkout@v4" so it references a specific commit SHA and includes
persist-credentials: false to ensure the job token is not written to the local
git config for subsequent steps.
| - **`regression-guard.sh`** — a portable, repo-only script. No SSH, no live site, no | ||
| secrets. It greps the source for the fixed code paths above and exits non-zero if any | ||
| is gone. Auto-detects core/woo/captcha, or pass it explicitly: | ||
| `bash regression-guard.sh [core|woo|captcha]`. | ||
| - **`.github/workflows/regression-guard.yml`** — runs the script on every push/PR. |
There was a problem hiding this comment.
Reference the actual guide filename in usage text.
The guard docs/file are named REGRESSION-GUARD.md, but surrounding instructions and script output reference a different filename elsewhere. Aligning wording here avoids operator confusion when troubleshooting CI failures.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@REGRESSION-GUARD.md` around lines 84 - 88, The documentation text references
a different guide filename than the actual file REGRESSION-GUARD.md; update all
usage text and any script or workflow output that mentions the guide to instead
reference REGRESSION-GUARD.md so operators see the correct file. Search for
occurrences in the regression-guard.sh usage/help messages and
.github/workflows/regression-guard.yml comments or step descriptions, and
replace the incorrect filename with REGRESSION-GUARD.md while keeping the rest
of the message intact.
| # | ||
| # EXIT CODES: 0 = all guarded paths present 1 = a regression was detected | ||
| # | ||
| # Each check maps to a documented bug (see PARA-DAVID-regression-guard.md). |
There was a problem hiding this comment.
Failure guidance points to a non-existent markdown file.
The script points users to PARA-DAVID-regression-guard.md, but this PR adds REGRESSION-GUARD.md. CI output should reference the real file to keep remediation fast.
Suggested fix
-# Each check maps to a documented bug (see PARA-DAVID-regression-guard.md).
+# Each check maps to a documented bug (see REGRESSION-GUARD.md).
...
- echo " See PARA-DAVID-regression-guard.md for the bug behind each failing check."
+ echo " See REGRESSION-GUARD.md for the bug behind each failing check."Also applies to: 138-138
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@regression-guard.sh` at line 20, The script references a non-existent help
file name "PARA-DAVID-regression-guard.md"; update all occurrences in
regression-guard.sh (e.g., the comment at the top and the occurrence around line
138) to reference the actual file name "REGRESSION-GUARD.md" so CI failure
guidance points to the real remediation doc; search for the string
"PARA-DAVID-regression-guard.md" in regression-guard.sh and replace it with
"REGRESSION-GUARD.md" (preserve surrounding messaging and formatting).
| if has "$MIN" "e&&e.token" || has "$MIN" "Captcha verification is taking longer than expected"; then | ||
| ok "minified resolver carries the guard / resolver string" |
There was a problem hiding this comment.
Minified captcha check can falsely pass without the actual fix.
Using e&&e.token or the generic timeout string allows PASS even if the null-widget/resolve logic regressed. For a regression guard, this weakens correctness and can silently miss the bug’s return.
Suggested tightening
- if has "$MIN" "e&&e.token" || has "$MIN" "Captcha verification is taking longer than expected"; then
+ if has "$MIN" "e&&e.token" && has "$MIN" "!e&&n()"; then
ok "minified resolver carries the guard / resolver string"(Use the exact second anchor that matches your current minified output, but make both required.)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@regression-guard.sh` around lines 115 - 116, The current guard uses an OR
which allows a false pass; change the conditional so both anchors are required:
call has with "$MIN" "e&&e.token" AND has with the exact second minified anchor
string (currently "Captcha verification is taking longer than expected") before
calling ok, i.e. replace the `if has "$MIN" "e&&e.token" || has "$MIN" "Captcha
verification is taking longer than expected"; then` check with a combined
(logical AND) check so both has checks must succeed for ok "minified resolver
carries the guard / resolver string" to run.
🔨 Build Complete - Ready for Testing!📦 Download Build Artifact (Recommended)Download the zip build, upload to WordPress and test:
🌐 Test in WordPress Playground (Very Experimental)Click the link below to instantly test this PR in your browser - no installation needed! Login credentials: |
Replaces the grep-based guard (PR #1312) with real PHPUnit tests that execute the methods and assert the expected result, per maintainer feedback. - Regression_Pending_Site_And_Subdomain_Test: bug #4 (pending site stuck, Membership_Manager::check_pending_site_created stale-flag reset wiring) + bug #5 (wu_add_subdomain enqueue guard under wildcard DNS, case Eva). - Cart_Should_Collect_Payment_Test: a trial with allow_trial_without_payment_method off must still collect payment (route to woocommerce gateway, not free). - Checkout_Step_Fields_Test: step_fields partitions fields per step so the multi-step registration does not demand all fields on Step 1. Co-authored-by: David Stone <david@nnucomputerwhiz.com>
2 participants
Hi David — this adds a tiny CI regression guard so the fixes for the 5 production bugs we hit on kursopro.com can't silently regress as features are added.
What's in it
regression-guard.sh— portable, repo-only (no SSH, no live site, no secrets). Greps the source for the fixed code paths and exits non-zero if any is gone. Auto-detects core/woo/captcha..github/workflows/regression-guard.yml— runs it on every push/PR.REGRESSION-GUARD.md— the 5 bugs, each with a real customer case + the PR that fixed it.All 5 fixes are already in your
main— the guard passes green today. Its only job is to keep it that way. Anchors are on stable signatures (function names,has_action('wu_add_subdomain'),has_status(['active','trialing']), the null-widget guard), not on comments/version strings, so refactors won't trip it.This repo's checks (core): BUG 4
is_publishing_stale()+ BUG 5wu_add_subdomainguard — both PASS.Feel free to adjust the anchors to your liking; the goal is just a cheap safety net. Thanks for the upstream fixes 🙏
Summary by CodeRabbit
Documentation
Chores