docs(release-notes): behaviour → behavior (Argus smoke test)#4816
Conversation
The Claude Code triage routine opens PRs as @bokelley (the only GitHub identity available in the Anthropic routine console), which means the project owner cannot approve their own PRs and branch protection forces admin-merge for everything the routine produces. Add a separate auto-approver workflow that uses the AAO Triage Bot GitHub App to post approving reviews on PRs that are: - Routine-authored (head branch starts with `claude/` or `auto/`) - Not draft - All CI checks green (success / neutral / skipped only) - No `do-not-auto-approve` label Triggers on pull_request_target (every PR event), check_suite.completed (when CI lands), and workflow_dispatch (manual fire). Idempotent — skips if the bot already approved the current head_sha. Required repo secrets: - TRIAGE_BOT_APP_ID - TRIAGE_BOT_APP_PRIVATE_KEY The App needs `pull_requests:write` + `contents:read` and must be installed on this repo. If the secrets are missing, the workflow fails fast at the token-generation step (no silent skip). Opt-out: add the `do-not-auto-approve` label to any PR that needs human eyes (e.g., feature work the routine couldn't fully verify). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The original workflow only rubber-stamped CI-green PRs on claude/* and auto/* branches. It solved the routine-PR admin-merge bottleneck but ignored the bigger surface: bokelley/* PRs that ship spec changes still need real review. Replace with Argus, an LLM reviewer adapted from scope3data/agentic-api. Reads the diff, delegates to AdCP subagents in parallel, posts a substantive approve/comment/request-changes review on every non-dependabot PR. Uses the same RELEASE_APP_* secrets the changeset release bot already has - no separate App registration needed. AdCP-specific MUST-FIX gates: - Spec drift on static/schemas/source/** - Breaking wire change without a major changeset - Missing changeset on wire-touching PR - oneOf discriminator regression against the audit walker baseline Requires ANTHROPIC_API_KEY to be added to repo or org secrets before the workflow can complete - claude-code-action mints API calls under that key. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two instances of British spelling "behaviour" in docs/reference/release-notes.mdx. Repo convention is US spelling (158 "behavior" vs 2 "behaviour" before this fix). Smoke test for the new Argus AI review workflow (#3488). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Smoke-test PR does what it says — two-char docs fix on a load-bearing US-spelling convention (158:2 before this PR), plus the Argus workflow bootstrap. Right shape: small payload, real wire (App-token mint → prompt build → claude-code-action → verifier).
Things I checked
docs/reference/release-notes.mdx— twobehaviour→behaviorreplacements, both inside narrative prose. No schema, no wire surface, no SDK impact..changeset/argus-smoke-test.md—patchbump onadcontextprotocolmatches the typo-only payload..changeset/auto-approve-routine-prs.md— empty---\n---frontmatter is valid for@changesets/cli(precedent incodeowners-maintainers-team.md,move-changeset-guidance-to-playbook.md). No bump emitted, release notes still surface. Intentional..github/workflows/ai-review.yml:34,42—pull_requesttrigger +actions/checkout@v4ofhead.sha. Fork PRs cannot mint the App token (secrets.RELEASE_APP_PRIVATE_KEYis unavailable onpull_requestevents from forks per GitHub secret-isolation policy). Fail-closed for forks..github/workflows/ai-review.yml:101-115—is_trivial()correctly excludes.github/workflows/*.yml,package.json,CODEOWNERS. Workflow/dependency changes trigger full re-review.- Largest-file rule satisfied — read
.github/workflows/ai-review.yml(219 net lines) and.github/ai-review/expert-adcp-reviewer.md(231 net lines) in full. - Schema-vs-docs audit — N/A. No
static/schemas/source/**touched. - Test-plan honesty — three unchecked items ("Argus posts `--approve`", "skips subagent delegation", "uses bokelley voice") are self-validating by this review. The author cannot manually verify them prior to merging the workflow that produces them; that's the smoke-test design. Not a block. Side note: subagent delegation was NOT skipped — the PR adds
.github/workflows/ai-review.yml(non-docs source), so per the prompt's own rulescode-reviewer+security-reviewerran. Expected outcome line in the PR body is wrong on that point.
Follow-ups (non-blocking — file as issues)
-
paths-ignoreis a no-op when a PR also touches a non-ignored path..github/workflows/ai-review.yml:28-30— GitHub paths filters are any-match-outside-the-list. A PR that edits.github/ai-review/expert-adcp-reviewer.mdAND any other file still runs the workflow, whichcats the modified prompt from the PR's working tree at L143. Threat: a same-repo contributor with write access injects prompt instructions that drive Argus to--approve, satisfying branch protection's "1 review required" check. (security-reviewer: High.) Override reasoning: fork vector is closed by secret-isolation; residual same-repo write-access attacker is within the trust boundary that already grants branch-push. Mitigation when convenient: read the prompt viagh api .../contents/...?ref=\$BASE_SHA, or move topull_request_targetwithhead.repo.full_name == github.repositorygating. -
Bash(gh api:*)allowlist is broad..github/workflows/ai-review.yml:170— the wildcard lets Argus call any REST endpoint within the App token's scope. Narrow to enumerated read-only patterns. Audit the App's installation scope; confirm it's installed only onadcontextprotocol/adcpand that its permissions don't includeadministrationor write surfaces beyondpull-requests:write. (Bothcode-reviewerandsecurity-reviewerflagged.) -
review_postedverifier false-positives on other bot reviews..github/workflows/ai-review.yml:78,185— both jq filters selectuser.type == \"Bot\"with no login pin. A--commentfrom changesets-release[bot] or any other bot in the last 10 minutes is treated as Argus's review. Add.user.login == \"<argus-app-bot-slug>\"to both filters. -
Pin third-party actions to commit SHAs.
actions/checkout@v4,actions/create-github-app-token@v1,anthropics/claude-code-action@v1,actions/github-script@v7— all moving tags. A force-push to any of these tags ships in the next workflow run with the App token andANTHROPIC_API_KEYin scope. -
Trivial-skip path carries stale approval across force-pushes.
.github/workflows/ai-review.yml:65-131— if a PR receives a bot approval, then force-pushes to a SHA whose diff against the prior-approved SHA only touches*.md/*.mdx/mintlify-docs/*, Argus skips re-review and the prior approval still counts. Real risk only when the bot's approval is the load-bearing one — which is the design. Either drop the shortcut when the prior review was bot-authored, or invalidate the bot approval on force-push.
Minor nits (non-blocking)
-
macOS
datefallback is unreachable..github/workflows/ai-review.yml:195— the|| date -u -j -f ...BSD form never fires underruns-on: ubuntu-latest, and underset -euo pipefailboth branches failing aborts the verify step. Harden with|| echo 0. -
\$GITHUB_OUTPUTheredoc sentinel is static..github/workflows/ai-review.yml:145,155—ARGUS_EOFis a constant. Repo-controlled prompt makes this informational today; becomes load-bearing if follow-up 1 isn't addressed.
LGTM. Follow-ups noted below.
Argus reviewed PR #4816 and flagged five real issues in its own workflow. Addressing them: - Prompt loaded from base SHA, not the PR head working tree. Closes the same-repo prompt-injection vector even when the PR doesn't directly modify the prompt file. - Workflow-mod gate: PRs that touch .github/ai-review/** or .github/workflows/ai-review.yml alongside other files are not auto-reviewed. Bot posts a --comment requesting human review. - Verifier and skip-check filters pin to the Argus bot login (aao-release-bot) rather than matching any bot user. Stops changesets-release[bot] and other bots from false-positiving. - Bash(gh api:*) allowlist narrowed to read-only patterns: repos/*/pulls/*, repos/*/contents/*, repos/*/issues/*. - Third-party actions pinned to commit SHAs to close the moving-tag vector. Includes actions/checkout, actions/create-github-app-token, actions/github-script, anthropics/claude-code-action. - Heredoc sentinel randomized (openssl rand -hex 8) so prompt content can't break framing. - macOS date fallback removed (workflow is ubuntu-latest only). Branch-protection requirement documented in workflow comments: "Require approval of the most recent reviewable push" MUST be enabled for the trivial-skip path to be fail-closed against force-pushes that touch only trivial paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Argus reviewed PR #4816 and flagged five real issues in its own workflow. Addressing them: - Prompt loaded from base SHA, not the PR head working tree. Closes the same-repo prompt-injection vector even when the PR doesn't directly modify the prompt file. - Workflow-mod gate: PRs that touch .github/ai-review/** or .github/workflows/ai-review.yml alongside other files are not auto-reviewed. Bot posts a --comment requesting human review. - Verifier and skip-check filters pin to the Argus bot login (aao-release-bot) rather than matching any bot user. Stops changesets-release[bot] and other bots from false-positiving. - Bash(gh api:*) allowlist narrowed to read-only patterns: repos/*/pulls/*, repos/*/contents/*, repos/*/issues/*. - Third-party actions pinned to commit SHAs to close the moving-tag vector. Includes actions/checkout, actions/create-github-app-token, actions/github-script, anthropics/claude-code-action. - Heredoc sentinel randomized (openssl rand -hex 8) so prompt content can't break framing. - macOS date fallback removed (workflow is ubuntu-latest only). Branch-protection requirement documented in workflow comments: "Require approval of the most recent reviewable push" MUST be enabled for the trivial-skip path to be fail-closed against force-pushes that touch only trivial paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
) Argus reviewed PR #4816 and flagged five real issues in its own workflow. Addressing them: - Prompt loaded from base SHA, not the PR head working tree. Closes the same-repo prompt-injection vector even when the PR doesn't directly modify the prompt file. - Workflow-mod gate: PRs that touch .github/ai-review/** or .github/workflows/ai-review.yml alongside other files are not auto-reviewed. Bot posts a --comment requesting human review. - Verifier and skip-check filters pin to the Argus bot login (aao-release-bot) rather than matching any bot user. Stops changesets-release[bot] and other bots from false-positiving. - Bash(gh api:*) allowlist narrowed to read-only patterns: repos/*/pulls/*, repos/*/contents/*, repos/*/issues/*. - Third-party actions pinned to commit SHAs to close the moving-tag vector. Includes actions/checkout, actions/create-github-app-token, actions/github-script, anthropics/claude-code-action. - Heredoc sentinel randomized (openssl rand -hex 8) so prompt content can't break framing. - macOS date fallback removed (workflow is ubuntu-latest only). Branch-protection requirement documented in workflow comments: "Require approval of the most recent reviewable push" MUST be enabled for the trivial-skip path to be fail-closed against force-pushes that touch only trivial paths. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summary
Two instances of British spelling "behaviour" in
docs/reference/release-notes.mdx. Repo convention is US spelling (158 "behavior" vs 2 "behaviour" before this fix).Why this PR exists
This PR is the smoke test for #3488 — the new Argus AI review workflow. The PR is intentionally tiny and benign so we can verify the end-to-end pipeline (App-token mint → prompt build → Argus LLM run → `gh pr review` post → verifier confirm) on a docs-only change before exercising any MUST-FIX gates.
Expected outcome: Argus reviews this PR, classifies as docs-only (no `code-reviewer` delegation needed), and posts `--approve`.
Test plan