ci(ai-review): harden Argus per #4816 self-review#3488
Conversation
|
Reviewed during PR triage spin. The implementation is sound for what it claims (App-identity approval, branch filter, CI-green gate, idempotency, opt-out label, fast-fail on missing secrets). But two prerequisites need to land before this actually unblocks the routine bottleneck: 1. CODEOWNERS gate, not just review-count gateBranch ruleset on
Every routine PR touches Three options:
Recommend (a). Schema PRs and workflow PRs would still need your review, which is the right tradeoff. 2. Secret names don't matchRepo has
3. Minor implementation notes
Suggest landing CODEOWNERS narrowing first, then the App registration, then this workflow. |
|
Agreed on all three points. The workflow code stands as-is; the two prerequisites are external to this diff. On CODEOWNERS (item 1): Option (a) — removing On the App identity (item 2): The per-purpose-keys principle is correct; reusing Sequencing confirmed: CODEOWNERS narrowing → App registration + secrets → merge this workflow. PR will stay as a draft until both prerequisites are in place. Generated by Claude Code |
|
Blocked on infra setup — the workflow expects To land this, we need to:
Leaving open as a tracking PR until that infra is in place. Admin-merge has been working fine in the meantime. — Triage 2026-05-13 |
a11ca3a to
73a2e44
Compare
|
The automated review encountered an issue (possibly reached max turns, timed out, or failed to post the final This is an automated message from the Argus AI review workflow. |
Two instances of British spelling "behaviour" in docs/reference/release-notes.mdx. Repo convention is US spelling (158 "behavior" vs 2 "behaviour" before this fix). Smoke test for the new Argus AI review workflow (#3488). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* ci(auto-approve): post approval reviews on routine-authored PRs The Claude Code triage routine opens PRs as @bokelley (the only GitHub identity available in the Anthropic routine console), which means the project owner cannot approve their own PRs and branch protection forces admin-merge for everything the routine produces. Add a separate auto-approver workflow that uses the AAO Triage Bot GitHub App to post approving reviews on PRs that are: - Routine-authored (head branch starts with `claude/` or `auto/`) - Not draft - All CI checks green (success / neutral / skipped only) - No `do-not-auto-approve` label Triggers on pull_request_target (every PR event), check_suite.completed (when CI lands), and workflow_dispatch (manual fire). Idempotent — skips if the bot already approved the current head_sha. Required repo secrets: - TRIAGE_BOT_APP_ID - TRIAGE_BOT_APP_PRIVATE_KEY The App needs `pull_requests:write` + `contents:read` and must be installed on this repo. If the secrets are missing, the workflow fails fast at the token-generation step (no silent skip). Opt-out: add the `do-not-auto-approve` label to any PR that needs human eyes (e.g., feature work the routine couldn't fully verify). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci(ai-review): replace mechanical auto-approve with Argus LLM reviewer The original workflow only rubber-stamped CI-green PRs on claude/* and auto/* branches. It solved the routine-PR admin-merge bottleneck but ignored the bigger surface: bokelley/* PRs that ship spec changes still need real review. Replace with Argus, an LLM reviewer adapted from scope3data/agentic-api. Reads the diff, delegates to AdCP subagents in parallel, posts a substantive approve/comment/request-changes review on every non-dependabot PR. Uses the same RELEASE_APP_* secrets the changeset release bot already has - no separate App registration needed. AdCP-specific MUST-FIX gates: - Spec drift on static/schemas/source/** - Breaking wire change without a major changeset - Missing changeset on wire-touching PR - oneOf discriminator regression against the audit walker baseline Requires ANTHROPIC_API_KEY to be added to repo or org secrets before the workflow can complete - claude-code-action mints API calls under that key. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(release-notes): behaviour → behavior to match repo convention Two instances of British spelling "behaviour" in docs/reference/release-notes.mdx. Repo convention is US spelling (158 "behavior" vs 2 "behaviour" before this fix). Smoke test for the new Argus AI review workflow (#3488). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Argus reviewed PR #4816 and flagged five real issues in its own workflow. Addressing them: - Prompt loaded from base SHA, not the PR head working tree. Closes the same-repo prompt-injection vector even when the PR doesn't directly modify the prompt file. - Workflow-mod gate: PRs that touch .github/ai-review/** or .github/workflows/ai-review.yml alongside other files are not auto-reviewed. Bot posts a --comment requesting human review. - Verifier and skip-check filters pin to the Argus bot login (aao-release-bot) rather than matching any bot user. Stops changesets-release[bot] and other bots from false-positiving. - Bash(gh api:*) allowlist narrowed to read-only patterns: repos/*/pulls/*, repos/*/contents/*, repos/*/issues/*. - Third-party actions pinned to commit SHAs to close the moving-tag vector. Includes actions/checkout, actions/create-github-app-token, actions/github-script, anthropics/claude-code-action. - Heredoc sentinel randomized (openssl rand -hex 8) so prompt content can't break framing. - macOS date fallback removed (workflow is ubuntu-latest only). Branch-protection requirement documented in workflow comments: "Require approval of the most recent reviewable push" MUST be enabled for the trivial-skip path to be fail-closed against force-pushes that touch only trivial paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4553b58 to
c4183f3
Compare
There was a problem hiding this comment.
Argus is not auto-reviewing this PR because it modifies the review workflow itself (.github/workflows/ai-review.yml). Running with the PR head's modified prompt/workflow is a prompt-injection vector. A human reviewer should review and merge this PR; Argus will resume on subsequent PRs once these changes land on main.
|
@EmmaLouise2018 — could you take a pass when you have a window? This is the hardening delta on top of what landed via #4816 (the unhardened Argus is live on main right now). Five real findings + 2 nits that Argus surfaced reviewing its own bootstrap PR. Self-attestation note: the workflow-mod gate this PR adds correctly blocks me from auto-approving — Argus posted |
Summary
Argus reviewed its own bootstrap PR (#4816) and flagged 5 real findings + 2 nits in the workflow. The smoke-test PR squash-merged the unhardened Argus to
main(an artifact of how #4816 was branched off this PR). This PR now contains only the hardening delta on top of what landed via #4816.Argus's findings, all addressed
paths-ignore+ same-PR multi-file edits\$BASE_SHAviagit show, not the PR working tree. Plus a new workflow-mod gate that bails to--commentif the PR touches.github/ai-review/**or.github/workflows/ai-review.ymlalongside other files.Bash(gh api:*)wildcardrepos/*/pulls/*,repos/*/contents/*,repos/*/issues/*aao-release-botviaARGUS_BOT_LOGINenv var, in both verifier and skip-checkactions/checkout,create-github-app-token,github-script,claude-code-action) with version commentmain) in workflow commentsopenssl rand -hex 8Self-attestation
This PR hits the new workflow-mod gate it adds — Argus will post a
--comment, not--approve. That's by design. Needs human approval.Branch protection requirement
Verify "Require approval of the most recent reviewable push" is enabled on
main(Settings → Branches → main → Edit rule). Without it, the trivial-skip path can leave a stale bot approval after a force-push that touches only trivial paths.Refs
🤖 Generated with Claude Code