From e85bd505d333a62841cfead28ed4e1f22a809832 Mon Sep 17 00:00:00 2001 From: Brian O'Kelley Date: Mon, 1 Jun 2026 04:26:38 -0400 Subject: [PATCH 1/3] chore(triage): lean compose/defer over inventing precedence rules; converge to one comment MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two response-discipline fixes prompted by the triage routine's behavior on PR #5219 (it invented a "brand wins on conflict" SHOULD for what was really a compose/defer call, then posted an answer followed by a self-retraction addendum): - Step 5: add "Design-question discipline" — for "how do these inputs interact / should the spec define X" questions, default to compose / agent-territory / file-an-issue. Don't synthesize a normative precedence/conflict/MUST/SHOULD rule absent a concrete *interop* failure (incompatible wire output) — aesthetic divergence and unpredictable creative output are not interop failures. - Step 6: "Finalize before you comment — one comment per run." Never post an answer then a correction/addendum; a public answer-then-retraction trail is worse than a slightly-later single answer. - Comment engagement rule 6: if a re-run changes a posted conclusion, EDIT the original comment (gh api PATCH), don't stack a new one. - Two reinforcing entries in the quick-reference "Never" list. Co-Authored-By: Claude Opus 4.8 (1M context) --- .agents/routines/triage-prompt.md | 49 +++++++++++++++++++++++++++++-- 1 file changed, 46 insertions(+), 3 deletions(-) diff --git a/.agents/routines/triage-prompt.md b/.agents/routines/triage-prompt.md index 671537d808..e033b505b4 100644 --- a/.agents/routines/triage-prompt.md +++ b/.agents/routines/triage-prompt.md @@ -514,6 +514,27 @@ Not every dimension matters for every issue — skip ones that aren't material. But if a dimension *is* material (e.g., SSAI behavior on a VAST asset-model RFC) and no expert addressed it, that's a gap. +**Design-question discipline (spec / protocol — read before drafting +an answer).** When the issue asks *how two existing inputs interact*, +or whether the spec *"should define"* a behavior, the default lean is +**compose / defer, not invent.** Most such questions resolve to one of: +(a) "these are orthogonal inputs the agent composes; the buyer evaluates +the output" — generative/creative output is never fully predictable, +which is exactly what produce → review → refine is for; (b) +"agent-implementation territory — AdCP specifies the wire, not agent +internals"; (c) "real, but out of scope here — file a spec issue." Do +**NOT** synthesize a new normative **precedence / conflict-resolution / +MUST / SHOULD** rule unless an expert demonstrates a concrete *interop* +failure — two conformant agents producing incompatible **wire** output. +Aesthetic divergence, unpredictable output, and "the buyer might not +like it" are **not** interop failures. Inventing a precedence rule to +"close a gap" that's really a creative-/operator-judgment call +over-constrains implementations and ages badly. When unsure, Flag with +the compose/defer framing and let the WG decide — don't bake the rule +into the comment. (A second expert pass that *reverses* your answer is +the signal you reached for a rule too early; re-converge before you +comment — see Step 6.) + **For RFC / epic / cross-cutting issues:** consider spawning 2× per expert type in parallel. Variance in expert framing is a feature for high-scope issues — different instances surface different angles @@ -575,6 +596,18 @@ Post a comment when: MEMBER/COLLABORATOR/OWNER. They don't need a "your issue is deferred" note. Just apply `claude-triaged` + labels. +**Finalize before you comment — one comment per run.** Complete the +full synthesis (Step 5, *including* any expert re-runs) before posting. +If a further expert pass would change your conclusion, you have not +finished synthesizing — keep going; never post an answer you then walk +back. **Never post an answer in one comment and a correction/addendum +in another:** a public answer-then-retraction trail is worse than a +slightly later single answer, and it erodes trust in the routine's +output. If genuinely new information arrives *after* you've commented +(a later human reply on the thread), **edit** the prior comment rather +than stacking a second one — see "Comment engagement (existing +threads)". + Comment format: default cap **≤1500 chars total, prose ≤4 sentences**, **lifted when option examples are required** (see below — a few fenced code blocks beat a short prose description the reader can't act on). @@ -1191,9 +1224,15 @@ body is in `<<>>`. comment posted) that the routine engaged. A second comment confirming receipt dilutes the threads where the routine actually has something to say. -6. Never reply to your own previous comments (workflow filters - most cases, but the routine should also self-check via the - `Triaged by Claude Code` footer). Never reply to bot authors. +6. Never reply to your own previous comments, and **never post a + correction, addendum, or retraction as a *new* comment.** If a + re-run (rule 3) changes a conclusion you already posted, **edit the + original comment** (fetch its id via `gh api + repos///issues//comments`, then + `gh api -X PATCH repos///issues/comments/ -f body=…`) + so the thread carries one coherent answer, not an answer-then-walkback + trail. (Workflow filters most self-loops; also self-check via the + `Triaged by Claude Code` footer.) Never reply to bot authors. **PR conversations are out of scope here.** The workflow filters `issue_comment` events where `issue.pull_request != null`. PR @@ -1226,6 +1265,10 @@ couldn't fetch. after the label - Never describe security-sensitive vectors in a public comment - Never invent AdCP features or fields not in `static/schemas/source/` +- Never invent normative semantics (precedence / conflict-resolution / + MUST / SHOULD rules) for a question that's really compose/defer — see + "Design-question discipline" in Step 5 +- Never post a correction/addendum as a new comment — edit the original - Never create new labels or milestones ## Never (organizational rules — from playbook) From d018e3ae497a0124abd07a5ff404a985c7168310 Mon Sep 17 00:00:00 2001 From: Brian O'Kelley Date: Mon, 1 Jun 2026 04:36:08 -0400 Subject: [PATCH 2/3] =?UTF-8?q?chore(triage):=20fix=20the=20PR-comment=20s?= =?UTF-8?q?urface=20=E2=80=94=20prompt=20said=20"out=20of=20scope",=20work?= =?UTF-8?q?flow=20routes=20them=20in?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Tracing why the routine answered WG questions on PR #5219 (the invented precedence rule + answer-then-addendum) surfaced a prompt/workflow contradiction: - claude-issue-triage.yml fires on issue_comment for issues AND PRs and injects a `MODE: PR-feedback` block telling the routine to answer PR questions — but triage-prompt.md claimed "PR conversations are out of scope; the workflow filters them; that's auto-fix's job." The workflow does the opposite, so the routine ran PR-comment answers ungoverned by the prompt's discipline. Fixes (same surface as the prior commit on this branch): - triage-prompt.md: replace the stale "PR conversations are out of scope" paragraph with an accurate `MODE: PR-feedback` section that routes PR questions through Step 5 (Design-question discipline) + Step 6 (finalize / one comment) + rule 6 (edit, don't addend). - claude-issue-triage.yml: tighten the payload's `MODE: PR-feedback` line to defer to those prompt rules (compose/defer; one coherent reply; edit don't addend) instead of a bare "answer it as a reply comment". Co-Authored-By: Claude Opus 4.8 (1M context) --- .agents/routines/triage-prompt.md | 27 ++++++++++++++++++----- .github/workflows/claude-issue-triage.yml | 2 +- 2 files changed, 22 insertions(+), 7 deletions(-) diff --git a/.agents/routines/triage-prompt.md b/.agents/routines/triage-prompt.md index e033b505b4..b6dbe7efce 100644 --- a/.agents/routines/triage-prompt.md +++ b/.agents/routines/triage-prompt.md @@ -1234,12 +1234,27 @@ body is in `<<>>`. trail. (Workflow filters most self-loops; also self-check via the `Triaged by Claude Code` footer.) Never reply to bot authors. -**PR conversations are out of scope here.** The workflow filters -`issue_comment` events where `issue.pull_request != null`. PR -review feedback is the **auto-fix** feature's job, not the -triage routine's. If a comment route to triage looks like PR -feedback (filter slipped), no-op silently and surface the -filter gap in the run summary. +**PR comments ARE in scope — `MODE: PR-feedback`.** The bridge +workflow (`.github/workflows/claude-issue-triage.yml`) routes +`issue_comment` events on **both issues and PRs** to this routine; a +PR comment arrives with `is_pr: true` and a `MODE: PR-feedback` +line in the payload. (Code-fix pushing on CI failures is a separate +concern handled by the **auto-fix** feature; this routine's job on a +PR comment is the human reply.) In PR-feedback mode: + +- **Fix request** → apply it as a follow-up commit on the PR head + branch; never open a new PR. +- **Question / design challenge** → answer in **one** reply comment. + This is where the routine has misfired (PR #5219): it invented a + precedence rule for a compose/defer question and posted an answer + followed by a self-retraction. **All of Step 5's + "Design-question discipline" and Step 6's "Finalize before you + comment — one comment per run" apply here verbatim.** Converge + before you post; lean compose/defer over inventing + precedence/MUST/SHOULD rules; if you must revise a posted answer, + **edit it** (rule 6), never stack a correction. +- **Conversational, no action** → short acknowledgement or silence; + apply the "Anti-patterns — never post these" list above. ## Failure handling diff --git a/.github/workflows/claude-issue-triage.yml b/.github/workflows/claude-issue-triage.yml index df02fca41f..dfe3c1bb05 100644 --- a/.github/workflows/claude-issue-triage.yml +++ b/.github/workflows/claude-issue-triage.yml @@ -188,7 +188,7 @@ jobs: (if $is_pr == "true" then "is_pr: true\n" + "pr: " + $pr_block + "\n" + - "MODE: PR-feedback. Treat new comment as actionable feedback on the PR diff. If the comment requests a fix, apply it as a follow-up commit on the PR head branch — do not open a new PR. If the comment asks a question, answer it as a reply comment. If the comment is conversational with no action implied, post a short acknowledgement and stop.\n" + "MODE: PR-feedback. Treat new comment as actionable feedback on the PR diff. Follow the `MODE: PR-feedback` rules in .agents/routines/triage-prompt.md. If the comment requests a fix, apply it as a follow-up commit on the PR head branch — do not open a new PR. If the comment asks a question or challenges a design choice, answer it in ONE coherent reply comment: lean compose/defer over inventing precedence/MUST/SHOULD rules, finalize your synthesis before posting, and if you must revise a prior answer, EDIT it rather than posting a correction/addendum. If the comment is conversational with no action implied, post a short acknowledgement and stop.\n" else "is_pr: false\n" + "LABEL POLICY: When your triage status is `ready-for-human` (a real scope/policy/protocol decision is needed before code can land), apply the `needs-wg-review` label via `gh issue edit --add-label needs-wg-review` AND apply the relevant domain label so the working group queue can route. Domain label vocabulary (pick one or more that match the issue's subject area):\n" + From 0cc02a142b3bdb993b710130542ebaa71112f5db Mon Sep 17 00:00:00 2001 From: Brian O'Kelley Date: Mon, 1 Jun 2026 04:37:42 -0400 Subject: [PATCH 3/3] chore(triage): fix the second stale "filters out PR comments" claim (Run type section) The Run-type `comment.created` entry still said the workflow filters out PR comments, contradicting the corrected MODE: PR-feedback section. Align it: PR comments ARE routed here (is_pr + MODE: PR-feedback), handled per the PR-feedback rules, not as issue triage. Co-Authored-By: Claude Opus 4.8 (1M context) --- .agents/routines/triage-prompt.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/.agents/routines/triage-prompt.md b/.agents/routines/triage-prompt.md index b6dbe7efce..52e818c579 100644 --- a/.agents/routines/triage-prompt.md +++ b/.agents/routines/triage-prompt.md @@ -36,9 +36,12 @@ trigger fired: re-filed). The user message has `<<>>` only. Act on that one issue with full triage. - **`comment.created`:** a non-bot, non-`/triage`, non-self comment - landed on an open issue (the workflow filters out PR comments, - `/triage` slash-commands, and the routine's own previous comments - to prevent loops). The user message has *both* a + landed on an open issue **or PR** (the workflow filters out `/triage` + slash-commands and the routine's own previous comments to prevent + loops, but routes **both** issue and PR comments here). A PR comment + arrives with `is_pr: true` and a `MODE: PR-feedback` line — handle it + per the **`MODE: PR-feedback`** rules under Comment engagement, not as + issue triage. The user message has *both* a `<<>>` block (the new comment) AND a `<<>>` block (the original issue). Read the full thread on GitHub before deciding (`gh api repos/.../issues/N/comments`).