From 3e3978ab807cbe553628e89fd6b038b5a88bf2d1 Mon Sep 17 00:00:00 2001 From: William Thorsen Date: Fri, 26 Jun 2026 03:45:45 -0700 Subject: [PATCH] agents|feat: Add capture-feedback skill MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds a `/capture-feedback` skill that records feedback on agent behavior — a correction to a misapplied rule, or a new desired behavior — and applies the immediate fix when there is something concrete to change. It records the feedback for a later guidance-refinement pass rather than refining guidance on the spot, and applies obvious fixes without prompting — confirming only when a correction is ambiguous or risky. The `collaborate` workflow now routes user feedback to it. --- .../agents/content/collections/recommended.md | 1 + .../content/skills/capture-feedback/SKILL.md | 64 +++++++++++++++++++ .../content/skills/collaborate/SKILL.md | 2 +- .../capture-feedback-delivery.test.ts | 20 ++++++ 4 files changed, 86 insertions(+), 1 deletion(-) create mode 100644 packages/agents/content/skills/capture-feedback/SKILL.md create mode 100644 packages/agents/src/lib/__tests__/capture-feedback-delivery.test.ts diff --git a/packages/agents/content/collections/recommended.md b/packages/agents/content/collections/recommended.md index 4a02e0f7..eca133e9 100644 --- a/packages/agents/content/collections/recommended.md +++ b/packages/agents/content/collections/recommended.md @@ -3,6 +3,7 @@ name: recommended description: The batteries-included default set of artifacts, opt-in via codeassembly.yaml. dependencies: skills: + - capture-feedback - people-report subagents: - canary diff --git a/packages/agents/content/skills/capture-feedback/SKILL.md b/packages/agents/content/skills/capture-feedback/SKILL.md new file mode 100644 index 00000000..a29a20cf --- /dev/null +++ b/packages/agents/content/skills/capture-feedback/SKILL.md @@ -0,0 +1,64 @@ +--- +name: capture-feedback +description: Capture feedback on agent behavior as a guidance-refinement candidate — apply the immediate fix when concrete, then record the generalized lesson. Use when correcting a misapplied rule or specifying a new desired behavior. +user-invocable: true +deploy: declared +--- + +# Capture feedback + +Record feedback on the agent's behavior as evidence for a future guidance refinement, applying the immediate fix when there is something concrete to change. The user describes a correction or a desired behavior, the agent applies it when applicable, and the agent appends a generalized record that a later recall-driven pass mines to refine guidance. + +The actual guidance refinement is **deferred** — this skill captures the candidate, it does not edit the skill, subagent, rulebook, general guidance, or helper. Refinement happens later, in bulk, via `kb-retrieve-events`, so a single data point never over-fits a rule. + +**Announce at start:** "Using capture-feedback to apply this and record it for guidance refinement." + +## When to use + +Invoke `/capture-feedback` whenever the user gives feedback that should change how the agent behaves — either: + +- **Correcting a misapplied rule** — the agent broke guidance that already exists (for example, title-cased a heading when the sentence-case rule was in force). +- **Specifying a new desired behavior** — the user wants a behavior that no current guidance covers. + +Both are captured the same way; the difference is recorded in the tags. + +## Process + +### 1. Classify the feedback + +Decide one question: **did relevant guidance already exist?** + +- **Yes — misapplied existing guidance.** A rule was in force and the agent missed it. This is a mistake plus its correction. +- **No — no such guidance.** The user is establishing a net-new expectation. There is no mistake, only a refinement to propose. + +Assess this best-effort from the guidance you can see. When you genuinely cannot tell, record the uncertainty in the body rather than forcing the call. + +### 2. Apply the immediate fix + +If the feedback names a concrete artifact — something the agent produced, such as code, a doc, a PR body, or a ticket — apply the fix: + +- **Apply it directly, without asking for approval.** Invoking this skill and describing the fix is the authorization. +- **Confirm first only when the fix is genuinely ambiguous** (more than one reasonable reading of what to change) **or risky, large, or hard to reverse.** Then state the change you intend to make and wait for a go-ahead before applying. + +If the feedback is purely behavioral — a standing rule with nothing to fix right now — skip this step. + +### 3. Capture the record + +Invoke the `capture-event` skill to append an immutable record, composing its arguments and body as follows: + +- `--store codeassembly` — the project's agent-guidance KB. Pass a different `--store` only when the user directs the record elsewhere. +- `--tags feedback` — always. Add `,mistake` when existing guidance was misapplied (step 1, "Yes"). +- `--skill ` — when the refinement target is a skill. +- `--summary` — a one-line recall label, for example "Agent title-cased a heading; sentence case is the rule." +- **Body** — the generalized lesson, only to the extent needed to act on it later: + - The **error→correction pair** (misapplied-guidance mode) or the **desired behavior** (no-guidance mode), generalized — not the raw artifact or diff. + - A **best-effort candidate refinement target**: the guidance artifact and its type (skill, subagent, rulebook, general guidance, or helper). Use `--skill` for a skill target; name the artifact in the body for the other four. Write "candidate: undetermined" when you genuinely cannot place it. + - Any uncertainty about the classification from step 1. + +### 4. Report + +State what was fixed — or that the feedback was behavioral-only — and the captured record's id and path. + +## Completion + +The immediate fix is applied (when applicable) and an immutable `feedback`-tagged record is written, ready for a later refinement pass to mine via `kb-retrieve-events`. diff --git a/packages/agents/content/skills/collaborate/SKILL.md b/packages/agents/content/skills/collaborate/SKILL.md index 09c08837..bd1bfb73 100644 --- a/packages/agents/content/skills/collaborate/SKILL.md +++ b/packages/agents/content/skills/collaborate/SKILL.md @@ -69,4 +69,4 @@ When you deem appropriate, proactively dispatch subagents to perform tasks. Good ## Skill improvement -- When the user corrects a mistake caused by unclear skill definition, invoke the `capture-event` skill to record it: pass `--tags mistake` and `--skill `, and put what went wrong and what the skill should have said in the body. +- When the user corrects the agent, or specifies a new desired behavior, that feedback is evidence for refining a skill, subagent, rulebook, general guidance, or helper. Invoke the `capture-feedback` skill: it applies the immediate fix when there is something concrete and records a generalized `feedback` event — tagged `mistake` when existing guidance was missed — for a later refinement pass to mine. diff --git a/packages/agents/src/lib/__tests__/capture-feedback-delivery.test.ts b/packages/agents/src/lib/__tests__/capture-feedback-delivery.test.ts new file mode 100644 index 00000000..d1b3166a --- /dev/null +++ b/packages/agents/src/lib/__tests__/capture-feedback-delivery.test.ts @@ -0,0 +1,20 @@ +import { readFile } from 'node:fs/promises'; +import path from 'node:path'; + +import { describe, expect, it } from 'vitest'; + +import { resolveContentDir } from '../content-resolver.ts'; +import { resolveClosure } from '../dependency-resolver.ts'; +import { readDeploy } from '../deploy-frontmatter.ts'; + +describe('capture-feedback delivery', () => { + it('ships capture-feedback as a declared skill in the recommended closure', async () => { + const contentDir = resolveContentDir(); + + const closure = await resolveClosure({ collection: ['recommended'] }, contentDir); + expect(closure.skills).toContain('capture-feedback'); + + const skillMd = await readFile(path.join(contentDir, 'skills', 'capture-feedback', 'SKILL.md'), 'utf8'); + expect(readDeploy(skillMd)).toBe('declared'); + }); +});