diff --git a/packages/agents/content/skills/kb-retrieve-events/SKILL.md b/packages/agents/content/skills/kb-retrieve-events/SKILL.md index 9ca2f7a5..07006cfe 100644 --- a/packages/agents/content/skills/kb-retrieve-events/SKILL.md +++ b/packages/agents/content/skills/kb-retrieve-events/SKILL.md @@ -14,14 +14,15 @@ For assertion recall — the canonical knowledge-base notes — use `kb-retrieve ## Arguments -| Argument | Description | Required | -| ----------- | --------------------------------------------------------------------------------- | -------- | -| `` | The free-text search query. All non-flag tokens are joined into the query string. | Yes | -| `--all-kbs` | Widen the search to every registered knowledge base, not just the default scope. | No | -| `--store` | Scope the search to a single registered knowledge base by name (alias: `--kb`). | No | -| `--tag` | Keep only events carrying this tag (canonical or alias form), case-insensitively. | No | +| Argument | Description | Required | +| -------------- | --------------------------------------------------------------------------------- | -------- | +| `` | The free-text search query. All non-flag tokens are joined into the query string. | Yes | +| `--all-kbs` | Widen the search to every registered knowledge base, not just the default scope. | No | +| `--min-impact` | Keep only events rated at or above the given level; unrated events are dropped. | No | +| `--store` | Scope the search to a single registered knowledge base by name (alias: `--kb`). | No | +| `--tag` | Keep only events carrying this tag (canonical or alias form), case-insensitively. | No | -A value-bearing flag accepts both `--tag fix` and `--tag=fix`. +A value-bearing flag accepts both `--tag fix` and `--tag=fix`. `--min-impact` takes one of the impact levels, ordered `low` < `medium` < `high` < `critical`; an absent or out-of-range value is rejected with a usage message. ### Scope @@ -43,7 +44,7 @@ Within each knowledge base, recall is limited to the notes the store declares Invoke the co-located bundled helper with `node`, passing the query and any flags through verbatim: ```bash -node "$(dirname "$SKILL_PATH")/kb-retrieve-events.mjs" [--all-kbs] [--store ] [--tag ] +node "$(dirname "$SKILL_PATH")/kb-retrieve-events.mjs" [--all-kbs] [--store ] [--tag ] [--min-impact ] ``` Or, when the skill directory is known: @@ -52,9 +53,15 @@ Or, when the skill directory is known: node {harness_home_dir}/skills/kb-retrieve-events/kb-retrieve-events.mjs "flaky timer" --store codeassembly ``` +Triage the most consequential events by floor on impact: + +```bash +node {harness_home_dir}/skills/kb-retrieve-events/kb-retrieve-events.mjs "flaky timer" --store codeassembly --min-impact high +``` + The helper prints a JSON object to stdout: -- `candidates` — an array of event candidates, each with `path`, `summary` (the event's human-readable summary, or the file basename when absent), `capturedAt` (its ISO-8601 capture timestamp, or `null`), `tags`, `snippet`, and `kbName`. Each also carries `occurrences` — a coarse recurrence count of how many query-matched events share its `repo` — and, when present, `repo` (its `owner/name` repository) and `addressedBy` (references recording what was done about the problem it notes). +- `candidates` — an array of event candidates, each with `path`, `summary` (the event's human-readable summary, or the file basename when absent), `capturedAt` (its ISO-8601 capture timestamp, or `null`), `tags`, `snippet`, and `kbName`. Each also carries `occurrences` — a coarse recurrence count of how many query-matched events share its `repo` — and, when present, `repo` (its `owner/name` repository), `addressedBy` (references recording what was done about the problem it notes), and `impact` (the author's rating, one of `low` < `medium` < `high` < `critical`; absent when the event is unrated). - `scopedKbs` — the knowledge bases that were actually searched. - `warnings` — an array (possibly empty) of registry-health problems, present even when candidates are returned. - `diagnostic` — present only when scope is empty or no events matched. @@ -65,9 +72,11 @@ Parse the JSON and rank the `candidates` by genuine relevance to the query's int Once relevance is established, rank by recurrence, then recency: a candidate with a higher `occurrences` count reflects a pattern seen repeatedly in the same `repo` and outranks a one-off of equal relevance; break ties by `capturedAt`, most recent first. Recurrence is a coarse count of query-matched events sharing the group, not a precise cluster — treat it as a strong-but-soft signal. +Do not rank by `impact`. It is the author's subjective rating, orthogonal to a query's relevance, so it is shown to the reader and available as the `--min-impact` filter but never folded into the ordering. + ### 3. Present a ranked list -Present the ranked events, each showing `summary`, `path`, `capturedAt`, and `snippet`. Apply this annotation: +Present the ranked events, each showing `summary`, `path`, `capturedAt`, and `snippet`, plus `impact` when the event carries one. Apply this annotation: - **Addressed problems** — when a candidate carries `addressedBy`, surface its references so a recurring-but-addressed problem reads as _addressed_ rather than _unaddressed_. The references are heterogeneous (a KB note, a commit, a PR/issue, or a URL), and the relation is neutral: it records what was done about the problem, not that the problem is verifiably resolved. The event remains a true observation worth keeping. @@ -80,6 +89,7 @@ When the helper returns a `diagnostic` and no candidates, report the empty resul - `registry invalid: …`: The only configured `kb.yaml` registry failed to load; this is a setup problem to fix, not a missing-events outcome. - `no notes matched the query`: The knowledge bases were searched but nothing matched; suggest broadening the query or adding `--all-kbs`. - `all matches were filtered out`: Matches were found but every one was excluded by `--tag`; suggest dropping or loosening the filter. +- `all matches were below the --min-impact threshold of `: Events matched, but every one was rated below the `--min-impact` floor or was unrated; suggest lowering or dropping the filter. - `matches were found but none are events; use kb-retrieve for assertion recall`: The query matched only non-event records; the reader likely wants `kb-retrieve`. ### 5. Relay registry-health warnings diff --git a/packages/agents/src/kb-retrieve-events/__tests__/cli.test.ts b/packages/agents/src/kb-retrieve-events/__tests__/cli.test.ts index 8989d46e..7bed1c80 100644 --- a/packages/agents/src/kb-retrieve-events/__tests__/cli.test.ts +++ b/packages/agents/src/kb-retrieve-events/__tests__/cli.test.ts @@ -37,6 +37,23 @@ describe(parseArgs, () => { it('throws when --tag has no value', () => { expect(() => parseArgs(['q', '--tag'])).toThrow(/--tag requires a value/); }); + + it('parses --min-impact as the level floor, in both spaced and inline forms', () => { + expect(parseArgs(['q', '--min-impact', 'high']).minImpact).toBe('high'); + expect(parseArgs(['q', '--min-impact=critical']).minImpact).toBe('critical'); + }); + + it('leaves minImpact null when --min-impact is absent', () => { + expect(parseArgs(['q']).minImpact).toBeNull(); + }); + + it('throws when --min-impact has no value', () => { + expect(() => parseArgs(['q', '--min-impact'])).toThrow(/--min-impact requires a value/); + }); + + it('throws when --min-impact is outside the declared levels', () => { + expect(() => parseArgs(['q', '--min-impact', 'louder'])).toThrow(/--min-impact must be one of/); + }); }); describe(runRetrieveEvents, () => { @@ -85,4 +102,34 @@ describe(runRetrieveEvents, () => { expect(result.candidates).toEqual([]); expect(result.diagnostic).toBe('no query provided'); }); + + it('surfaces a declared impact on a candidate and omits it on an unrated one', async () => { + const result = await runRetrieveEvents({ argv: ['snorkleweft'], startDir: NOTES_VAULT, home: FIXTURES }); + + const byImpact = new Map(result.candidates.map((candidate) => [candidate.summary, candidate.impact])); + expect(byImpact.get('A snorkleweft outage rated high')).toBe('high'); + expect(byImpact.get('A snorkleweft observation left unrated')).toBeUndefined(); + }); + + it('keeps only events rated at or above --min-impact, excluding lower and unrated events', async () => { + const result = await runRetrieveEvents({ + argv: ['snorkleweft', '--min-impact', 'high'], + startDir: NOTES_VAULT, + home: FIXTURES, + }); + + expect(result.candidates).toHaveLength(1); + expect(result.candidates[0]).toMatchObject({ summary: 'A snorkleweft outage rated high', impact: 'high' }); + }); + + it('reports a threshold diagnostic when --min-impact filters every match out', async () => { + const result = await runRetrieveEvents({ + argv: ['snorkleweft', '--min-impact', 'critical'], + startDir: NOTES_VAULT, + home: FIXTURES, + }); + + expect(result.candidates).toEqual([]); + expect(result.diagnostic).toBe('all matches were below the --min-impact threshold of critical'); + }); }); diff --git a/packages/agents/src/kb-retrieve-events/__tests__/normalize.test.ts b/packages/agents/src/kb-retrieve-events/__tests__/normalize.test.ts index 57481d18..71c178db 100644 --- a/packages/agents/src/kb-retrieve-events/__tests__/normalize.test.ts +++ b/packages/agents/src/kb-retrieve-events/__tests__/normalize.test.ts @@ -65,6 +65,25 @@ describe(normalizeEvents, () => { addressedBy: ['abc1234', 'owner/repo-x#42', 'https://example.com/fix'], }); }); + + it('carries a declared impact level onto an event candidate', async () => { + const candidates = normalizeEvents({ hits: [await hitFor(join(EVENTS, 'event-impact-high.md'))] }); + + expect(candidates[0]?.impact).toBe('high'); + }); + + it('omits impact when the event declares none', async () => { + const candidates = normalizeEvents({ hits: [await hitFor(join(EVENTS, 'event-a.md'))] }); + + expect(candidates[0]?.impact).toBeUndefined(); + }); + + it('omits an impact value outside the declared levels', async () => { + const candidates = normalizeEvents({ hits: [await hitFor(join(EVENTS, 'event-impact-bad.md'))] }); + + expect(candidates[0]?.summary).toBe('An event carrying an out-of-range impact value'); + expect(candidates[0]?.impact).toBeUndefined(); + }); }); /** Builds a `SearchHit` for a fixture event by parsing it. */ diff --git a/packages/agents/src/kb-retrieve-events/cli.ts b/packages/agents/src/kb-retrieve-events/cli.ts index 1bea90f8..a8f6abe8 100644 --- a/packages/agents/src/kb-retrieve-events/cli.ts +++ b/packages/agents/src/kb-retrieve-events/cli.ts @@ -4,11 +4,13 @@ import { realpathSync } from 'node:fs'; import process from 'node:process'; import { fileURLToPath } from 'node:url'; +import { EVENT_IMPACT_LEVELS, type EventImpact, isEventImpact } from '@codeassembly/kb/records'; + import { recordTypeOf, searchNotes } from '../kb-search/search.ts'; import type { RecallFilters } from '../kb-search/types.ts'; import { type FlagSpec, scanFlags, valueFlagMap } from '../lib/parse-flags.ts'; import { normalizeEvents } from './normalize.ts'; -import type { EventRetrieveResult } from './types.ts'; +import type { EventCandidate, EventRetrieveResult } from './types.ts'; /** The record type kb-retrieve-events owns; every other type (e.g. `assertion`) is left to its own retrieve command. */ const EVENT = 'event'; @@ -16,6 +18,7 @@ const EVENT = 'event'; /** The flags this helper accepts; positionals join into the free-text query. `--kb` is an alias for `--store`. */ const FLAGS: readonly FlagSpec[] = [ { name: 'all-kbs', takesValue: false }, + { name: 'min-impact', takesValue: true }, { name: 'store', aliases: ['kb'], takesValue: true }, { name: 'tag', takesValue: true }, ]; @@ -30,6 +33,8 @@ export interface ParsedArgs { storeName: string | null; /** The mechanical filter from `--tag`. */ filters: RecallFilters; + /** The `--min-impact` floor; candidates rated below it (and unrated ones) are dropped. `null` when absent. */ + minImpact: EventImpact | null; } /** Executes the helper from `process.argv` and write the JSON result to stdout. */ @@ -68,9 +73,10 @@ function isEntryPoint(): boolean { } /** - * Parses the helper's argv into a query, the `--all-kbs` flag, the `--store`/`--kb` store scope, and the `--tag` filter. - * Each value-bearing flag accepts both `--flag value` and `--flag=value`. An unknown flag, or a value-bearing flag given - * no value (or an empty one), throws with a usage-style message. + * Parses the helper's argv into a query, the `--all-kbs` flag, the `--store`/`--kb` store scope, the `--tag` filter, and + * the `--min-impact` floor. Each value-bearing flag accepts both `--flag value` and `--flag=value`. An unknown flag, a + * value-bearing flag given no value (or an empty one), or a `--min-impact` outside the declared levels throws with a + * usage-style message. * * @internal - Exported to allow testing. */ @@ -93,6 +99,7 @@ export function parseArgs(argv: readonly string[]): ParsedArgs { allKbs: flags.some((flag) => flag.name === 'all-kbs'), storeName, filters: tag !== undefined ? { tag } : {}, + minImpact: parseMinImpact(values['min-impact']), }; } @@ -111,7 +118,7 @@ export async function runRetrieveEvents(input: { startDir: string; home?: string; }): Promise { - const { query, allKbs, storeName, filters } = parseArgs(input.argv); + const { query, allKbs, storeName, filters, minImpact } = parseArgs(input.argv); if (query === '') { return { candidates: [], scopedKbs: [], warnings: [], diagnostic: 'no query provided' }; @@ -135,7 +142,9 @@ export async function runRetrieveEvents(input: { }; } - const candidates = normalizeEvents({ hits: search.hits.filter((hit) => recordTypeOf(hit) === EVENT) }); + const eventCandidates = normalizeEvents({ hits: search.hits.filter((hit) => recordTypeOf(hit) === EVENT) }); + const candidates = + minImpact === null ? eventCandidates : eventCandidates.filter((candidate) => meetsMinImpact(candidate, minImpact)); const result: EventRetrieveResult = { candidates, @@ -143,10 +152,10 @@ export async function runRetrieveEvents(input: { warnings: search.warnings, }; if (candidates.length === 0) { - result.diagnostic = emptyResultDiagnostic({ - recalledCount: search.recalledCount, - filteredHits: search.hits.length, - }); + result.diagnostic = + minImpact !== null && eventCandidates.length > 0 + ? `all matches were below the --min-impact threshold of ${minImpact}` + : emptyResultDiagnostic({ recalledCount: search.recalledCount, filteredHits: search.hits.length }); } return result; } @@ -166,4 +175,29 @@ function emptyResultDiagnostic(input: { recalledCount: number; filteredHits: num return 'matches were found but none are events; use kb-retrieve for assertion recall'; } +/** Reports whether a candidate's impact is set and ranks at or above `floor` by the declared level ordering. */ +function meetsMinImpact(candidate: EventCandidate, floor: EventImpact): boolean { + if (candidate.impact === undefined) { + return false; + } + return EVENT_IMPACT_LEVELS.indexOf(candidate.impact) >= EVENT_IMPACT_LEVELS.indexOf(floor); +} + +/** + * Resolves the `--min-impact` value to a level floor, or `null` when the flag is absent. An empty value, or one outside + * the declared levels, throws a usage-style message. + */ +function parseMinImpact(value: string | undefined): EventImpact | null { + if (value === undefined) { + return null; + } + if (value === '') { + throw new Error('--min-impact requires a value'); + } + if (!isEventImpact(value)) { + throw new Error(`--min-impact must be one of ${EVENT_IMPACT_LEVELS.join(', ')}`); + } + return value; +} + // endregion | Helpers diff --git a/packages/agents/src/kb-retrieve-events/normalize.ts b/packages/agents/src/kb-retrieve-events/normalize.ts index b70e04c4..474afc1d 100644 --- a/packages/agents/src/kb-retrieve-events/normalize.ts +++ b/packages/agents/src/kb-retrieve-events/normalize.ts @@ -1,13 +1,16 @@ import { basename } from 'node:path'; +import { isEventImpact } from '@codeassembly/kb/records'; + import type { SearchHit } from '../kb-search/types.ts'; import { extractString, readStringList } from '../kb-shared/note-helpers.ts'; import type { EventCandidate } from './types.ts'; /** * Projects the shared search primitive's event hits onto the event candidate table. Each candidate carries its - * recurrence signals — `captured-at`, `repo`, and an `occurrences` count — plus its `summary`, tags, and any - * `addressed-by` references. After projection, each candidate is stamped with the size of its `repo` recurrence group: + * recurrence signals — `captured-at`, `repo`, and an `occurrences` count — plus its `summary`, tags, any + * `addressed-by` references, and its `impact` rating when set. After projection, each candidate is stamped with the + * size of its `repo` recurrence group: * the count of query-matched events sharing the same repository. A note whose frontmatter is missing or malformed still * projects to a low-signal candidate carrying a diagnostic rather than being dropped. */ @@ -19,13 +22,15 @@ export function normalizeEvents(input: { hits: SearchHit[] }): EventCandidate[] // region | Helpers -/** Projects a single event hit onto a candidate, reading its recurrence signals from frontmatter. */ +/** Projects a single event hit onto a candidate, reading its recurrence signals and impact from frontmatter. */ function toEventCandidate(searchHit: SearchHit): EventCandidate { const { hit, note } = searchHit; const extra = note.frontmatter?.extra; const repo = extractString(extra, 'repo'); const addressedBy = readStringList(extra, 'addressed-by'); + const rawImpact = extra?.impact; + const impact = isEventImpact(rawImpact) ? rawImpact : undefined; const candidate: EventCandidate = { path: hit.path, @@ -37,6 +42,7 @@ function toEventCandidate(searchHit: SearchHit): EventCandidate { kbName: hit.kbName, ...(repo !== null && { repo }), ...(addressedBy.length > 0 && { addressedBy }), + ...(impact !== undefined && { impact }), }; if (note.frontmatter === null) { candidate.diagnostic = 'frontmatter missing or malformed; degraded to a low-signal candidate'; diff --git a/packages/agents/src/kb-retrieve-events/types.ts b/packages/agents/src/kb-retrieve-events/types.ts index 73424a16..9214ba6a 100644 --- a/packages/agents/src/kb-retrieve-events/types.ts +++ b/packages/agents/src/kb-retrieve-events/types.ts @@ -2,6 +2,8 @@ // `kb-search` primitive and projects event records into candidates carrying their recurrence signals; the agent ranks // by recurrence then recency and presents. +import type { EventImpact } from '@codeassembly/kb/records'; + import type { ScopedKb } from '../kb-search/types.ts'; /** A normalized event candidate ready for the agent to rank by recurrence then recency. */ @@ -25,6 +27,12 @@ export interface EventCandidate { * wikilink/relative path, commit SHA, PR/issue ref, or URL. `undefined` when the event declares none. */ addressedBy?: string[]; + /** + * The author's revisable rating of how much addressing this event matters. `undefined` when the event is unrated or + * carries a value outside the declared levels. Shown to the reader and usable by `--min-impact`, but not a ranking + * signal. + */ + impact?: EventImpact; /** Name of the source KB, or `null` for a registry-less discovered KB. */ kbName: string | null; /** A diagnostic note for this candidate, e.g. malformed frontmatter degraded to a low-signal hit. */ diff --git a/packages/agents/src/kb-search/__tests__/fixtures/events/event-impact-bad.md b/packages/agents/src/kb-search/__tests__/fixtures/events/event-impact-bad.md new file mode 100644 index 00000000..006a878d --- /dev/null +++ b/packages/agents/src/kb-search/__tests__/fixtures/events/event-impact-bad.md @@ -0,0 +1,12 @@ +--- +recordType: event +id: 01HZEVENTBADAAAAAAAAAAAAAA +captured-at: 2026-05-25T11:00:00.000Z +session: session-5 +cwd: /tmp/work +repo: owner/repo-x +summary: An event carrying an out-of-range impact value +impact: louder +--- + +An event whose impact value falls outside the declared levels. diff --git a/packages/agents/src/kb-search/__tests__/fixtures/events/event-impact-high.md b/packages/agents/src/kb-search/__tests__/fixtures/events/event-impact-high.md new file mode 100644 index 00000000..0bd86458 --- /dev/null +++ b/packages/agents/src/kb-search/__tests__/fixtures/events/event-impact-high.md @@ -0,0 +1,12 @@ +--- +recordType: event +id: 01HZEVENTHGHAAAAAAAAAAAAAA +captured-at: 2026-05-25T10:00:00.000Z +session: session-5 +cwd: /tmp/work +repo: owner/repo-x +summary: A high-impact regression worth fast-tracking +impact: high +--- + +An event whose impact was rated high during triage. diff --git a/packages/agents/src/kb-search/__tests__/fixtures/notes-vault/content/events/01HZCEVENTHGHAAAAAAAAAAAAA.md b/packages/agents/src/kb-search/__tests__/fixtures/notes-vault/content/events/01HZCEVENTHGHAAAAAAAAAAAAA.md new file mode 100644 index 00000000..cc8609b2 --- /dev/null +++ b/packages/agents/src/kb-search/__tests__/fixtures/notes-vault/content/events/01HZCEVENTHGHAAAAAAAAAAAAA.md @@ -0,0 +1,12 @@ +--- +recordType: event +id: 01HZCEVENTHGHAAAAAAAAAAAAA +captured-at: 2026-04-21T09:00:00.000Z +session: session-content +cwd: /tmp/work +repo: owner/repo-content +summary: A snorkleweft outage rated high +impact: high +--- + +The snorkleweft outage took down the whole pipeline. diff --git a/packages/agents/src/kb-search/__tests__/fixtures/notes-vault/content/events/01HZCEVENTMNRAAAAAAAAAAAAA.md b/packages/agents/src/kb-search/__tests__/fixtures/notes-vault/content/events/01HZCEVENTMNRAAAAAAAAAAAAA.md new file mode 100644 index 00000000..4a638cd7 --- /dev/null +++ b/packages/agents/src/kb-search/__tests__/fixtures/notes-vault/content/events/01HZCEVENTMNRAAAAAAAAAAAAA.md @@ -0,0 +1,12 @@ +--- +recordType: event +id: 01HZCEVENTMNRAAAAAAAAAAAAA +captured-at: 2026-04-21T09:05:00.000Z +session: session-content +cwd: /tmp/work +repo: owner/repo-content +summary: A snorkleweft cosmetic quirk rated low +impact: low +--- + +The snorkleweft quirk is purely cosmetic. diff --git a/packages/agents/src/kb-search/__tests__/fixtures/notes-vault/content/events/01HZCEVENTNRTAAAAAAAAAAAAA.md b/packages/agents/src/kb-search/__tests__/fixtures/notes-vault/content/events/01HZCEVENTNRTAAAAAAAAAAAAA.md new file mode 100644 index 00000000..25a4974c --- /dev/null +++ b/packages/agents/src/kb-search/__tests__/fixtures/notes-vault/content/events/01HZCEVENTNRTAAAAAAAAAAAAA.md @@ -0,0 +1,11 @@ +--- +recordType: event +id: 01HZCEVENTNRTAAAAAAAAAAAAA +captured-at: 2026-04-21T09:10:00.000Z +session: session-content +cwd: /tmp/work +repo: owner/repo-content +summary: A snorkleweft observation left unrated +--- + +The snorkleweft observation has no impact rating.