Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 20 additions & 10 deletions packages/agents/content/skills/kb-retrieve-events/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,15 @@ For assertion recall — the canonical knowledge-base notes — use `kb-retrieve

## Arguments

| Argument | Description | Required |
| ----------- | --------------------------------------------------------------------------------- | -------- |
| `<query>` | The free-text search query. All non-flag tokens are joined into the query string. | Yes |
| `--all-kbs` | Widen the search to every registered knowledge base, not just the default scope. | No |
| `--store` | Scope the search to a single registered knowledge base by name (alias: `--kb`). | No |
| `--tag` | Keep only events carrying this tag (canonical or alias form), case-insensitively. | No |
| Argument | Description | Required |
| -------------- | --------------------------------------------------------------------------------- | -------- |
| `<query>` | The free-text search query. All non-flag tokens are joined into the query string. | Yes |
| `--all-kbs` | Widen the search to every registered knowledge base, not just the default scope. | No |
| `--min-impact` | Keep only events rated at or above the given level; unrated events are dropped. | No |
| `--store` | Scope the search to a single registered knowledge base by name (alias: `--kb`). | No |
| `--tag` | Keep only events carrying this tag (canonical or alias form), case-insensitively. | No |

A value-bearing flag accepts both `--tag fix` and `--tag=fix`.
A value-bearing flag accepts both `--tag fix` and `--tag=fix`. `--min-impact` takes one of the impact levels, ordered `low` < `medium` < `high` < `critical`; an absent or out-of-range value is rejected with a usage message.

### Scope

Expand All @@ -43,7 +44,7 @@ Within each knowledge base, recall is limited to the notes the store declares
Invoke the co-located bundled helper with `node`, passing the query and any flags through verbatim:

```bash
node "$(dirname "$SKILL_PATH")/kb-retrieve-events.mjs" <query> [--all-kbs] [--store <name>] [--tag <tag>]
node "$(dirname "$SKILL_PATH")/kb-retrieve-events.mjs" <query> [--all-kbs] [--store <name>] [--tag <tag>] [--min-impact <level>]
```

Or, when the skill directory is known:
Expand All @@ -52,9 +53,15 @@ Or, when the skill directory is known:
node {harness_home_dir}/skills/kb-retrieve-events/kb-retrieve-events.mjs "flaky timer" --store codeassembly
```

Triage the most consequential events by floor on impact:

```bash
node {harness_home_dir}/skills/kb-retrieve-events/kb-retrieve-events.mjs "flaky timer" --store codeassembly --min-impact high
```

The helper prints a JSON object to stdout:

- `candidates` — an array of event candidates, each with `path`, `summary` (the event's human-readable summary, or the file basename when absent), `capturedAt` (its ISO-8601 capture timestamp, or `null`), `tags`, `snippet`, and `kbName`. Each also carries `occurrences` — a coarse recurrence count of how many query-matched events share its `repo` — and, when present, `repo` (its `owner/name` repository) and `addressedBy` (references recording what was done about the problem it notes).
- `candidates` — an array of event candidates, each with `path`, `summary` (the event's human-readable summary, or the file basename when absent), `capturedAt` (its ISO-8601 capture timestamp, or `null`), `tags`, `snippet`, and `kbName`. Each also carries `occurrences` — a coarse recurrence count of how many query-matched events share its `repo` — and, when present, `repo` (its `owner/name` repository), `addressedBy` (references recording what was done about the problem it notes), and `impact` (the author's rating, one of `low` < `medium` < `high` < `critical`; absent when the event is unrated).
- `scopedKbs` — the knowledge bases that were actually searched.
- `warnings` — an array (possibly empty) of registry-health problems, present even when candidates are returned.
- `diagnostic` — present only when scope is empty or no events matched.
Expand All @@ -65,9 +72,11 @@ Parse the JSON and rank the `candidates` by genuine relevance to the query's int

Once relevance is established, rank by recurrence, then recency: a candidate with a higher `occurrences` count reflects a pattern seen repeatedly in the same `repo` and outranks a one-off of equal relevance; break ties by `capturedAt`, most recent first. Recurrence is a coarse count of query-matched events sharing the group, not a precise cluster — treat it as a strong-but-soft signal.

Do not rank by `impact`. It is the author's subjective rating, orthogonal to a query's relevance, so it is shown to the reader and available as the `--min-impact` filter but never folded into the ordering.

### 3. Present a ranked list

Present the ranked events, each showing `summary`, `path`, `capturedAt`, and `snippet`. Apply this annotation:
Present the ranked events, each showing `summary`, `path`, `capturedAt`, and `snippet`, plus `impact` when the event carries one. Apply this annotation:

- **Addressed problems** — when a candidate carries `addressedBy`, surface its references so a recurring-but-addressed problem reads as _addressed_ rather than _unaddressed_. The references are heterogeneous (a KB note, a commit, a PR/issue, or a URL), and the relation is neutral: it records what was done about the problem, not that the problem is verifiably resolved. The event remains a true observation worth keeping.

Expand All @@ -80,6 +89,7 @@ When the helper returns a `diagnostic` and no candidates, report the empty resul
- `registry invalid: …`: The only configured `kb.yaml` registry failed to load; this is a setup problem to fix, not a missing-events outcome.
- `no notes matched the query`: The knowledge bases were searched but nothing matched; suggest broadening the query or adding `--all-kbs`.
- `all matches were filtered out`: Matches were found but every one was excluded by `--tag`; suggest dropping or loosening the filter.
- `all matches were below the --min-impact threshold of <level>`: Events matched, but every one was rated below the `--min-impact` floor or was unrated; suggest lowering or dropping the filter.
- `matches were found but none are events; use kb-retrieve for assertion recall`: The query matched only non-event records; the reader likely wants `kb-retrieve`.

### 5. Relay registry-health warnings
Expand Down
47 changes: 47 additions & 0 deletions packages/agents/src/kb-retrieve-events/__tests__/cli.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,23 @@ describe(parseArgs, () => {
it('throws when --tag has no value', () => {
expect(() => parseArgs(['q', '--tag'])).toThrow(/--tag requires a value/);
});

it('parses --min-impact as the level floor, in both spaced and inline forms', () => {
expect(parseArgs(['q', '--min-impact', 'high']).minImpact).toBe('high');
expect(parseArgs(['q', '--min-impact=critical']).minImpact).toBe('critical');
});

it('leaves minImpact null when --min-impact is absent', () => {
expect(parseArgs(['q']).minImpact).toBeNull();
});

it('throws when --min-impact has no value', () => {
expect(() => parseArgs(['q', '--min-impact'])).toThrow(/--min-impact requires a value/);
});

it('throws when --min-impact is outside the declared levels', () => {
expect(() => parseArgs(['q', '--min-impact', 'louder'])).toThrow(/--min-impact must be one of/);
});
});

describe(runRetrieveEvents, () => {
Expand Down Expand Up @@ -85,4 +102,34 @@ describe(runRetrieveEvents, () => {
expect(result.candidates).toEqual([]);
expect(result.diagnostic).toBe('no query provided');
});

it('surfaces a declared impact on a candidate and omits it on an unrated one', async () => {
const result = await runRetrieveEvents({ argv: ['snorkleweft'], startDir: NOTES_VAULT, home: FIXTURES });

const byImpact = new Map(result.candidates.map((candidate) => [candidate.summary, candidate.impact]));
expect(byImpact.get('A snorkleweft outage rated high')).toBe('high');
expect(byImpact.get('A snorkleweft observation left unrated')).toBeUndefined();
});

it('keeps only events rated at or above --min-impact, excluding lower and unrated events', async () => {
const result = await runRetrieveEvents({
argv: ['snorkleweft', '--min-impact', 'high'],
startDir: NOTES_VAULT,
home: FIXTURES,
});

expect(result.candidates).toHaveLength(1);
expect(result.candidates[0]).toMatchObject({ summary: 'A snorkleweft outage rated high', impact: 'high' });
});

it('reports a threshold diagnostic when --min-impact filters every match out', async () => {
const result = await runRetrieveEvents({
argv: ['snorkleweft', '--min-impact', 'critical'],
startDir: NOTES_VAULT,
home: FIXTURES,
});

expect(result.candidates).toEqual([]);
expect(result.diagnostic).toBe('all matches were below the --min-impact threshold of critical');
});
});
19 changes: 19 additions & 0 deletions packages/agents/src/kb-retrieve-events/__tests__/normalize.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,25 @@ describe(normalizeEvents, () => {
addressedBy: ['abc1234', 'owner/repo-x#42', 'https://example.com/fix'],
});
});

it('carries a declared impact level onto an event candidate', async () => {
const candidates = normalizeEvents({ hits: [await hitFor(join(EVENTS, 'event-impact-high.md'))] });

expect(candidates[0]?.impact).toBe('high');
});

it('omits impact when the event declares none', async () => {
const candidates = normalizeEvents({ hits: [await hitFor(join(EVENTS, 'event-a.md'))] });

expect(candidates[0]?.impact).toBeUndefined();
});

it('omits an impact value outside the declared levels', async () => {
const candidates = normalizeEvents({ hits: [await hitFor(join(EVENTS, 'event-impact-bad.md'))] });

expect(candidates[0]?.summary).toBe('An event carrying an out-of-range impact value');
expect(candidates[0]?.impact).toBeUndefined();
});
});

/** Builds a `SearchHit` for a fixture event by parsing it. */
Expand Down
54 changes: 44 additions & 10 deletions packages/agents/src/kb-retrieve-events/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,21 @@ import { realpathSync } from 'node:fs';
import process from 'node:process';
import { fileURLToPath } from 'node:url';

import { EVENT_IMPACT_LEVELS, type EventImpact, isEventImpact } from '@codeassembly/kb/records';

import { recordTypeOf, searchNotes } from '../kb-search/search.ts';
import type { RecallFilters } from '../kb-search/types.ts';
import { type FlagSpec, scanFlags, valueFlagMap } from '../lib/parse-flags.ts';
import { normalizeEvents } from './normalize.ts';
import type { EventRetrieveResult } from './types.ts';
import type { EventCandidate, EventRetrieveResult } from './types.ts';

/** The record type kb-retrieve-events owns; every other type (e.g. `assertion`) is left to its own retrieve command. */
const EVENT = 'event';

/** The flags this helper accepts; positionals join into the free-text query. `--kb` is an alias for `--store`. */
const FLAGS: readonly FlagSpec[] = [
{ name: 'all-kbs', takesValue: false },
{ name: 'min-impact', takesValue: true },
{ name: 'store', aliases: ['kb'], takesValue: true },
{ name: 'tag', takesValue: true },
];
Expand All @@ -30,6 +33,8 @@ export interface ParsedArgs {
storeName: string | null;
/** The mechanical filter from `--tag`. */
filters: RecallFilters;
/** The `--min-impact` floor; candidates rated below it (and unrated ones) are dropped. `null` when absent. */
minImpact: EventImpact | null;
}

/** Executes the helper from `process.argv` and write the JSON result to stdout. */
Expand Down Expand Up @@ -68,9 +73,10 @@ function isEntryPoint(): boolean {
}

/**
* Parses the helper's argv into a query, the `--all-kbs` flag, the `--store`/`--kb` store scope, and the `--tag` filter.
* Each value-bearing flag accepts both `--flag value` and `--flag=value`. An unknown flag, or a value-bearing flag given
* no value (or an empty one), throws with a usage-style message.
* Parses the helper's argv into a query, the `--all-kbs` flag, the `--store`/`--kb` store scope, the `--tag` filter, and
* the `--min-impact` floor. Each value-bearing flag accepts both `--flag value` and `--flag=value`. An unknown flag, a
* value-bearing flag given no value (or an empty one), or a `--min-impact` outside the declared levels throws with a
* usage-style message.
*
* @internal - Exported to allow testing.
*/
Expand All @@ -93,6 +99,7 @@ export function parseArgs(argv: readonly string[]): ParsedArgs {
allKbs: flags.some((flag) => flag.name === 'all-kbs'),
storeName,
filters: tag !== undefined ? { tag } : {},
minImpact: parseMinImpact(values['min-impact']),
};
}

Expand All @@ -111,7 +118,7 @@ export async function runRetrieveEvents(input: {
startDir: string;
home?: string;
}): Promise<EventRetrieveResult> {
const { query, allKbs, storeName, filters } = parseArgs(input.argv);
const { query, allKbs, storeName, filters, minImpact } = parseArgs(input.argv);

if (query === '') {
return { candidates: [], scopedKbs: [], warnings: [], diagnostic: 'no query provided' };
Expand All @@ -135,18 +142,20 @@ export async function runRetrieveEvents(input: {
};
}

const candidates = normalizeEvents({ hits: search.hits.filter((hit) => recordTypeOf(hit) === EVENT) });
const eventCandidates = normalizeEvents({ hits: search.hits.filter((hit) => recordTypeOf(hit) === EVENT) });
const candidates =
minImpact === null ? eventCandidates : eventCandidates.filter((candidate) => meetsMinImpact(candidate, minImpact));

const result: EventRetrieveResult = {
candidates,
scopedKbs: search.scopedKbs,
warnings: search.warnings,
};
if (candidates.length === 0) {
result.diagnostic = emptyResultDiagnostic({
recalledCount: search.recalledCount,
filteredHits: search.hits.length,
});
result.diagnostic =
minImpact !== null && eventCandidates.length > 0
? `all matches were below the --min-impact threshold of ${minImpact}`
: emptyResultDiagnostic({ recalledCount: search.recalledCount, filteredHits: search.hits.length });
}
return result;
}
Expand All @@ -166,4 +175,29 @@ function emptyResultDiagnostic(input: { recalledCount: number; filteredHits: num
return 'matches were found but none are events; use kb-retrieve for assertion recall';
}

/** Reports whether a candidate's impact is set and ranks at or above `floor` by the declared level ordering. */
function meetsMinImpact(candidate: EventCandidate, floor: EventImpact): boolean {
if (candidate.impact === undefined) {
return false;
}
return EVENT_IMPACT_LEVELS.indexOf(candidate.impact) >= EVENT_IMPACT_LEVELS.indexOf(floor);
}

/**
* Resolves the `--min-impact` value to a level floor, or `null` when the flag is absent. An empty value, or one outside
* the declared levels, throws a usage-style message.
*/
function parseMinImpact(value: string | undefined): EventImpact | null {
if (value === undefined) {
return null;
}
if (value === '') {
throw new Error('--min-impact requires a value');
}
if (!isEventImpact(value)) {
throw new Error(`--min-impact must be one of ${EVENT_IMPACT_LEVELS.join(', ')}`);
}
return value;
}

// endregion | Helpers
12 changes: 9 additions & 3 deletions packages/agents/src/kb-retrieve-events/normalize.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,16 @@
import { basename } from 'node:path';

import { isEventImpact } from '@codeassembly/kb/records';

import type { SearchHit } from '../kb-search/types.ts';
import { extractString, readStringList } from '../kb-shared/note-helpers.ts';
import type { EventCandidate } from './types.ts';

/**
* Projects the shared search primitive's event hits onto the event candidate table. Each candidate carries its
* recurrence signals — `captured-at`, `repo`, and an `occurrences` count — plus its `summary`, tags, and any
* `addressed-by` references. After projection, each candidate is stamped with the size of its `repo` recurrence group:
* recurrence signals — `captured-at`, `repo`, and an `occurrences` count — plus its `summary`, tags, any
* `addressed-by` references, and its `impact` rating when set. After projection, each candidate is stamped with the
* size of its `repo` recurrence group:
* the count of query-matched events sharing the same repository. A note whose frontmatter is missing or malformed still
* projects to a low-signal candidate carrying a diagnostic rather than being dropped.
*/
Expand All @@ -19,13 +22,15 @@ export function normalizeEvents(input: { hits: SearchHit[] }): EventCandidate[]

// region | Helpers

/** Projects a single event hit onto a candidate, reading its recurrence signals from frontmatter. */
/** Projects a single event hit onto a candidate, reading its recurrence signals and impact from frontmatter. */
function toEventCandidate(searchHit: SearchHit): EventCandidate {
const { hit, note } = searchHit;
const extra = note.frontmatter?.extra;

const repo = extractString(extra, 'repo');
const addressedBy = readStringList(extra, 'addressed-by');
const rawImpact = extra?.impact;
const impact = isEventImpact(rawImpact) ? rawImpact : undefined;

const candidate: EventCandidate = {
path: hit.path,
Expand All @@ -37,6 +42,7 @@ function toEventCandidate(searchHit: SearchHit): EventCandidate {
kbName: hit.kbName,
...(repo !== null && { repo }),
...(addressedBy.length > 0 && { addressedBy }),
...(impact !== undefined && { impact }),
};
if (note.frontmatter === null) {
candidate.diagnostic = 'frontmatter missing or malformed; degraded to a low-signal candidate';
Expand Down
8 changes: 8 additions & 0 deletions packages/agents/src/kb-retrieve-events/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
// `kb-search` primitive and projects event records into candidates carrying their recurrence signals; the agent ranks
// by recurrence then recency and presents.

import type { EventImpact } from '@codeassembly/kb/records';

import type { ScopedKb } from '../kb-search/types.ts';

/** A normalized event candidate ready for the agent to rank by recurrence then recency. */
Expand All @@ -25,6 +27,12 @@ export interface EventCandidate {
* wikilink/relative path, commit SHA, PR/issue ref, or URL. `undefined` when the event declares none.
*/
addressedBy?: string[];
/**
* The author's revisable rating of how much addressing this event matters. `undefined` when the event is unrated or
* carries a value outside the declared levels. Shown to the reader and usable by `--min-impact`, but not a ranking
* signal.
*/
impact?: EventImpact;
/** Name of the source KB, or `null` for a registry-less discovered KB. */
kbName: string | null;
/** A diagnostic note for this candidate, e.g. malformed frontmatter degraded to a low-signal hit. */
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
recordType: event
id: 01HZEVENTBADAAAAAAAAAAAAAA
captured-at: 2026-05-25T11:00:00.000Z
session: session-5
cwd: /tmp/work
repo: owner/repo-x
summary: An event carrying an out-of-range impact value
impact: louder
---

An event whose impact value falls outside the declared levels.
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
recordType: event
id: 01HZEVENTHGHAAAAAAAAAAAAAA
captured-at: 2026-05-25T10:00:00.000Z
session: session-5
cwd: /tmp/work
repo: owner/repo-x
summary: A high-impact regression worth fast-tracking
impact: high
---

An event whose impact was rated high during triage.
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
recordType: event
id: 01HZCEVENTHGHAAAAAAAAAAAAA
captured-at: 2026-04-21T09:00:00.000Z
session: session-content
cwd: /tmp/work
repo: owner/repo-content
summary: A snorkleweft outage rated high
impact: high
---

The snorkleweft outage took down the whole pipeline.
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
recordType: event
id: 01HZCEVENTMNRAAAAAAAAAAAAA
captured-at: 2026-04-21T09:05:00.000Z
session: session-content
cwd: /tmp/work
repo: owner/repo-content
summary: A snorkleweft cosmetic quirk rated low
impact: low
---

The snorkleweft quirk is purely cosmetic.
Loading
Loading