feat(analyst): registry + findings envelope over existing primitives

[drewstone]

Summary

Adds a generic, model-agnostic, transport-agnostic Analyst layer that orchestrates agent-eval's existing analyzers (analyzeTraces, MultiLayerVerifier, RunCritic, SemanticConceptJudge, JudgeFn) without re-implementing them. One contract, one runner, one persistence path — reusable by VB operator bench, the leaderboard submission pipeline, and the orchestrator on-completion /v1/intelligence/reports mode:'agentic' surface with the same code.

This is PR-A of a three-PR series:

• PR-A (this): shared substrate in @tangle-network/agent-eval@0.28.0
• PR-B (next, blueprint-agent): adopt 0.28.0, extract VB analysts (archetype miner, propose-fixes), wire submit-external-run.ts to persist findings + render
• PR-C (next, agent-dev-container): register generateIntelligenceReport as an Analyst; route mode:'agentic' through the registry

What's in

• src/analyst/types.ts — Analyst<TInput> contract, AnalystFinding envelope with sha-stable finding_id (so cross-run diffs work), AnalystRunInputs discriminated by inputKind (trace-store | artifact-dir | run-record | judge-input | custom)
• src/analyst/chat-client.ts — unified ChatClient over router | sandbox-sdk | cli-bridge | direct-provider | mock so analyst code never knows the transport (production binds to sandbox-sdk; tests bind to mock)
• src/analyst/registry.ts — register/list/run with input routing, per-analyst isolation (one analyst failure does not stop the run), equal-split budgeting, per-analyst telemetry (latency_ms, cost_usd, status, error.class)
• src/analyst/findings-store.ts — locked JSONL append + diffFindings returning { appeared, disappeared, persisted, changed } keyed by stable id, with a narrow material-change definition (severity / confidence > 0.05 / evidence count) to avoid LLM-reword noise
• src/analyst/adapters.ts — five thin lifters that wrap the existing primitives into the Analyst shape (no duplication; the wrapped code is unchanged)
• src/analyst/analyst.test.ts — 12 unit tests

Version bump: 0.27.2 → 0.28.0.

Test plan

• pnpm tsc --noEmit — clean
• pnpm vitest run src/analyst/analyst.test.ts — 12/12 pass
• pnpm test (full suite) — 1117/1117 pass, no regressions
• pnpm build — clean, OpenAPI spec re-emits

Design notes

• Model-agnostic. Nothing in analyst/ imports a specific provider or model name. Analysts that call an LLM go through ChatClient on AnalystContext.chat.
• Transport-agnostic. Production runs bind to sandbox-sdk; local dev binds to cli-bridge; tests bind to mock. Swapping is a one-line createChatClient(...) call.
• Extends, doesn't duplicate. Existing LlmClient callers and TCloud-based judges keep working untouched. The five adapters lift, they don't re-implement.
• finding_id stability. Hash over {analyst_id, area, subject, normalized_claim} (or explicit id_basis) — two runs that "find the same thing" share the id, which is what makes diffFindings meaningful for regression tracking.
• Isolation invariant. A throwing analyst becomes a status: 'failed' summary row with error.class + error.message; siblings still run.

[drewstone]

Pushed 7d03b7a addressing the review. Took the architectural framing — most of these concerns are policies, so they collapse into a small hook + policy surface instead of per-issue patches.

#1 ChatClient wiring — fixed. AnalystContext.chat is now typed as ChatClient directly. The cast in registry.run() is gone; analysts that call ctx.chat.chat(...) get the documented API.

#2 Signal threading — partial, with honest scope. wrapLlmClient now races the call against ChatCallOpts.signal so awaiting code unblocks on abort. The in-flight HTTP request still completes (or hits its own timeoutMs) because LlmClient.call doesn't accept an external AbortSignal yet. That's a one-line addition to LlmClient in a follow-up; the inline comment now reflects the actual contract instead of a TODO.

#4 Diff materiality — pluggable. diffFindings(prev, cur, { isMaterial }) takes an optional predicate. Default exported as defaultIsMaterial so consumers can layer (e.g. (a,b) => defaultIsMaterial(a,b) || a.rationale !== b.rationale) without re-implementing the base.

#5 Budget — pluggable. RegistryRunOpts.budget is now { totalUsd, weights, allocate }. Default is still equal split; weighted split via weights; precise control via allocate({ analyst, totalUsd, remainingUsd, runningCount }).

#6 Rotation, #+ telemetry, #+ error→finding — covered by AnalystHooks. New surface: onBeforeAnalyze | onAfterAnalyze | onError | onComplete. onAfterAnalyze fires for ok/failed/skipped — that's where rotation, ingestion, cost-aggregation belong. onError MAY return findings to convert a crash into structured signal. This was the architectural answer the reviewer was probing for: a generic seam for cross-cutting concerns instead of a hardcoded list of features.

#3 AxAIService coupling — non-issue. @tangle-network/agent-eval already declares @ax-llm/ax as a dep (the trace analyst itself lives in this package). Importing createTraceAnalystAdapter doesn't add a transitive — it surfaces what's already there. Consumers that don't need that adapter don't import it.

#7 schema_version literal — leaving for now. Migration helper is a 0.29 concern; current literal forces a deliberate type bump on breaking changes, which is the right ergonomic for 1.0.

#8 Isolation under load — added. Hook tests + error-conversion tests cover the failure-isolation invariant from another angle (and the failing analyst's siblings are observed via onAfterAnalyze).

Tests: 20 in the analyst file, 1125 across the suite, all green.