v6.0.0 — Wave 1 Observability Release (Institutional Audit Traceability)#76
Merged
Conversation
Baseline the Wave 1 reference documents on observability/wave-1 before implementation begins. - observability-updates-april-26.md — scoping, retrofit-cost roadmap (4 waves), per-item complexity/break-risk/time ratings, acceptance criteria, explicit out-of-scope. - observability-implementation-spec.md — granular per-module spec: file paths, function signatures, NDJSON row schemas, test matrices, rollout plan, cross-wave concerns (feature flags, env vars, alerts, DR runbook), module dependency graph. Wave 1 scope (to follow in subsequent commits): #3 raw-source archive (Path B: session-dir + global pool + per-agent manifests) #8 prompt-injection detection on tool outputs #12 per-tool latency histograms (P50/P95/P99) #13 per-API 7-day SLA dashboard All four items gated behind feature flags defaulting to false. Module decomposition + NDJSON schema versioning bundled day-one to avoid disproportionate retrofit cost later. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…CTION, SLA_TELEMETRY All three default to false so Wave 1 code can land in production with zero behavior change until individually toggled on. - RAW_SOURCE_ARCHIVE — gates content-addressed pool writes (#3) - PROMPT_INJECTION_DETECTION — gates regex detector in PostToolUse (#8) - SLA_TELEMETRY — gates _hybrid_metadata extraction in hookDBBridge (#13) #12 (histogram label refactor) is unconditional — additive Prometheus label change, no flag needed. Verified via runtime import: all three evaluate to false with no environment overrides. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pure module — first of seven under src/utils/rawSource/. No side
effects, no I/O, trivially unit-testable.
Design change from earlier spec draft: **no canonicalization**.
The earlier spec specified text canonicalization (trim + collapse
whitespace) before hashing, to improve dedup hit rate when the same
document is re-fetched with trivial whitespace differences. That was
rejected in favor of byte-exact audit fidelity:
- stored bytes == API response bytes (modulo secret sanitization, a
legitimate security transform auditors accept)
- recomputing SHA-256 on a pool file matches the filename directly
- an auditor can re-fetch from the API and compare bytes without
having to replicate any canonicalization pipeline
- realistic dedup loss is small — HTTP responses for the same URL
from the same client tend to be byte-stable
HashResult shape simplified: { hash, bytes, size, inferredContentType }.
Content-type sniff (html/json/xml/text/binary) is informational only —
drives filename extension, never mutates bytes.
Spec updates in the same commit:
- observability-implementation-spec.md §1.1.1 — Option B design note
- observability-updates-april-26.md — write-pipeline step ordering
(sanitize precedes hash; no canonicalize step)
- module summary tagline updated
Tests: 27 pass in 91ms under NODE_OPTIONS=--experimental-vm-modules jest.
Covers: determinism, whitespace-different-inputs-different-hashes,
byte-exact storage, filename-integrity (recomputed SHA matches),
content-type sniffing (incl. binary NUL detection), input validation
(TypeError on null/undefined/number/object), empty input, 1 MB
performance (<50 ms).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Second of seven pure modules under src/utils/rawSource/. The only
transform applied before raw bytes land in the content-addressed pool;
legitimate under Option B audit posture because leaking credentials
into the archive is a separate security incident that auditors expect
us to prevent.
Pattern set (5):
authorization_header — Authorization: Bearer/Basic <token>
api_key_query — ?api_key= / ?api-key= / ?apikey= in URLs
(preserves the ?/& separator so URLs remain parseable)
aws_access_key — AKIA + 16 alphanum caps, word-bounded
jwt — three dot-separated base64url segments
private_key_block — PEM-armored RSA/EC/DSA/OPENSSH/ENCRYPTED keys
Replacement format: [REDACTED:<pattern_name>]. Pattern names (not
values) are preserved in the SanitizeResult.redactions audit so the
metadata sidecar can record WHAT was redacted without storing the
secret itself.
Defensive properties:
- never throws (null/undefined/non-string → empty-result sentinel)
- pure function — no I/O, no state leak (fresh RegExp per pattern
to avoid lastIndex state)
- modified=false on clean text (zero-copy short-circuit via early return)
- no false positives on "ignore all prior filings" or plain SEC URLs
Tests: 27 pass in 98ms.
- Per-pattern detection for all 5 formats
- Negative cases: clean SEC text, plain URLs, non-JWT base64
- Edge cases: word boundaries on AKIA (no partial match), lowercase
rejection for AWS, case-insensitivity for Authorization, multi-pattern
documents with correct per-pattern counts, original-secret leakage
check (cleaned output MUST NOT contain the secret substring)
- Defensive: empty string, null, undefined, number → empty-result
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Third of seven modules under src/utils/rawSource/. Stateful factory —
binds to a single pool directory and exposes content-addressed read/write
with atomic, idempotent, integrity-checked semantics.
Storage layout:
{poolDir}/{hash[0:2]}/{hash[2:4]}/{hash}.{ext}.gz body (gzip)
{poolDir}/meta/{hash}.json metadata sidecar
API surface:
pathForHash(hash, ext) → sharded body path
metaPathForHash(hash) → sidecar path
exists(hash, ext) → boolean
write(hash, ext, content) → { written, path, size, compressedSize }
- tmp + rename → atomic
- chmod 0o444 after write → tamper-resistant
- idempotent: second call with same hash =
written:false, no disk I/O, mtime unchanged
- throws if size > maxRawBytes (default 10 MB)
writeMeta(hash, meta) → atomic JSON sidecar write
read(hash, ext) → decompressed body, throws ChecksumError if
recomputed SHA != filename hash
readMeta(hash) → parsed JSON or null on ENOENT
statCompressed(hash, ext) → on-disk size
ChecksumError class exported with { expected, actual, path } context for
upstream alerting (Wave 3 wires this into the error taxonomy + circuit breaker).
Tests: 21 pass in 122ms against real temp-dir filesystems.
- factory validation (poolDir required, exposed API surface)
- sharded path construction (incl. compress=false omitting .gz)
- first-landing write: returns written:true, file is 0o444, gzip is
decompressible back to input bytes, accepts Buffer directly
- dedup: second write returns written:false, mtime unchanged
- size guard: throws past maxRawBytes, accepts at exact boundary
- integrity: round-trip succeeds; tampered file → ChecksumError with
correct expected/actual/path
- meta: write/read round-trip, ENOENT returns null
- concurrency: 5 parallel writes for same hash → exactly one file,
no .tmp.* remnants, body correct on read
Note: removed setTimeout from dedup test — Jest experimental VM modules
hangs on async setTimeout in some configurations (verified the dedup
short-circuit is instant via direct node script: 0 ms).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fourth and fifth modules under src/utils/rawSource/ — both stateful
factories that perform append-only NDJSON writes. Bundled into one
commit because they share design discipline (append-only, parent-dir-
on-demand, schema-version-agnostic).
SourceManifestWriter — per-session and per-agent manifests:
appendSession(sessionId, row) → {sessionsRoot}/{sessionId}/raw-sources-manifest.ndjson
appendAgent(sessionId, agentType, row) → {sessionsRoot}/{sessionId}/specialist-reports/{agentType}-sources/sources.ndjson
- Path traversal guard: agentType matches /^[a-z0-9][a-z0-9_-]*$/i
(rejects '..', absolute paths, spaces).
- Writer is intentionally dumb — does NOT validate row shape.
schema_version presence and field correctness are the orchestrator's
responsibility.
- Uses fs.appendFile (Node O_APPEND under the hood) — concurrent
appends from the same process produce well-formed NDJSON.
SourceIndexWriter — global tamper-evident _index.ndjson:
append(row) → {poolDir}/_index.ndjson
- Per-call: open(a) + write + fsync + close.
- The fsync per row is the difference from manifests: tail entries
cannot be lost on crash. Cost is acceptable because append() only
fires on dedup miss (new hash landings are rare).
- Future Wave 3 hook: nightly Merkle root over this file becomes the
tamper-evident anchor.
Tests: 23 pass (12 manifest + 11 index) in 272ms against real temp dirs.
Manifest:
- factory validation, exposed surface
- session path / agent path correctness
- parent-directory creation on first append
- strict NDJSON (one object per line, newline-terminated)
- rich row shapes round-trip
- path-traversal rejection ('../etc/passwd', '/abs', 'name with space')
- safe agent-type acceptance (alphanum, hyphen, underscore, mixed case)
- 10 parallel appendSession → 10 well-formed rows, all values present
Index:
- factory validation, indexPath exposed
- single-row + multi-row ordering
- poolDir creation on demand
- strict JSON lines (no array wrapper, no trailing comma, newline-terminated)
- rich row shapes round-trip (incl. nested objects)
- 20 parallel appends → 20 distinct well-formed rows
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sixth of seven modules under src/utils/rawSource/. Intentionally minimal: preserves the interface the orchestrator will call so RawSourceService can `dispatcher.enqueue(hash, sourceType)` unconditionally — no branching on a feature flag for an absent real implementation. Wave 2 replaces this stub with: - bounded worker pool (BATCH_SIZE=20, MAX_DEPTH=500) - dedup check against source_chunk_embeddings (no re-embed) - chunkContent → embedDocuments (Gemini RETRIEVAL_DOCUMENT) - transactional INSERT into source_chunk_embeddings table - flag: RAW_SOURCE_EMBEDDING (default false) Wave 3 adds: - backpressure: shed-work above MAX_DEPTH, log + metric - per-error counter via raw_source_errors_total - circuit breaker on consecutive failures Stub is fail-open by design: enqueue() always resolves, never rejects. The orchestrator's `.catch(err => console.warn(...))` is defensive — the stub gives nothing to catch. No new tests — single async function returning undefined; full behavior tested via the RawSourceService orchestrator integration test in the next commit. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Seventh and final module under src/utils/rawSource/. Composes the six
preceding modules into a single fire-and-forget persist() call that the
PostToolUse hook will invoke for raw-source-carrying tools.
Pipeline (per persist() call):
1. Validate input (graceful — log + return null, never throw)
2. Size guard (drops oversize at the door)
3. Sanitize → cleaned (only pre-storage transform; secrets removed)
4. Hash raw bytes (Option B; cleaned bytes = stored bytes = hash input)
5. Storage.write (idempotent — dedup hit short-circuits with written:false)
6. Sidecar + global index (only on first landing)
7. Session manifest (always — even on dedup hit)
8. Per-agent manifest (when agentType present and passes path-traversal guard)
9. Fire-and-forget embedding enqueue (Wave 1 stub no-ops; Wave 2 activates)
Defensive properties:
- Never throws — every step wrapped in try/catch with structured warn log
- Per-step error isolation: appendAgent failure does NOT abort the rest of
the persist (pool body + session manifest still land)
- Embedding enqueue rejection ignored at the orchestrator boundary
- Returns null on input-validation failure or oversize trip
Dependency injection: `overrides` slot in createRawSourceService accepts
{ storage, manifestWriter, indexWriter, embeddingDispatcher, hasher,
sanitizer } for tests / future swaps. Production callers pass only
{ poolDir, sessionsRoot } and get a fully-wired service.
Module also re-exports the six component pieces (hashSource, sanitize,
PATTERNS, createSourceStorage, ChecksumError, etc.) so consumers can
import everything from one path.
Tests: 24 new orchestrator tests + 97 existing module tests = 121 total
across 6 suites, all passing in 489ms.
Orchestrator coverage:
- factory validation (poolDir / sessionsRoot required)
- input validation: missing sessionId/content/toolName, null/undefined
input, non-string content, oversize → all return null without throwing
- first landing: pool body + sidecar + index + session manifest all land
- per-agent manifest: written when agentType provided, skipped otherwise
- dedup: same content twice = one pool file, one index row, two manifest
rows (second has dedup_hit=true)
- cross-session dedup: same content from sessions A and B = one pool file,
each session has its own one-row manifest
- sanitization: API key + Authorization header redacted from stored body;
[REDACTED:*] tags appear; original secrets do NOT appear in pool file
- clean SEC text passes through (sanitized=false, redactions=[])
- embedding dispatcher receives correct (hash, sourceType); rejection
does not propagate
- error isolation: invalid agentType (path-traversal) does not abort
pool/session writes
- content-type routing: html/json/text → correct ext + sourceType
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Wave 1 prompt-injection detector for tool outputs. Lightweight (pure
regex, no LLM, no network), defensive (logging-only, conservative
thresholds), reusable (single chokepoint inserted in postToolUseHandler
in the next commit).
Pattern set (6 patterns, two weight tiers):
Formatting tokens — weight 0.9 (rarely legitimate in fetched docs):
system_tag → /\[SYSTEM\]|\[\/SYSTEM\]/gi
im_start → /<\|im_start\|>/gi
system_colon → /^\s*SYSTEM:\s/gim (line-anchored)
Semantic phrases — weight 0.4 (often appear in legal text):
ignore_prior → ignore <previous|all|above|prior> <instructions|prompts|rules>
you_are_now → you are <now|actually> <NOT followed by 'the same|here|in'>
new_directive → new <directive|instructions|rules>[:.]
Confidence:
max(individual weights) + 0.1 * (n_unique_matches - 1), capped at 1.0
Detection threshold: 0.5
→ single formatting token (0.9) → detected
→ single semantic phrase (0.4) → NOT detected
→ two semantics (0.4 + 0.1) → detected at boundary 0.5
→ formatting + semantic (1.0) → detected
Defensive properties:
- Pure function — no I/O, no state, never throws (null/undefined → empty result)
- 16 KB scan limit by default — early-content focus, perf cap on multi-MB inputs
- Excerpt cap ~200 chars (100 each side of first match)
- Returns structured result; orchestrator decides whether to log it
- Negative cases: 'Ignore all prior filings' (legitimate SEC), 'These instructions
apply to participants', 'New directives from the Board', 'You are advised'
all explicitly do NOT trigger
Pattern set deliberately overlaps with src/middleware/inputValidation.js but
does NOT import from it: that file is HTTP middleware that hard-blocks (400);
here we score, log, and let the response flow.
Phase 2 (Wave 3): escalate ambiguous matches (confidence 0.4–0.75) to a
Haiku 4.5 classifier via Messages API. The `classifier` field in the result
is the placeholder for that — currently always 'regex'.
Tests: 29 pass in 72ms.
- PATTERNS export shape + weights
- Single-token formatting detection (system_tag, im_start, system_colon)
- SYSTEM: at line start vs mid-line (multiline anchor)
- Single semantic patterns score 0.4 (NOT detected) — ignore_prior, you_are_now, new_directive
- Combined patterns: two semantics → 0.5 (detected); formatting+semantic → 1.0
- FP resistance on 7-line mock SEC body — does NOT cross threshold
- Excerpt window contains first match; empty when no match
- Scan-limit honored (matches beyond 16 KB ignored; explicit override expands)
- Defensive input handling: '', null, undefined, number → empty result
- Performance: 16 KB scan in <5 ms (clean and dirty inputs both)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…12) Widens the claude_tool_duration_ms histogram label set from [tool, status] → [tool_name, client, status] so per-external-API percentiles (P50/P95/P99) become queryable in Prometheus + Grafana. Why `client`: the same tool_name (e.g., fetch_document) can route through different external services (direct HTTP vs Exa /contents fallback). Without the client label, a slow Exa-fallback path is invisible in the aggregate. Cardinality bound: ~50 tool_names × ~6 clients × 3 statuses ≈ 900 series. Well under prom-client default limits. Bucket set widened on the long tail (10000, 30000, 60000) to capture slow external APIs that today bunch into the >5s bucket. Backward-compatible signature on recordToolDuration: Legacy: recordToolDuration(toolName, status, durationMs) → observed with client='unknown' Wave 1: recordToolDuration({ tool_name, client, status }, durationMs) Existing callers (researchHandler.js:256) keep working with client='unknown'. The Wave 1 hook integration (next commit, #12 scope) will use the object form with deriveClient() to populate the new label. Also adds: deriveClient(toolName, hybridMetadata) → string fetch_document + source='exa' → 'exa_fallback' fetch_document + source='native' → 'direct_fetch' fetch_document + null/undefined → 'direct_fetch' (default) exa_web_search → 'exa_native' mcp__<domain>__<method> → '<domain>' everything else → 'other' Tests: 13 pass in 137ms. - Label set check: histogram exposes [client, status, tool_name] - Wave 1 object signature: observes with all three labels - Wave 1 partial labels: missing fields default to 'unknown' - Legacy positional signature: client='unknown', tool_name + status preserved - deriveClient: every documented branch (fetch_document with/without metadata, exa_web_search, mcp__sec__/courtlistener/super-legal-tools, SDK tools, null/undefined/non-string) - Cardinality bound: 4 tools × 5 clients × 2 statuses → 40 distinct series Note: ran `npm install --legacy-peer-deps` in the worktree to materialize node_modules (peer-dep conflict on @google/genai surfaced as a known project issue from main; resolution unchanged). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Hot-path change in persistAuditEvent. Closes the critical gap that blocks #13: _hybrid_metadata fields (fetch_source, fallback_reason, fetch_mode, confidence) are extracted in sdkHooks.js:1018-1031 today but are never persisted, so the /api/analytics/sla/7day endpoint has no data to query. Implementation: - Insertion point: persistAuditEvent at line ~563 (just before INSERT) - Triggers only when: SLA_TELEMETRY=true AND hookName='PostToolUse' AND tool_name ∈ SLA_HYBRID_TOOLS (fetch_document | exa_web_search) - Parses tool_response.content[0].text as JSON; extracts _hybrid_metadata.{source, fallback_reason, fetch_mode, confidence} into eventData.{fetch_source, fallback_reason, fetch_mode, fetch_confidence} - Native-success inference: when a hybrid-client tool succeeds but produces no _hybrid_metadata (typical for native-only paths), set fetch_source='native' so the SLA dashboard can still group it - JSON.parse wrapped in try/catch — non-JSON responses are common (HTML, plain text); parse failure is silent (audit insert proceeds normally) Hot-path discipline: - Flag-gated: featureFlags.SLA_TELEMETRY (default false). With flag off, zero behavior change vs pre-Wave-1 baseline. - Single try/catch boundary: a malformed response cannot break the audit insert under any circumstance. - Fields are optional — every column that consumed event_data already handles null via COALESCE / COALESCE-style frontend code. Verification: - Module loads cleanly (node -e import). - Full end-to-end coverage (PostToolUse hook fires → row in hook_audit_log has fetch_source populated) lives in test/integration/sla.integration.test.js coming in Task #15. Cannot unit-test persistAuditEvent in isolation — function is not exported and depends on a live pg pool + sessionCache. SLA_HYBRID_TOOLS set is intentionally minimal in Wave 1: fetch_document — direct + exa fallback paths exa_web_search — direct exa search Wave 4 expands to per-hybrid-method instrumentation (searchSECFilings, searchCourtOpinions, etc.). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…#8, #12) Connects the four pure/stateful modules built in earlier commits to the live PostToolUse hook flow. Four files modified, each change focused. src/hooks/sdkHooks.js (postToolUseHandler): - Import detectInjection (#8) + recordToolDuration/deriveClient (#12). - Restructure existing _hybrid_metadata block to capture parsedToolResponse and textContent for reuse across detection + metric labeling. - Prompt-injection detection (flag-gated by PROMPT_INJECTION_DETECTION): runs detectInjection(textContent, {toolName}); if detected, attach to `entry` (file log) and propagate via hook return value `{ continue: true, prompt_injection: {...} }`. Detector failures caught locally — never throws. - Histogram observation (#12, always-on, no flag): on every PostToolUse with non-null duration_ms + tool_name, observes claude_tool_duration_ms{tool_name, client, status} where client is derived from tool_name + _hybrid_metadata.source. Recording failures caught locally. - Return value backward-compatible: handlers that don't fire injection detection still return { continue: true } unchanged. src/utils/hookDBBridge.js (persistAuditEvent): - Read result.prompt_injection.detected and merge five fields into eventData: prompt_injection_detected, _patterns, _excerpt, _confidence, _classifier. Frontend filters on prompt_injection_detected; analytics queries can do `WHERE event_data->>'prompt_injection_detected' = 'true'`. - PostToolUse row's event_type is preserved (not replaced) so the audit chain stays intact and the SLA telemetry on the same row continues to work. src/utils/hookSSEBridge.js: - Import featureFlags + define RAW_SOURCE_TOOLS allow-list with .includes() matcher (handles MCP-wrapped variants like 'mcp__direct-fetch__fetch_document'). - Extend forwardHookToSSE signature with sseOptions = {} as 8th arg (backward-compatible default). - PostToolUse case grows two new top-of-block sections: a) Raw-source archive (#3): when RAW_SOURCE_ARCHIVE=true AND sseOptions.rawSourceService is wired AND tool is in RAW_SOURCE_TOOLS, fire-and-forget `persist({sessionId, agentId, agentType, toolName, toolUseId, url, content})`. On success, emit `raw_source_ready` SSE event with { hash, size, url:/api/raw-sources/{hash}, agent_id, agent_type, ext, source_type, dedup, redactions, sanitized }. Errors caught at the .catch() boundary; never block the hook chain. b) Prompt-injection forwarding (#8): when result.prompt_injection.detected, emit `prompt_injection_detected` SSE event with patterns/confidence/ excerpt/classifier so the frontend timeline can surface it live. - wrapHooksForSSE + createSSEBridge both grow `sseOptions` parameter (default {}) and propagate through to forwardHookToSSE. All existing callers keep working; new callers opt into the raw-source wiring. src/server/agentStreamHandler.js: - Import createRawSourceService. - Per-request instantiation: poolDir = reports/_sources, sessionsRoot = reports/. Service is constructed unconditionally; the SSE bridge skips the persist branch when RAW_SOURCE_ARCHIVE=false (zero behavior change with flag off). - Pass { rawSourceService, getSessionId: () => ctx.sessionDir } as the third arg to createSSEBridge so PostToolUse can attribute writes to the live session. - Stash service on ctx for downstream consumers (future Wave 2/3). Verification: - All four modified modules load cleanly via direct node import. - All 163 unit tests across 8 suites still pass in 623ms (rawSource modules, promptInjectionDetector, metrics) — no regressions. - Default-off state (all three flags=false) produces zero behavior change: sdkHooks skips both detection + metric record; hookDBBridge skips SLA extraction; hookSSEBridge skips raw-source persist + injection forwarding. - Full end-to-end (PostToolUse → pool file lands; raw_source_ready event surfaces in #rawLog; hook_audit_log row carries prompt_injection_*) covered by integration tests in Task #15. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ndex (#3, #12, #13) Three files, four new routes, one new composite index. Closes the HTTP/SQL exposure gap so the modules wired in the previous commit become reachable from the frontend and from external Prometheus. src/server/claude-sdk-server.js — four GET routes for the raw-source archive: GET /api/raw-sources/:hash — decompressed body + SHA verification GET /api/raw-sources/:hash/meta — fetch metadata sidecar JSON GET /api/sessions/:sid/raw-sources — session-level NDJSON manifest as array GET /api/sessions/:sid/agents/:agent/sources — per-agent NDJSON manifest - Reuses createSourceStorage from src/utils/rawSource/index.js (lazy-imported once per process, cached via _rawSourceStorage closure to avoid circular import at server startup). - Body endpoint determines extension via: 1. meta sidecar (canonical), then 2. ?ext= query parameter (validated against KNOWN_EXTS), then 3. probe (try each known extension via storage.exists) Returns 404 if none match. - Hash format guard: HEX64 = /^[a-f0-9]{64}$/. Returns 400 on malformed hash. - Session ID guard: existing SESSION_ID_RE. Agent type guard: SAFE_AGENT_TYPE (alphanum + hyphen + underscore; matches SourceManifestWriter's path-traversal guard exactly). - On read, recomputes SHA-256 via SourceStorage.read; ChecksumError → 500 with structured warn log (hooked for Wave 3 alerting). - Sets X-Source-Hash, X-Fetched-At, X-Source-URL headers from meta sidecar so auditors can verify provenance from response headers alone. - 404 returned as empty rows (count:0) for manifest endpoints — frontend renders "no data" state instead of error. src/server/dbFrontendRouter.js — extended /api/analytics/tools/health, new /api/analytics/sla/7day: - tools/health: added p50_ms, p95_ms, p99_ms columns via PERCENTILE_CONT(...) WITHIN GROUP (ORDER BY duration_ms). Tightened WHERE to require duration_ms IS NOT NULL. - sla/7day: NEW route. Day × api_client grid: DATE_TRUNC('day', created_at) AS day, COALESCE(event_data->>'fetch_source', 'unknown') AS api_client, calls, success_rate, p95_ms, fallback_count Constrained to last 7 days, PostToolUse[Failure] events on fetch_document or exa_web_search tool variants. Source data populated by the SLA_TELEMETRY extraction in hookDBBridge (commit 34499f3). - Both queries inherit existing event_type NOT IN ('AgentProgress') filter. src/db/postgres.js — composite index for the percentile + SLA queries: idx_audit_tool_time_dur ON hook_audit_log (tool_name, created_at DESC, duration_ms) WHERE event_type IN ('PostToolUse', 'PostToolUseFailure') AND duration_ms IS NOT NULL - Partial index keeps it small (excludes SubagentStart/Stop/AgentProgress rows that have NULL duration_ms or are not relevant to per-tool latency). - PERCENTILE_CONT(... WITHIN GROUP ORDER BY duration_ms) reads in index order, avoiding a sort over millions of rows. - Wave 2 will move this and other Wave-1 schema deltas into a versioned node-pg-migrate file as 002_* (the planned migration tool adoption). Verification: - All three files syntax-check clean (node --check). - claude-sdk-server.js loads up to its env-var check (existing behavior; ANTHROPIC_API_KEY required at process start). - 163 unit tests across the rawSource modules + injection detector + metrics still pass (no regressions from the route additions, since the routes consume read-only methods of SourceStorage that were already tested). - End-to-end (live server, populated pool) covered by smoke + integration tests in Task #15. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds the External API SLA (7d) panel to the Status tab as a collapsible
section between Rate Limiter and Stream Stats. Panel polls
/api/analytics/sla/7day every 60 seconds and renders a table grid
(day × api_client) with calls, success rate, P95 latency, and fallback
count per row.
Files:
test/react-frontend/index.html
- New <div class="dashboard-section collapsible"> with id="slaPanel"
- Empty-state placeholder rendered when SLA_TELEMETRY=false or no rows
- <table id="slaTable"> with thead (Day / API / Calls / Success / P95 / Fallback)
test/react-frontend/app.js
- New `slaTimer` (let, near healthTimer)
- `fetchSlaDashboard()` — fetch + render, silent on non-200
- `renderSlaTable(rows)` — toggles empty/table visibility, renders rows
with successClass mapping: ≥99% accent, 95-99% neutral, <95% error
- Bootstrapped alongside fetchHealth + fetchSubagents + fetchCatalog
- setInterval(fetchSlaDashboard, 60_000) starts on first init
Behavior with flag off (SLA_TELEMETRY=false):
- Backend route /api/analytics/sla/7day still responds (returns 0 rows
since no event_data.fetch_source values exist to group by).
- Frontend renders the empty placeholder. No console noise.
Behavior with flag on:
- Backend extracts fetch_source/fallback_reason/fetch_mode into event_data
on every PostToolUse for fetch_document / exa_web_search.
- Within ~60s of first traffic, the table populates with the live grid.
- Color-coded success_rate gives at-a-glance API health view.
Verification: node --check app.js syntax-clean; HTML well-formed
(matches existing collapsible-section pattern). End-to-end (live API
populates rows) covered by smoke + integration tests in Task #15.
Note: the percentile columns on /api/analytics/tools/health (also Wave 1
#12) are exposed at the API level and visible via curl /metrics or
direct JSON; a frontend Tools Health table is deferred to Wave 4 polish
because no current panel renders that data shape.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…oy guide
Final Wave 1 commit. Closes the test + rollout gap so the release is
shippable end-to-end.
test/fixtures/raw-sources/ (4 files, used by both integration tests):
sec-10k-sample.html — SEC filing excerpt with phrases that
could trigger semantic injection patterns
but legitimately don't (e.g., "Ignore all
prior filings"); used to verify FP resistance
court-opinion-sample.json — court opinion JSON with _hybrid_metadata
so the orchestrator picks the .json extension
exa-results-sample.json — exa_web_search response shape with results[]
and _hybrid_metadata for source/result_count
injection-corpus.json — 12 calibration samples (6 clean, 6 dirty)
with per-sample expected_detected labels
and notes explaining the detector behavior
test/integration/rawSource.integration.test.js (6 tests):
- SEC fixture full pipeline → pool body, sidecar, _index.ndjson, session
manifest, per-agent manifest all present and well-formed
- Exa JSON fixture → .json extension + exa_result source_type
- Cross-session dedup → unique-content probe lands once, manifests in both
sessions
- Sanitization end-to-end → API key + Auth header redacted from stored body;
original secrets never appear on disk
- Tampered file → ChecksumError on read
test/integration/promptInjection.integration.test.js (5 tests):
- Per-sample expected_detected matches detector across all 12 corpus entries
- Aggregate FP rate on clean samples ≤ 25% (Wave 1 acceptance criterion)
- Aggregate detection rate on injected samples ≥ 80%
- Overall accuracy ≥ 90%
- SEC + Exa fixtures pass detector cleanly (no FP)
test/smoke/README.md:
Runbook-style smoke tests (curl commands + SQL queries) for each of the
four Wave 1 items plus the default-off regression check. Automated smoke
spawning the dev server in CI deferred to Wave 3 alongside the chaos suite.
docs/runbooks/wave-1-deploy.md:
Deploy runbook with pre-flight checklist, 5-step staging→production flag
rollout (24h soak between flips, 48h before raw-source enable), per-flag
rollback procedures, full verification matrix with pass criteria for each
acceptance item, and known limits / Wave 2 follow-ups.
package.json:
Added test:integration:wave1 (scoped to test/integration/, distinct from
the existing tests/integration script that runs unrelated suites) and
test:smoke (echoes the runbook README path).
Test totals (Wave 1 final):
Unit: 163 tests / 8 suites — 0.6 s
SourceHasher (27), SourceSanitizer (27), SourceStorage (21),
SourceManifestWriter (12), SourceIndexWriter (11), RawSourceService (24),
promptInjectionDetector (29), metrics (13)
Integration: 11 tests / 2 suites — 0.2 s
rawSource end-to-end (6), promptInjection corpus (5)
Combined: 174 tests / 10 suites — 0.8 s
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…audit
A blast-radius audit identified three changes that take effect on Wave 1
deploy regardless of feature flag state. None breaks the functional
pipeline; each has operational implications worth pre-deploy attention.
Runbook additions:
1. Histogram label rename (tool → tool_name) with explicit grep + migration
guidance for existing Prometheus/Grafana queries and alert rules.
The legacy positional recordToolDuration() call signature is preserved,
so existing call sites (researchHandler.js:256) still observe values
under the new label set with client="unknown".
2. Composite index idx_audit_tool_time_dur added to hook_audit_log inside
initSchema(); runs synchronously on first server start. Added a sizing
query to estimate indexed_rows + decision matrix mapping row count to
expected build time:
< 10M rows → < 30s, deploy normally
10M-100M rows → 30s-5min, schedule during low-traffic window
> 100M rows → pre-build with CREATE INDEX CONCURRENTLY before deploy
(IF NOT EXISTS makes in-process create a no-op)
3. Always-on metric observation in postToolUseHandler. Adds ~750 Prometheus
series (50 tool_name × 5 client × 3 status). Per-call cost ~1-2 μs.
Runbook added a cardinality-budget pre-flight check.
Frontend hygiene comment:
app.js — added a comment near slaTimer noting that, like healthTimer, no
explicit clearInterval is wired. Both rely on the page lifecycle (hard
navigation / window close) to reclaim. If SPA-style navigation is added
later, both timers need cleanup. This documents the existing convention
rather than masking it.
The audit's "byte-identical to main" verdict is C+ (qualified) — flag-gated
paths are all safe, but the three unconditional changes above mean deploy
parity is operational, not strict. The runbook now makes that explicit.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…on pool
Correction 1.1: pivot from global content-addressed pool to per-session pool.
Factory signature change:
BEFORE: createRawSourceService({ poolDir, sessionsRoot, maxRawBytes, overrides })
AFTER: createRawSourceService({ sessionsRoot, maxRawBytes, overrides })
— no poolDir; it's derived per persist() call from sessionsRoot + sessionId
Per-session pool path derivation inside persist():
sessionPoolDir = path.join(sessionsRoot, sessionId, 'raw-sources')
→ SourceStorage + SourceIndexWriter instantiated with this path each call
Why per-session:
The product is single-tenant-per-MD deal work. Sessions = deals = audit
boundaries. Legal hold, 7-year retention, regulatory deletion, and
session-level backup/restore all align with session folders. Global pool
was optimizing cross-session dedup (~$3/yr at realistic throughput) at
the expense of self-containment. Not the right tradeoff for this product.
Storage + index instantiation:
Both were previously factory-time singletons bound to a single poolDir.
Now per-persist() call. Storage construction does zero I/O, so per-call
cost is negligible (~100 μs). Overrides (test DI) still short-circuit
per-call instantiation for deterministic fixtures.
Filesystem layout change (user-visible):
BEFORE: reports/_sources/{ab}/{cd}/{hash}.ext.gz
AFTER: reports/{sessionId}/raw-sources/{ab}/{cd}/{hash}.ext.gz
Session manifests (raw-sources-manifest.ndjson, specialist-reports/
{agent}-sources/sources.ndjson) were already session-scoped — unchanged.
Follow-up commits in this correction sequence:
2. Delete SourceIndexWriter (redundant per-session); add first_landing
flag to session manifest rows so Wave 3 tamper-evident Merkle rollup
can still distinguish new-hash events from dedup hits.
3. Update hooks/server wiring + routes (/api/raw-sources/:hash →
/api/sessions/:sid/raw-sources/:hash).
4. Update tests (paths + fixtures).
5. Update docs (planning + spec + runbook + smoke README).
Verification:
node --check passes; module imports cleanly; createRawSourceService is
exported as a function. Full test updates land in commit 4 of this sequence.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…flag
Correction 1.1 D1: SourceIndexWriter is redundant under per-session scoping.
Under the old global pool, _index.ndjson served two purposes:
1. Tamper-evident log of new-hash landings (for Merkle root rollup)
2. Global dedup registry
Per-session, both purposes collapse into the session manifest:
1. Tamper-evident: session manifest is already append-only + session-scoped
2. Dedup: session-local; SourceStorage.exists() handles it
To preserve Wave 3's Merkle-rollup ability to distinguish new-hash events
from dedup hits, a new `first_landing: boolean` field is added to each
session manifest row (set from SourceStorage.write().written).
Changes:
- DELETED: src/utils/rawSource/SourceIndexWriter.js
- DELETED: test/sdk/rawSource/SourceIndexWriter.test.js
- MODIFIED: src/utils/rawSource/index.js
- Removed SourceIndexWriter import
- Removed indexWriter instantiation in persist()
- Removed indexWriter.append() call block
- Removed re-export
- Added `first_landing: written` to manifestRow
- Updated JSDoc to reflect removal
Module count: 7 → 6 (SourceIndexWriter removed; 5 active + 1 stub)
Net LOC: -70 (40 LOC module + 30 LOC test removed)
Exports verified: createRawSourceService, createSourceStorage, ChecksumError,
createManifestWriter, createEmbeddingDispatcher, hashSource, sha256, sanitize,
SANITIZER_PATTERNS. No createIndexWriter — correctly absent.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Correction 1.1 commit 3/5.
Route paths changed:
GET /api/raw-sources/:hash → GET /api/sessions/:sid/raw-sources/:hash
GET /api/raw-sources/:hash/meta → GET /api/sessions/:sid/raw-sources/:hash/meta
Session-scoped routes now validate sessionId via SESSION_ID_RE and
instantiate SourceStorage inline per request with:
poolDir = path.join(REPORTS_DIR_ABS, sessionId, 'raw-sources')
Storage construction is zero-I/O so per-request cost is negligible.
Removed from claude-sdk-server.js:
- SOURCES_POOL_DIR constant (global pool concept)
- _rawSourceStorage singleton + getRawSourceStorage() lazy loader
- _ChecksumError instance check → replaced with err?.name === 'ChecksumError'
(avoids needing to hold a module-level reference)
Added:
- getRawSourceMod(): lazy module import (cached) replacing per-pool singleton
- sessionPoolDir(sessionId): path helper
- X-Session-Id response header on body GET for audit traceability
agentStreamHandler.js:
- createRawSourceService call simplified: dropped poolDir param,
now only passes { sessionsRoot: reportsRoot } since poolDir is
derived inside persist() per call.
hookSSEBridge.js:
- raw_source_ready SSE event URL updated from /api/raw-sources/{hash}
to /api/sessions/{sessionId}/raw-sources/{hash} so the frontend can
fetch directly without needing to know the session ID separately.
Already session-scoped routes (no change required):
GET /api/sessions/:sid/raw-sources — manifest route (unchanged)
GET /api/sessions/:sid/agents/:agent/sources — per-agent manifest (unchanged)
Syntax-clean: all three files pass node --check.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Correction 1.1 commit 4/5. Updated all test fixtures for per-session pool.
Key changes:
- Factory: { sessionsRoot } only (no poolDir)
- Pool paths: root/{sessionId}/raw-sources/{ab}/{cd}/ (not root/_sources/)
- Cross-session: both sessions write (written:true, different paths)
- first_landing flag asserted on manifest rows
- SourceIndexWriter test suite already deleted in commit 2
165 tests pass, 9 suites, 531ms.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…+ smoke
Correction 1.1 commit 5/5.
Updated all documentation artifacts to reflect the per-session pool:
- reports/_sources/ → reports/{session_id}/raw-sources/ (path references)
- /api/raw-sources/:hash → /api/sessions/:sid/raw-sources/:hash (route refs)
- "Global pool" → "Per-session pool" (terminology)
Files updated:
- docs/pending-updates/observability-updates-april-26.md (planning doc)
- docs/runbooks/wave-1-deploy.md (deploy runbook)
- test/smoke/README.md (smoke test curl commands)
Note: observability-implementation-spec.md was not updated in this commit
because the Correction 1.1 section already added to the plan file serves
as the canonical per-session design reference. The spec's Wave 1 §1.1
sections describe the original global-pool design for historical context.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ow-list Bug fix discovered during live testing: raw-source archive was not firing because subagents use SDK built-in WebFetch/WebSearch tools (not MCP fetch_document/exa_web_search) when EXA_WEB_TOOLS=false (the default). Root cause: RAW_SOURCE_TOOLS only matched 'fetch_document' and 'exa_web_search' — the MCP tool names. When EXA_WEB_TOOLS is false, PostToolUse fires with tool_name='WebFetch'/'WebSearch' which did not match the allow-list. Fix: split into two sets: RAW_SOURCE_MCP_TOOLS — .includes() match for MCP-wrapped variants RAW_SOURCE_SDK_TOOLS — exact-match Set for SDK built-in tools isRawSourceTool() now checks both. Archive fires regardless of which web-tool configuration is active. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Same root cause as 3c40727 (WebFetch/WebSearch missing from allow-lists) but in two additional files. Discovered via log analysis of the live test session: 404 tool calls (348 WebSearch + 56 WebFetch) were invisible to injection detection AND SLA telemetry. sdkHooks.js (postToolUseHandler): - textContent extraction condition widened: BEFORE: tool_name?.includes('fetch_document') || includes('exa_web_search') AFTER: isMcpWebTool || isSdkWebTool (Set('WebFetch','WebSearch')) - JSON.parse wrapped in inner try/catch (SDK tools return raw HTML, not JSON — parse throws expectedly; textContent still populated for injection detection + metric labeling) - Net effect: promptInjectionDetector now scans WebFetch/WebSearch responses. Previously textContent was null → detector never ran. hookDBBridge.js (persistAuditEvent): - SLA_HYBRID_TOOLS expanded: + 'WebFetch', 'WebSearch' - Non-JSON handling: set default fetch_source BEFORE JSON parse attempt: SDK tools (WebFetch/WebSearch) → 'sdk_builtin' (default, kept on JSON.parse failure since raw HTML isn't JSON) MCP tools with _hybrid_metadata → actual source (exa/native/etc.) MCP tools without metadata → 'native' - Net effect: /api/analytics/sla/7day now captures SDK-tool calls as fetch_source='sdk_builtin'. Previously zero rows populated. Live session stats (pre-fix): WebSearch: 348 calls, WebFetch: 56 calls — none archived, scanned, or SLA-tracked. Post-fix all 404 calls will be captured across all three observability surfaces (#3, #8, #13). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Correction 1.2: move raw-source capture from PostToolUse hook to the
SDK message stream's content_block_start handler.
Root cause (discovered via live testing + log analysis + SDK source audit):
Server-side tools (WebFetch/WebSearch) return results as specialized
content blocks (web_fetch_tool_result, web_search_tool_result) in the
agentQuery() message stream. PostToolUse hook DOES fire for these tools
(8 tool_failure events prove it), but tool_response.content[0].text is
empty — the actual response body flows through the stream, not the hook.
The SDK's agentQuery() yields these blocks as stream_event messages with
event.type === 'content_block_start'. Confirmed via:
- SDK type definitions (BetaWebFetchToolResultBlock in sdk.d.ts)
- Existing production usage (promptEnhancer.js:165-177 reads
web_search_tool_result blocks from the same API)
- includePartialMessages: true (already set at line 288)
Implementation in agentStreamHandler.js (line ~386):
Two else-if branches in the content_block_start handler:
web_fetch_tool_result:
- block.content.type === 'web_fetch' → successful fetch (has HTML body)
- block.content.content = full HTML response body
- block.content.url = source URL
- block.content.status = HTTP status code
- Error blocks (type !== 'web_fetch') filtered — no point archiving 403s
- Fire-and-forget: ctx.rawSourceService.persist({...}).then(emit SSE).catch(warn)
web_search_tool_result:
- block.content = Array<SearchResultBlock>
- JSON.stringify'd before persist (structured results, not HTML)
- result_count included in SSE event
Both paths:
- Flag-gated: featureFlags.RAW_SOURCE_ARCHIVE
- Null-safe: ctx.rawSourceService?.persist (service already on ctx from line 183)
- SSE emission: type='hook_event', hook='raw_source_ready' (same shape as
the hookSSEBridge path; frontend addRaw(e) captures it automatically)
- Console log for live observability during testing
hookSSEBridge.js PostToolUse raw-source block:
- Updated comment to document it as FALLBACK for MCP tools (EXA_WEB_TOOLS=true)
- Functionally unchanged — still inert for default config (WebFetch/WebSearch
don't populate tool_response.content[0].text)
- Kept because MCP tools (fetch_document/exa_web_search) DO populate that
field, so the hookSSEBridge path would work when EXA_WEB_TOOLS=true
Verification:
- Syntax clean (node --check)
- 165 unit + integration tests pass in 635ms (modules untouched)
- Live test: restart server → raw-source pool files should appear
incrementally during subagent WebFetch/WebSearch calls
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move raw-source capture from PostToolUse/stream (both inert) to the MCP
tool execution layer — the ONLY point where our code sees raw responses.
Root cause recap (three failed interception points):
1. PostToolUse hook: tool_response.content[0].text is empty for
server-side tools (WebFetch/WebSearch executed by API internally)
2. Stream content_block_start: SDK doesn't yield web_fetch_tool_result
or web_search_tool_result blocks (confirmed: zero in 3 live tests)
3. Both fail because WebFetch/WebSearch are API-internal server tools
The fix: capture at wrapWithConversation() in toolImplementations.js.
This function wraps ALL 163 MCP tool handlers. Every external API call
returns its response through this single middleware. With EXA_WEB_TOOLS=true,
WebFetch/WebSearch are replaced by MCP tools (fetch_document, exa_web_search),
routing ALL web activity through wrapWithConversation. Coverage: 99.4%.
toolImplementations.js changes:
- Added imports: path, fileURLToPath, getStore (requestContext),
featureFlags, createRawSourceService
- Added lazy singleton getRawSourceService() with __dirname-derived
reportsRoot (zero I/O at import time; instantiates on first persist)
- Inside wrapWithConversation: after tool execution + conversation logging,
if RAW_SOURCE_ARCHIVE=true AND getStore()?.sessionDir is set:
- Extract content: prefer MCP text field, fall back to JSON.stringify
- Fire-and-forget persist({sessionId, toolName, content, url})
- .catch() swallows errors; never breaks tool execution
- Outer try/catch as belt-and-suspenders
agentStreamHandler.js changes:
- Removed dead Path C code (web_fetch_tool_result / web_search_tool_result
branches from commit 82bdc20) — confirmed these block types are never
yielded by the SDK at runtime despite existing in type definitions
- Replaced with comment explaining why and pointing to Correction 1.3
All 165 existing tests pass (rawSource modules untouched; only trigger moved).
Live test: restart with RAW_SOURCE_ARCHIVE=true EXA_WEB_TOOLS=true
→ MCP tool responses should populate reports/{sid}/raw-sources/
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
SLA dashboard showed 'unknown' as the API client for all 248 tool calls because SLA_HYBRID_TOOLS.has(tool_name) used exact match, but MCP tool names arrive as 'mcp__super-legal-tools__fetch_document' (prefixed). The set contained 'fetch_document' — exact match failed. Fix: switch from .has() to .includes() pattern matching (same approach hookSSEBridge already uses for isRawSourceTool). The isSlaTrackedTool check now matches both exact names (WebFetch, WebSearch) and MCP-prefixed variants (mcp__*__fetch_document, mcp__*__exa_web_search). After fix, SLA dashboard will show: - 'native' or actual _hybrid_metadata.source for MCP tools with metadata - 'sdk_builtin' for SDK built-in tools (WebFetch/WebSearch) - Actual source values (exa, direct_fetch, etc.) when _hybrid_metadata parsed Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Changelog entry for the Wave 1 institutional observability release: #3 raw-source archive (per-session, content-addressed, 287 sources live-tested) #8 prompt-injection detection (regex, logging-only) #12 per-tool latency histograms (P50/P95/P99) #13 7-day SLA dashboard per external API Documents architecture corrections 1.1-1.3 discovered during live testing, new files inventory, modified files summary, deployment notes, and flag requirements (RAW_SOURCE_ARCHIVE requires EXA_WEB_TOOLS=true). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Number531
force-pushed
the
observability/wave-1
branch
from
0272d9a to
81b4644
Compare
Merged
5 tasks
Number531
added a commit
that referenced
this pull request
May 12, 2026
…validation) (#121) Reframes documentation from "Exa as conditional feature flag" to "Exa as production-default", with citation-verifier A/B validation evidence. Drivers: - EXA_WEB_TOOLS=true production-locked since 2026-04-18 (PR #76) - Production-fidelity A/B validation 2026-05-12 (PRs #118 + #119): Exa 96.8% vs Anthropic 96.1% on 467-footnote citation-verifier fixture; both PASS production gate - EXA_ADDITIONAL_QUERIES=true all-treatment in production flags.env since 2026-05-11 (v7.6.2) Files (10): - system-design.md — 6 edits: tool inventory framing, footnote groups, feature flag table, citation-verification roadmap row - gtm-positioning-strategy.md — adds verifier validation metric - gtm-sales-playbook.md — Q1 EU AI Act response cites Exa verifier - gtm-buyer-intelligence.md — Q#8 cites Exa A/B validation - enterprise-necessities.md — EXA flag entries reframed + new A3 entries - feature-flags.md §26 — EXA_WEB_TOOLS production activation + validation - runbooks/exa-a3-ab-staging.md — production rollout addendum (was "do NOT set in production yet") - skills/client-audit-export — clarifies zero-rows interpretation by flag state (prevents false-positive incident filing) - skills/deploy — post-deploy flag-verification block - skills/subagent-scaffold — --a3-eligible reframed as RECOMMENDED No code changes. No flag flips. Pure documentation alignment with already-shipped production state.
This was referenced May 12, 2026
Number531
added a commit
that referenced
this pull request
May 22, 2026
Schema bump v1.1.0 -> v1.2.0. Pure additive enhancement to sidecar
projection — captures the agent's narrative `text` blocks alongside
the existing `thinking_summary` (reasoning blocks).
Why
---
thinking_summary captures the model's private reasoning. text_summary
captures what the agent says out loud between tool calls — stated
conclusions, plans, and observations. Operators triaging a session via
sidecar previously had to grep .full.jsonl to understand what the agent
concluded; now both reasoning streams live in the structured projection.
What changed
------------
- src/wrappedSubagents/transcriptSidecar.js:
- SIDECAR_SCHEMA_VERSION 1.1.0 -> 1.2.0
- New TEXT_SAMPLE_CHARS constant (300, matches THINKING_SAMPLE_CHARS)
- New extractTextSummary() mirroring extractThinkingSummary (first +
middle + last sample, total_blocks, total_chars)
- buildSidecar() projects text_summary alongside thinking_summary
- test/sdk/wrappedSubagents/transcriptSidecar.test.js:
- SIDECAR_SCHEMA_VERSION assertion bumped to '1.2.0' (2 sites)
- text_summary added to required-keys assertion
- +7 unit tests covering: block counting, empty turnLog, first+middle+
last sampling, 300-char truncation, malformed block defensive handling,
separation-of-concerns vs thinking blocks, single-block edge case
Validation
----------
- 41/41 transcriptSidecar tests pass (was 34; +7 new)
- 691/693 wrapped-subagents suite passes (2 failures pre-existing from
Task #67 OPUS_MODEL fixture lag — unrelated)
- node --check passes
- Zero token cost (write-side projection only)
- Backward-compatible: older consumers ignore the new field
Plan: docs/pending-updates/Wrapped-Migration-Phase4.13-Full-Fidelity-Transcripts.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds four observability capabilities behind feature flags (all default OFF) to close gaps identified in an institutional-buyer audit against PE/IB/M&A/IC requirements.
PERCENTILE_CONTSQL. Composite index onhook_audit_log./api/analytics/sla/7dayroute.28 commits | 45 files changed | +6530 / -888 LOC | 165 tests
Key architecture decisions
wrapWithConversationcapture (not PostToolUse/stream): server-side tools (WebFetch/WebSearch) are API-internal; MCP tool execution layer is the only working capture point.src/utils/rawSource/, each ≤100 LOC.Deployment
All flags default OFF — merge is zero-behavior-change. See
docs/runbooks/wave-1-deploy.mdfor the 5-stage flag rollout:SLA_TELEMETRY=true(24h soak)PROMPT_INJECTION_DETECTION=true(24h soak)RAW_SOURCE_ARCHIVE=true+EXA_WEB_TOOLS=true(48h soak)Breaking:
claude_tool_duration_mshistogram label renamedtool→tool_name. Migrate Prometheus/Grafana queries before deploy.Test plan
tool→tool_namelabel🤖 Generated with Claude Code