diff --git a/docs/branch-review-ledger.md b/docs/branch-review-ledger.md index 83f0e40eb..5c30d165b 100644 --- a/docs/branch-review-ledger.md +++ b/docs/branch-review-ledger.md @@ -392,11 +392,14 @@ Use this ledger to prevent repeated branch and PR reviews when the reviewed HEAD | 2026-07-13 | origin/codex/rag-canary-completion | d1b7647bde73e7d0558472ad6aa19c3b49d94639 | branch-cleanup | Retained for open PR #612 and active worktree activity. | Fresh GitHub open-PR query and moving-head observation; deletion prohibited. | | 2026-07-13 | origin/main | 528a1752f41cd29a518ca9341c93c724030173ae | branch-cleanup | Protected base branch retained. | Final refreshed origin/main snapshot before the ledger PR. | | 2026-07-14 | PR #629 / codex/eval-canary-quota-handling | 8fe1be6d0c58c60afeeb82f720b03c90ce57c2cf | review-followup | One P2 structured-error retry defect confirmed and fixed on `codex/eval-canary-structured-errors`. | GitHub connector thread inspection; `tests/eval-utils.test.ts` 14/14; focused ESLint and Prettier. | +| 2026-07-14 | codex/global-answer-reliability | 01d2749978d4e8f3b4ac508e068ec7b8d331f816 | release-readiness / answer generation, source governance, progress, and document access | No unresolved P0-P2 findings after fixing demo-path governance parity, preserving feedback-token binding during the `origin/main` rebase, and binding legacy-summary feedback to the delivered governed answer. Ready for hosted PR checks. | `npm run verify:pr-local` (2,223 passed, 1 skipped; production build and offline RAG evaluation passed); focused Chromium answer regressions (4/4 passed); focused answer and route regressions (124/124 passed). | | 2026-07-14 | claude/repo-next-steps-e53523 | 3f7d6d76f597f2a0311af1052480f84b68ecc259 | launch-readiness and RAG-performance follow-up | Branch changes were already squash-merged by PR #513. Fresh review found one P1 governance inconsistency (Singapore app/worker processing omitted from the PIA's cross-border account) and three P2 follow-ups: degraded-answer SLO overcounting, a second blocking shared-cache miss diagnostic query, and stale launch/RAG backlog status. No source fix was applied in this review. | `git diff --check c3828ceb9f3812abeebc1b653361fc254dda9f5e..3f7d6d76f597f2a0311af1052480f84b68ecc259`; focused Vitest 92/92; `npm run eval:rag:offline` 36 fixtures and 59/59 contract tests; `npm run verify:cheap` passed runtime, action pins, sitemap, type scale, lint, typecheck, and 1,672 passed/1 skipped tests. Provider-backed Supabase/OpenAI, browser, release, drift, and live retrieval-quality checks were not run. | | 2026-07-14 | codex/rag-performance-followups | 5502fd498ea2069f810795a8659f98ab3abf8c80 | release-readiness review | The local release review found no P0-P2 defect after correcting one stale PIA statement; the later hosted review follow-up is recorded below. The scoped RAG round-trip, SLO, retention, privacy, and documentation changes were ready for PR handoff. Highest residual risk is the known live hybrid-RPC latency tail; model experiments remain blocked by provider quota and legal execution remains operator/counsel work. | Rebased onto `origin/main`; runtime and full Prettier check; ESLint; TypeScript; Vitest 2,213 passed/1 skipped; Next.js production build (636 pages) plus client-bundle secret scan; offline RAG 36 fixtures and 277/277 contract tests; `git diff --check`. Live retention jobs 13/16 were already verified during this workstream. | | 2026-07-14 | PR #632 / codex/rag-performance-followups | aa264e92c44b42fdcceeac6292011ba51169b862 | review-followup | One P2 SLO-classification defect was confirmed: the broad source-only `degraded` flag also included healthy extractive answers. Fixed by persisting and counting a separate `provider_generation_degraded` flag derived only from `generation_fallback`. | GitHub connector plus UTF-8 thread-aware review inspection; focused ESLint; 45/45 targeted Vitest tests; TypeScript; offline RAG 36 fixtures and 277/277 contract tests; `git diff --check`. | | 2026-07-14 | PR #632 / codex/rag-performance-followups | 07cac8f4e094bf4b5ccd264feb6420fbd4e35e2b | review-followup | CodeRabbit's APP 1/5 governance contradiction was confirmed and corrected; Railway's pending DPA and Singapore processor basis are now explicit checklist/status items. The optional cache-miss granularity suggestion was not adopted because it would undo the reviewed single-read cold-miss design or require a broader schema/RPC change. | GitHub review-thread inspection; focused Prettier and `git diff --check` passed. Local production-readiness inspection reported the expected missing provider env; the Railway production-context check had already passed on this workstream. | | 2026-07-14 | PR #632 / codex/rag-performance-followups | 0c792b8c10516c85d18832fc96562dd70a671b5f | review-followup | One P1 privacy-disclosure defect was confirmed: OpenAI retrieval embedding can receive query text even when answer synthesis remains source-only. Fixed the PIA and public `/privacy` copy to distinguish retrieval-embedding egress from model-backed synthesis egress, with focused copy coverage. | GitHub connector review-thread inspection; focused privacy UI test, ESLint, TypeScript, Prettier, production-readiness, and `git diff --check`. | | 2026-07-14 | PR #632 / codex/rag-performance-followups | a2eb6db0efbef983e1b3242261d5cc6b2b9d839d | review-followup | One late P2 rollout-compatibility defect was confirmed: the provider-fallback SLO would omit recent rows written before `provider_generation_degraded` existed. Fixed the count predicate to include the new flag or legacy `generation_fallback:` reasons while continuing to exclude intentional extractive routes. | GitHub connector review-thread inspection; focused answer-SLO test, ESLint, TypeScript, Prettier, and `git diff --check`. | +| 2026-07-14 | PR #634 / codex/global-answer-reliability | 588c34c3d455c367e6aa38dc9fce64191678631b | review-followup | One late P2 quota-bypass defect was confirmed: streamed full-document summaries consumed only the general answer quota. Fixed the route to enforce the stricter `document_summarize` quota before starting the stream or provider work while retaining the general answer ceiling. | GitHub connector review-thread inspection; focused private-access route suite 116/116; ESLint; TypeScript; Prettier; `git diff --check`. | | 2026-07-14 | codex/specifiers-design | c3e7024e15d2157bbc0b989276324cc6ad3eea5c | Specifiers UI, clinical decision-support, accessibility, and release-readiness review | One P2 WCAG contrast defect was reproduced and fixed across the builder, comparison, and map accent eyebrows. No remaining high-confidence defect was found in the changed scope; residual risk is clinical/manual governance of the original specifier summaries. | Focused Vitest 18/18; focused Chromium desktop/mobile 2/2 with serious/critical axe WCAG A/AA scanning and overflow checks; `npm run check:production-readiness:ci` READY; `npm run verify:cheap` 2,210 passed/1 skipped; `npm run verify:pr-local` 2,213 passed/1 skipped plus production build and client-bundle secret scan; `git diff --check`. Full advisory `verify:ui` exceeded the local 10-minute execution window; provider-backed checks were not run. | | 2026-07-14 | PR #633 / codex/specifiers-design | 03daca3bfdf86517b9956f3c7d91be27d9f1a751 | Review follow-up and failed UI regression | Eight distinct P2 behaviors across nine review threads were confirmed and fixed: initial selection normalization, duplicate query removal, clinical applicability mapping, canonical Specifiers routing, severity-neutral base wording, diagnostic-section ordering, base/specifier compatibility, and severe psychotic-features wording. The failed app-menu keyboard test was a stale order assertion and now covers the inserted Specifiers item. | Focused Vitest 6/6; focused Chromium 4/4; scoped ESLint and TypeScript; CI-mode production readiness READY; production build and client-bundle secret scan passed. Hosted required checks passed on the refreshed `main` merge before the final compatibility fixes; provider-backed Supabase/OpenAI checks were not run. | +| 2026-07-14 | PR #634 / codex/global-answer-reliability | b411329ec5f181661e5d49276c398440aa928fa2 | review-followup | One late P2 fast-context defect was confirmed: Australian tier ordering could push a higher-ranked supplementary passage outside the four-chunk routine fast budget. Fixed by preserving the retrieval-ranked, crowding-capped candidate budget before applying the order-only Australian preference within that set. | GitHub connector review-thread inspection; focused RAG context-budget suite 22/22; ESLint; TypeScript; Prettier; `git diff --check`. | diff --git a/src/app/api/answer/route.ts b/src/app/api/answer/route.ts index 4cdb7610d..de3cb580c 100644 --- a/src/app/api/answer/route.ts +++ b/src/app/api/answer/route.ts @@ -16,20 +16,19 @@ import { buildSmartRagApiPlan } from "@/lib/smart-rag-api"; import { clinicalQueryModeSchema, queryClassForClinicalMode, queryForClinicalMode } from "@/lib/clinical-query-mode"; import { resolveSearchScope, searchScopeFiltersSchema } from "@/lib/search-scope"; import { resolveRetrievalAccessScope } from "@/lib/owner-scope"; -import { - hasDangerSourceGovernanceWarning, - sourceGovernanceRefusalAnswer, - sourceGovernanceWarnings, -} from "@/lib/source-governance"; +import { sourceGovernanceWarnings } from "@/lib/source-governance"; import { parseJsonBody } from "@/lib/validation/body"; -import { toClientAnswerPayload } from "@/lib/answer-client-payload"; +import { + answerDegradedModeSignal, + buildGovernedAnswerClientResponse, + buildGovernedDemoAnswerClientResponse, +} from "@/lib/answer-response"; import { answerServerTimingEntries, buildServerTimingHeader } from "@/lib/server-timing"; import { createAdminClient } from "@/lib/supabase/admin"; import { captureServerException } from "@/lib/observability/error-capture"; import { logAnswerDiagnostics } from "@/lib/answer-telemetry"; import { nonProductionSupabaseDemoFallbackReason } from "@/lib/supabase/errors"; import * as serverAuth from "@/lib/supabase/auth"; -import type { RagAnswer } from "@/lib/types"; import { answerFeedbackMetadata } from "@/lib/answer-feedback-token"; export const runtime = "nodejs"; @@ -46,15 +45,6 @@ type AnswerRequestBody = z.infer; const emptyScopeAnswer = "The selected filters did not match any indexed documents, so I cannot generate an answer for that scope."; -function answerDegradedModeSignal(answer?: Pick) { - if (answer?.degradedMode) return answer.degradedMode; - const active = answer?.answerQualityTier === "source_only"; - return { - active, - reason: active ? (answer?.fallbackReason ?? "source_only") : null, - }; -} - function buildDemoAnswerPayload(body: AnswerRequestBody, fallbackReason?: string) { const answer = demoAnswer(body.query, body.documentId, body.documentIds); const answerFocusQuery = queryForClinicalMode(body.query, body.queryMode); @@ -65,14 +55,14 @@ function buildDemoAnswerPayload(body: AnswerRequestBody, fallbackReason?: string routeMode: answer.routingMode, retrievalStrategy: "hybrid", }); - return { - ...answer, - responseMode: smartApiPlan.displayMode, - smartApiPlan, - demoMode: true, - degradedMode: fallbackReason ? { active: true, reason: fallbackReason } : answerDegradedModeSignal(answer), - ...(fallbackReason ? { fallbackMode: "non_production_demo", fallbackReason } : {}), - }; + return buildGovernedDemoAnswerClientResponse( + { + ...answer, + responseMode: smartApiPlan.displayMode, + smartApiPlan, + }, + fallbackReason, + ); } export async function POST(request: Request) { @@ -137,45 +127,13 @@ export async function POST(request: Request) { queryMode: answerBody.queryMode, signal: request.signal, }); - const warnings = sourceGovernanceWarnings({ - results: answer.sources ?? [], - relevance: answer.relevance ?? answer.smartPanel?.relevance ?? null, + const governedResponse = buildGovernedAnswerClientResponse(answer); + logAnswerDiagnostics({ + supabase, + query: answerBody.query, + ownerId: access.ownerId, + answer: governedResponse.telemetryAnswer, }); - const shouldUseSourceGovernanceRefusal = - answer.grounded !== false && answer.confidence !== "unsupported" && answer.responseMode !== "evidence_gap"; - if (shouldUseSourceGovernanceRefusal && hasDangerSourceGovernanceWarning(warnings)) { - // Build the refusal explicitly — never spread ...answer here, or the original - // (refused) sources/smartPanel/smartApiPlan would still reach the client and - // defeat the refusal. Keep only the safe "unsupported" contract fields, matching - // the empty-scope branch above. - void logAnswerDiagnostics({ - supabase, - query: answerBody.query, - ownerId: access.ownerId, - answer: { - ...answer, - grounded: false, - confidence: "unsupported", - sources: [], - responseMode: "evidence_gap", - fallbackReason: "source_governance_refusal", - routingReason: [answer.routingReason, "source_governance_refusal"].filter(Boolean).join("; "), - }, - }); - return NextResponse.json({ - answer: sourceGovernanceRefusalAnswer, - grounded: false, - confidence: "unsupported", - citations: [], - sources: [], - degradedMode: answerDegradedModeSignal(answer), - scope: { ...scope, queryMode: answerBody.queryMode }, - sourceGovernanceWarnings: warnings, - ...answerFeedbackMetadata(interactionId, sourceGovernanceRefusalAnswer), - }); - } - - logAnswerDiagnostics({ supabase, query: answerBody.query, ownerId: access.ownerId, answer }); // Durations only — see server-timing.ts for the trust-boundary constraint. const serverTiming = buildServerTimingHeader( @@ -183,13 +141,9 @@ export async function POST(request: Request) { ); return NextResponse.json( { - // Boundary trim only — governance warnings and diagnostics above - // consumed the full answer (see answer-client-payload.ts). - ...toClientAnswerPayload(answer), - degradedMode: answerDegradedModeSignal(answer), + ...governedResponse.payload, scope: { ...scope, queryMode: answerBody.queryMode }, - sourceGovernanceWarnings: warnings, - ...answerFeedbackMetadata(interactionId, answer.answer), + ...answerFeedbackMetadata(interactionId, governedResponse.payload.answer), }, serverTiming ? { headers: { "Server-Timing": serverTiming } } : undefined, ); diff --git a/src/app/api/answer/stream/route.ts b/src/app/api/answer/stream/route.ts index 6bf8ff972..f5a86d491 100644 --- a/src/app/api/answer/stream/route.ts +++ b/src/app/api/answer/stream/route.ts @@ -1,6 +1,6 @@ import { randomUUID } from "node:crypto"; import { z } from "zod"; -import { demoAnswer } from "@/lib/demo-data"; +import { demoAnswer, demoSummary } from "@/lib/demo-data"; import { isDemoMode } from "@/lib/env"; import { PublicApiError, jsonError } from "@/lib/http"; import { @@ -10,19 +10,19 @@ import { type ApiRateLimitResult, } from "@/lib/api-rate-limit"; import { publicAccessContext } from "@/lib/public-api-access"; -import { toClientAnswerPayload } from "@/lib/answer-client-payload"; -import { answerQuestionWithScope, type AnswerProgressEvent } from "@/lib/rag"; +import { + answerDegradedModeSignal, + buildGovernedAnswerClientResponse, + buildGovernedDemoAnswerClientResponse, +} from "@/lib/answer-response"; +import { answerQuestionWithScope, summarizeDocument, type AnswerProgressEvent } from "@/lib/rag"; import { classifyRagQuery } from "@/lib/clinical-search"; import { annotateSearchResults, buildEvidenceRelevance } from "@/lib/evidence-relevance"; import { buildSmartRagApiPlan } from "@/lib/smart-rag-api"; import { clinicalQueryModeSchema, queryClassForClinicalMode, queryForClinicalMode } from "@/lib/clinical-query-mode"; import { resolveSearchScope, searchScopeFiltersSchema } from "@/lib/search-scope"; import { resolveRetrievalAccessScope, type RetrievalAccessScope } from "@/lib/owner-scope"; -import { - hasDangerSourceGovernanceWarning, - sourceGovernanceRefusalAnswer, - sourceGovernanceWarnings, -} from "@/lib/source-governance"; +import { sourceGovernanceWarnings } from "@/lib/source-governance"; import { createAdminClient } from "@/lib/supabase/admin"; import { captureServerException } from "@/lib/observability/error-capture"; import { logAnswerDiagnostics } from "@/lib/answer-telemetry"; @@ -33,32 +33,33 @@ import { safeErrorLogDetails } from "@/lib/privacy"; import { startSseHeartbeat } from "@/lib/sse-heartbeat"; import { parseJsonBody } from "@/lib/validation/body"; import { toPublicAnswerProgressEvent } from "@/lib/answer-progress-public"; -import type { RagAnswer } from "@/lib/types"; import { answerFeedbackMetadata } from "@/lib/answer-feedback-token"; export const runtime = "nodejs"; -const answerSchema = z.object({ - query: z.string().trim().min(1).max(2000), - documentId: z.string().uuid().optional(), - documentIds: z.array(z.string().uuid()).max(25).optional(), - filters: searchScopeFiltersSchema.optional(), - queryMode: clinicalQueryModeSchema.optional().default("auto"), -}); +const answerSchema = z + .object({ + query: z.string().trim().min(1).max(2000), + documentId: z.string().uuid().optional(), + documentIds: z.array(z.string().uuid()).max(25).optional(), + filters: searchScopeFiltersSchema.optional(), + queryMode: clinicalQueryModeSchema.optional().default("auto"), + summaryMode: z.boolean().optional().default(false), + }) + .superRefine((value, context) => { + if (value.summaryMode && !value.documentId) { + context.addIssue({ + code: "custom", + path: ["documentId"], + message: "Document summary mode requires a document id.", + }); + } + }); type AnswerBody = z.infer; const emptyScopeAnswer = "The selected filters did not match any indexed documents, so I cannot generate an answer for that scope."; -function answerDegradedModeSignal(answer?: Pick) { - if (answer?.degradedMode) return answer.degradedMode; - const active = answer?.answerQualityTier === "source_only"; - return { - active, - reason: active ? (answer?.fallbackReason ?? "source_only") : null, - }; -} - function encodeSse(event: string, data: unknown) { return `event: ${event}\ndata: ${JSON.stringify(data)}\n\n`; } @@ -67,6 +68,10 @@ function rateLimitStream(rateLimit: ApiRateLimitResult) { return rateLimitJsonResponse("Too many answer requests. Retry shortly.", rateLimit); } +function documentSummaryRateLimitStream(rateLimit: ApiRateLimitResult) { + return rateLimitJsonResponse("Too many document summary requests. Retry shortly.", rateLimit); +} + function streamErrorPayload(error: unknown) { if (error instanceof PublicApiError) { return { @@ -116,26 +121,29 @@ function logStreamError(error: unknown, signal?: AbortSignal) { } function buildDemoStreamAnswer(body: AnswerBody, fallbackReason?: string) { - const demo = demoAnswer(body.query, body.documentId, body.documentIds); + const demo = + body.summaryMode && body.documentId + ? demoSummary(body.documentId) + : demoAnswer(body.query, body.documentId, body.documentIds); const answerFocusQuery = queryForClinicalMode(body.query, body.queryMode); const sources = annotateSearchResults(answerFocusQuery, demo.sources); const relevance = buildEvidenceRelevance(answerFocusQuery, sources); - return { - ...demo, - sources, - relevance, - smartPanel: demo.smartPanel ? { ...demo.smartPanel, relevance } : demo.smartPanel, - smartApiPlan: buildSmartRagApiPlan({ - query: answerFocusQuery, - queryClass: queryClassForClinicalMode(body.queryMode) ?? classifyRagQuery(answerFocusQuery).queryClass, - results: sources, - routeMode: demo.routingMode, - retrievalStrategy: "hybrid", - }), - demoMode: true, - degradedMode: fallbackReason ? { active: true, reason: fallbackReason } : answerDegradedModeSignal(demo), - ...(fallbackReason ? { fallbackMode: "non_production_demo", fallbackReason } : {}), - }; + return buildGovernedDemoAnswerClientResponse( + { + ...demo, + sources, + relevance, + smartPanel: demo.smartPanel ? { ...demo.smartPanel, relevance } : demo.smartPanel, + smartApiPlan: buildSmartRagApiPlan({ + query: answerFocusQuery, + queryClass: queryClassForClinicalMode(body.queryMode) ?? classifyRagQuery(answerFocusQuery).queryClass, + results: sources, + routeMode: demo.routingMode, + retrievalStrategy: "hybrid", + }), + }, + fallbackReason, + ); } function streamAnswer(body: AnswerBody, accessScope: RetrievalAccessScope, signal?: AbortSignal) { @@ -204,77 +212,49 @@ function streamAnswer(body: AnswerBody, accessScope: RetrievalAccessScope, signa }); return; } + if (isDemoMode()) { + sendFinal({ ...buildDemoStreamAnswer(body), interactionId }); + return; + } + const singleDocumentScope = Boolean( body.documentId && !body.documentIds?.length && scope?.activeFilterCount === 0, ); - const answer = isDemoMode() - ? buildDemoStreamAnswer(body) - : await answerQuestionWithScope({ - query: body.query, - documentId: singleDocumentScope ? body.documentId : undefined, - documentIds: singleDocumentScope - ? undefined - : (scope?.documentIds ?? body.documentIds ?? (body.documentId ? [body.documentId] : undefined)), - ownerId, - accessScope, - allowGlobalSearch: !ownerId, - queryMode: body.queryMode, - onProgress, - onToken, - onRevising, - signal, - }); - const warnings = sourceGovernanceWarnings({ - results: answer.sources ?? [], - relevance: answer.relevance ?? answer.smartPanel?.relevance ?? null, - }); - const shouldUseSourceGovernanceRefusal = - answer.grounded !== false && answer.confidence !== "unsupported" && answer.responseMode !== "evidence_gap"; - if (shouldUseSourceGovernanceRefusal && hasDangerSourceGovernanceWarning(warnings)) { - // Explicit refusal payload — do not spread ...answer (see /api/answer): - // the refused sources/smartPanel/smartApiPlan must not reach the client. - if (!isDemoMode()) { - void logAnswerDiagnostics({ - supabase: createAdminClient(), - query: body.query, - ownerId, - answer: { - ...answer, - grounded: false, - confidence: "unsupported", - sources: [], - responseMode: "evidence_gap", - fallbackReason: "source_governance_refusal", - routingReason: [answer.routingReason, "source_governance_refusal"].filter(Boolean).join("; "), - }, - }); - } - sendFinal({ - answer: sourceGovernanceRefusalAnswer, - grounded: false, - confidence: "unsupported", - citations: [], - sources: [], - degradedMode: answerDegradedModeSignal(answer), - scope: scope ? { ...scope, queryMode: body.queryMode } : undefined, - sourceGovernanceWarnings: warnings, - ...streamAnswerFeedbackMetadata(interactionId, sourceGovernanceRefusalAnswer), - }); - return; + if (body.summaryMode) { + sendProgress({ stage: "analyzing", message: "Reading the committed document sections." }); + sendProgress({ stage: "generating", message: "Building the governed clinical summary." }); } + const answer = + body.summaryMode && body.documentId + ? await summarizeDocument(body.documentId, ownerId, { signal }) + : await answerQuestionWithScope({ + query: body.query, + documentId: singleDocumentScope ? body.documentId : undefined, + documentIds: singleDocumentScope + ? undefined + : (scope?.documentIds ?? body.documentIds ?? (body.documentId ? [body.documentId] : undefined)), + ownerId, + accessScope, + allowGlobalSearch: !ownerId, + queryMode: body.queryMode, + onProgress, + onToken, + onRevising, + signal, + }); + const governedResponse = buildGovernedAnswerClientResponse(answer); - if (!isDemoMode()) { - logAnswerDiagnostics({ supabase: createAdminClient(), query: body.query, ownerId, answer }); - } + logAnswerDiagnostics({ + supabase: createAdminClient(), + query: body.query, + ownerId, + answer: governedResponse.telemetryAnswer, + }); sendFinal({ - // Boundary trim only — governance warnings and diagnostics above - // consumed the full answer (see answer-client-payload.ts). - ...toClientAnswerPayload(answer), - degradedMode: answerDegradedModeSignal(answer), + ...governedResponse.payload, scope: scope ? { ...scope, queryMode: body.queryMode } : undefined, - sourceGovernanceWarnings: warnings, - ...streamAnswerFeedbackMetadata(interactionId, answer.answer), + ...streamAnswerFeedbackMetadata(interactionId, governedResponse.payload.answer), }); } catch (error) { // Parity with /api/answer (PR #315): outside production, a misconfigured @@ -326,6 +306,18 @@ export async function POST(request: Request) { }); if (rateLimit.limited) return rateLimitStream(rateLimit); + if (body.summaryMode) { + // Streamed full-document summaries use the same paid provider path as the + // legacy summary endpoint. Preserve the general answer ceiling, then also + // enforce the stricter summary quota before the SSE stream can start. + const summaryRateLimit = await consumeSubjectApiRateLimit({ + supabase, + subject: access.rateLimitSubject, + bucket: "document_summarize", + }); + if (summaryRateLimit.limited) return documentSummaryRateLimitStream(summaryRateLimit); + } + return streamAnswer(body, resolveRetrievalAccessScope(access.ownerId), request.signal); } catch (error) { if (error instanceof AuthenticationError) { diff --git a/src/app/api/documents/[id]/summarize/route.ts b/src/app/api/documents/[id]/summarize/route.ts index 0c6e2594b..1a5fb8219 100644 --- a/src/app/api/documents/[id]/summarize/route.ts +++ b/src/app/api/documents/[id]/summarize/route.ts @@ -1,8 +1,13 @@ +import { randomUUID } from "node:crypto"; import { NextResponse } from "next/server"; import { z } from "zod"; import { demoSummary, getDemoDocument } from "@/lib/demo-data"; import { isDemoMode } from "@/lib/env"; +import { documentSummaryQuestion } from "@/lib/answer-contract"; import { summarizeDocument } from "@/lib/rag"; +import { buildGovernedAnswerClientResponse, buildGovernedDemoAnswerClientResponse } from "@/lib/answer-response"; +import { logAnswerDiagnostics } from "@/lib/answer-telemetry"; +import { answerFeedbackMetadata } from "@/lib/answer-feedback-token"; import { jsonError } from "@/lib/http"; import { consumeApiRateLimit, rateLimitJsonResponse } from "@/lib/api-rate-limit"; import { createAdminClient } from "@/lib/supabase/admin"; @@ -23,7 +28,10 @@ export async function POST(request: Request, { params }: { params: Promise<{ id: if (!getDemoDocument(id)) { return NextResponse.json({ error: "Demo document not found." }, { status: 404 }); } - return NextResponse.json({ ...demoSummary(id), demoMode: true }); + return NextResponse.json({ + ...buildGovernedDemoAnswerClientResponse(demoSummary(id)), + interactionId: randomUUID(), + }); } const supabase = createAdminClient(); @@ -31,7 +39,19 @@ export async function POST(request: Request, { params }: { params: Promise<{ id: const rateLimit = await consumeApiRateLimit({ supabase, ownerId: user.id, bucket: "document_summarize" }); if (rateLimit.limited) return rateLimitJsonResponse("Too many document summary requests. Retry shortly.", rateLimit); - return NextResponse.json(await summarizeDocument(id, user.id)); + const answer = await summarizeDocument(id, user.id); + const governedResponse = buildGovernedAnswerClientResponse(answer); + logAnswerDiagnostics({ + supabase, + query: documentSummaryQuestion, + ownerId: user.id, + answer: governedResponse.telemetryAnswer, + }); + const interactionId = randomUUID(); + return NextResponse.json({ + ...governedResponse.payload, + ...answerFeedbackMetadata(interactionId, governedResponse.payload.answer), + }); } catch (error) { if (error instanceof AuthenticationError) { return unauthorizedResponse(); diff --git a/src/components/ClinicalDashboard.tsx b/src/components/ClinicalDashboard.tsx index 59556b204..6b0151e04 100644 --- a/src/components/ClinicalDashboard.tsx +++ b/src/components/ClinicalDashboard.tsx @@ -75,7 +75,6 @@ import { } from "@/components/clinical-dashboard/answer-content"; import { AnswerEmptyState, AnswerProgressStepper, AnswerSkeleton } from "@/components/clinical-dashboard/answer-status"; import { - normalizeAnswerProgressEvent, type AnswerProgressUpdate, type TimedAnswerProgressUpdate, } from "@/components/clinical-dashboard/answer-progress"; @@ -136,13 +135,11 @@ import { answerPayloadIsUsable, classifyAnswerError, createAnswerRequestWatchdog, - isAnswerPayload, isRetryableError, - isRetryableMessage, - isRetryableStatus, keywordQueryFromNaturalLanguage, makeSearchError, progressForRetry, + readAnswerStream, searchRetryCount, searchRetryDelaysMs, sleep, @@ -328,140 +325,11 @@ function hasNonProductionSupabaseApiKeyFallback(checks: SetupCheck[]) { ); } -function parseSseData(lines: string[]) { - const data = lines.join("\n").trim(); - if (!data) return null; - try { - return JSON.parse(data); - } catch { - throw makeSearchError("Answer stream returned malformed data.", 500, true); - } -} - /** True when an error originates from an AbortController (user pressed Stop / component unmounted). */ function isAbortError(error: unknown): boolean { return error instanceof DOMException && error.name === "AbortError"; } -function findSseSeparator(buffer: string) { - const match = /\r?\n\r?\n/.exec(buffer); - return match ? { index: match.index, length: match[0].length } : null; -} - -async function readAnswerStream( - response: Response, - onProgress: (progress: AnswerProgressUpdate) => void, - onToken?: (delta: string) => void, - onRevising?: () => void, - onActivity?: () => void, -): Promise { - if (!response.body) throw makeSearchError("Answer stream could not be opened.", undefined, true); - - const reader = response.body.getReader(); - const decoder = new TextDecoder(); - let buffer = ""; - let pendingCompletion: AnswerProgressUpdate | null = null; - - function processEvent(block: string) { - const lines = block.split(/\r?\n/); - let event = "message"; - const dataLines: string[] = []; - - for (const line of lines) { - if (line.startsWith("event:")) event = line.slice("event:".length).trim(); - if (line.startsWith("data:")) dataLines.push(line.slice("data:".length).trimStart()); - } - - if (dataLines.length === 0) return; - const data = parseSseData(dataLines); - if (data === null) return; - if (event === "progress") { - const progress = normalizeAnswerProgressEvent(data); - if (progress) { - if (progress.stage === "complete") { - pendingCompletion = progress; - } else { - onProgress(progress); - if (progress.stage === "fallback") onRevising?.(); - } - } - return; - } - if (event === "token") { - const delta = data && typeof data === "object" ? (data as { delta?: unknown }).delta : null; - if (typeof delta === "string" && delta) onToken?.(delta); - return; - } - if (event === "revising") { - onRevising?.(); - return; - } - if (event === "error") { - pendingCompletion = null; - const message = data && typeof data === "object" ? (data as { error?: unknown }).error : null; - const details = - data && typeof data === "object" ? (data as { details?: { message?: unknown } | unknown }).details : null; - const detailMessage = - details && typeof details === "object" && "message" in details && typeof details.message === "string" - ? details.message - : null; - const status = data && typeof data === "object" ? (data as { status?: unknown }).status : null; - const statusCode = typeof status === "number" ? status : undefined; - const errorMessage = - typeof message === "string" && message.trim() - ? message - : typeof detailMessage === "string" && detailMessage.trim() - ? detailMessage - : "Answer generation failed due to a streaming error."; - throw makeSearchError( - errorMessage, - statusCode, - isRetryableStatus(statusCode ?? 0) || isRetryableMessage(errorMessage), - ); - } - if (event === "final") { - if (!isAnswerPayload(data)) { - pendingCompletion = null; - throw makeSearchError("Answer stream returned an invalid final payload.", 502, true); - } - if (pendingCompletion) { - onProgress(pendingCompletion); - pendingCompletion = null; - } - return data; - } - - return null; - } - - while (true) { - const { value, done } = await reader.read(); - // Any received bytes — progress events, token deltas, or server heartbeat - // comments — count as liveness for the caller's stall watchdog. - if (value && value.length > 0) onActivity?.(); - buffer += decoder.decode(value, { stream: !done }); - - let separator = findSseSeparator(buffer); - while (separator) { - const block = buffer.slice(0, separator.index).trim(); - buffer = buffer.slice(separator.index + separator.length); - const finalPayload = block ? processEvent(block) : null; - if (finalPayload) { - await reader.cancel().catch(() => undefined); - return finalPayload; - } - separator = findSseSeparator(buffer); - } - - if (done) break; - } - - const finalPayload = buffer.trim() ? processEvent(buffer.trim()) : null; - if (finalPayload) return finalPayload; - pendingCompletion = null; - throw makeSearchError("Answer stream ended before a final answer was received.", undefined, true); -} - // Provisional view shown while an answer streams in. The prose is content-preserving (the same // text the final payload will carry); the caret conveys that generation is still in flight. On a // quality-gate escalation the pipeline sends a `revising` signal and this switches to a neutral diff --git a/src/components/DocumentViewer.tsx b/src/components/DocumentViewer.tsx index 60cda1b3b..49863ae35 100644 --- a/src/components/DocumentViewer.tsx +++ b/src/components/DocumentViewer.tsx @@ -44,6 +44,9 @@ import { documentTileTone, } from "@/components/clinical-dashboard/document-ui"; import { useHideOnScroll } from "@/components/clinical-dashboard/use-hide-on-scroll"; +import { AnswerProgressStepper } from "@/components/clinical-dashboard/answer-status"; +import type { TimedAnswerProgressUpdate } from "@/components/clinical-dashboard/answer-progress"; +import { readAnswerStream } from "@/components/clinical-dashboard/search-utils"; import { DocumentTagCloud } from "@/components/DocumentTagCloud"; import type { PDFDocumentLoadingTask, PDFDocumentProxy, RenderTask } from "pdfjs-dist"; import { @@ -97,6 +100,7 @@ import { type FormattedDocumentSummary as FormattedDocumentSummaryModel, } from "@/lib/document-summary-formatting"; import { buildDocumentSummaryBadges } from "@/lib/document-summary-badges"; +import { documentSummaryQuestion } from "@/lib/answer-contract"; type PageRow = { id: string; @@ -2022,6 +2026,9 @@ export function DocumentViewer({ const [signedUrl, setSignedUrl] = useState(null); const [downloadSignedUrl, setDownloadSignedUrl] = useState(null); const [summary, setSummary] = useState(null); + const [summaryQuery, setSummaryQuery] = useState(documentSummaryQuestion); + const [summaryProgressEvents, setSummaryProgressEvents] = useState([]); + const [summaryProgressStartedAt, setSummaryProgressStartedAt] = useState(null); const [loadingDocument, setLoadingDocument] = useState(true); const [viewerError, setViewerError] = useState(null); const [previewError, setPreviewError] = useState(null); @@ -2065,6 +2072,13 @@ export function DocumentViewer({ ); const [viewerModeInitialized] = useState(true); const generatedSummaryRef = useRef(null); + const summaryAbortRef = useRef(null); + useEffect( + () => () => { + summaryAbortRef.current?.abort(); + }, + [], + ); const { status: authStatus, isConfigured, @@ -2404,27 +2418,86 @@ export function DocumentViewer({ setSummaryError("Sign in before summarising private documents."); return; } + const summaryMode = sourceSearch.trim().length === 0; + const query = summaryMode ? documentSummaryQuestion : sourceSearch.trim(); + const controller = new AbortController(); + summaryAbortRef.current?.abort(); + summaryAbortRef.current = controller; + const authRequest = registerAuthRequest(controller); + const startedAt = Date.now(); setLoadingSummary(true); + setSummary(null); + setSummaryQuery(query); setSummaryError(null); + setSummaryProgressStartedAt(startedAt); + setSummaryProgressEvents([ + { + stage: "scoping", + message: "Preparing the clinical search scope.", + receivedAt: startedAt, + }, + ]); try { - const response = await fetch(`/api/documents/${documentId}/summarize`, { + if (!isAuthEpochCurrent(authRequest.epoch)) { + throw new DOMException("Stale authentication epoch", "AbortError"); + } + const response = await fetch("/api/answer/stream", { method: "POST", - headers: clientDemoMode ? undefined : authorizationHeader, + headers: { + "Content-Type": "application/json", + ...(clientDemoMode ? {} : authorizationHeader), + }, + body: JSON.stringify({ query, documentId, ...(summaryMode ? { summaryMode: true } : {}) }), + signal: controller.signal, }); - const payload = await response.json(); if (response.status === 401) markSessionExpired(); - if (!response.ok) throw new Error(payload.error || "Summary could not be generated."); - setSummary({ ...payload, answer: cleanClinicalSummaryText(String(payload.answer ?? "")) }); + if (!response.ok) { + const payload = (await response.json().catch(() => null)) as { error?: unknown } | null; + throw new Error( + typeof payload?.error === "string" && payload.error.trim() + ? payload.error + : "Answer could not be generated from this document.", + ); + } + const payload = await readAnswerStream(response, (progress) => { + if ( + controller.signal.aborted || + summaryAbortRef.current !== controller || + !isAuthEpochCurrent(authRequest.epoch) + ) + return; + setSummaryProgressEvents((events) => [...events, { ...progress, receivedAt: Date.now() }].slice(-20)); + }); + if (controller.signal.aborted || summaryAbortRef.current !== controller || !isAuthEpochCurrent(authRequest.epoch)) + return; + setSummary(payload); window.requestAnimationFrame(() => { generatedSummaryRef.current?.scrollIntoView({ block: "start", behavior: "smooth" }); }); } catch (error) { - setSummaryError(error instanceof Error ? error.message : "Summary could not be generated."); + if (error instanceof DOMException && error.name === "AbortError") return; + if (controller.signal.aborted || summaryAbortRef.current !== controller || !isAuthEpochCurrent(authRequest.epoch)) + return; + setSummaryProgressEvents([]); + setSummaryProgressStartedAt(null); + setSummaryError(error instanceof Error ? error.message : "Answer could not be generated from this document."); } finally { - setLoadingSummary(false); + authRequest.release(); + if (summaryAbortRef.current === controller) { + summaryAbortRef.current = null; + setLoadingSummary(false); + } } } + function stopSummary() { + summaryAbortRef.current?.abort(); + summaryAbortRef.current = null; + setLoadingSummary(false); + setSummaryProgressEvents([]); + setSummaryProgressStartedAt(null); + } + const authViewerError = !canUsePrivateApis && !clientDemoMode && @@ -2469,6 +2542,7 @@ export function DocumentViewer({ const selectedChunk = chunkId ? chunks.find((chunk) => chunk.id === chunkId) : undefined; const { clinicalImages, auditImages } = partitionViewerImages(images); const generatedSummaryText = summary ? cleanClinicalSummaryText(summary.answer) : ""; + const generatedAnswerIsSummary = summaryQuery === documentSummaryQuestion; const storedSummaryText = document?.summary?.summary ?? null; const documentLabels = document?.labels; const formattedStoredSummary = useMemo(() => formatDocumentSummary(storedSummaryText), [storedSummaryText]); @@ -2710,8 +2784,16 @@ export function DocumentViewer({ ) : null}
- {(summary || summaryError) && ( + {(loadingSummary || summary || summaryError) && (
+ {summaryProgressStartedAt && summaryProgressEvents.length > 0 ? ( + + ) : null} {summary && (

diff --git a/src/components/clinical-dashboard/search-utils.ts b/src/components/clinical-dashboard/search-utils.ts index f15800731..cf7325a62 100644 --- a/src/components/clinical-dashboard/search-utils.ts +++ b/src/components/clinical-dashboard/search-utils.ts @@ -1,3 +1,7 @@ +import { + normalizeAnswerProgressEvent, + type AnswerProgressUpdate, +} from "@/components/clinical-dashboard/answer-progress"; import type { RagAnswer } from "@/lib/types"; export { keywordQueryFromNaturalLanguage } from "@/lib/keyword-query"; @@ -36,6 +40,134 @@ export function makeSearchError(message: string, status?: number, retryable = fa return error; } +function parseSseData(lines: string[]) { + const data = lines.join("\n").trim(); + if (!data) return null; + try { + return JSON.parse(data); + } catch { + throw makeSearchError("Answer stream returned malformed data.", 500, true); + } +} + +function findSseSeparator(buffer: string) { + const match = /\r?\n\r?\n/.exec(buffer); + return match ? { index: match.index, length: match[0].length } : null; +} + +/** Consume the answer SSE contract shared by every browser answer surface. */ +export async function readAnswerStream( + response: Response, + onProgress: (progress: AnswerProgressUpdate) => void, + onToken?: (delta: string) => void, + onRevising?: () => void, + onActivity?: () => void, +): Promise { + if (!response.body) throw makeSearchError("Answer stream could not be opened.", undefined, true); + + const reader = response.body.getReader(); + const decoder = new TextDecoder(); + let buffer = ""; + let pendingCompletion: AnswerProgressUpdate | null = null; + + function processEvent(block: string) { + const lines = block.split(/\r?\n/); + let event = "message"; + const dataLines: string[] = []; + + for (const line of lines) { + if (line.startsWith("event:")) event = line.slice("event:".length).trim(); + if (line.startsWith("data:")) dataLines.push(line.slice("data:".length).trimStart()); + } + + if (dataLines.length === 0) return null; + const data = parseSseData(dataLines); + if (data === null) return null; + if (event === "progress") { + const progress = normalizeAnswerProgressEvent(data); + if (progress) { + if (progress.stage === "complete") { + pendingCompletion = progress; + } else { + onProgress(progress); + if (progress.stage === "fallback") onRevising?.(); + } + } + return null; + } + if (event === "token") { + const delta = data && typeof data === "object" ? (data as { delta?: unknown }).delta : null; + if (typeof delta === "string" && delta) onToken?.(delta); + return null; + } + if (event === "revising") { + onRevising?.(); + return null; + } + if (event === "error") { + pendingCompletion = null; + const message = data && typeof data === "object" ? (data as { error?: unknown }).error : null; + const details = + data && typeof data === "object" ? (data as { details?: { message?: unknown } | unknown }).details : null; + const detailMessage = + details && typeof details === "object" && "message" in details && typeof details.message === "string" + ? details.message + : null; + const status = data && typeof data === "object" ? (data as { status?: unknown }).status : null; + const statusCode = typeof status === "number" ? status : undefined; + const errorMessage = + typeof message === "string" && message.trim() + ? message + : typeof detailMessage === "string" && detailMessage.trim() + ? detailMessage + : "Answer generation failed due to a streaming error."; + throw makeSearchError( + errorMessage, + statusCode, + isRetryableStatus(statusCode ?? 0) || isRetryableMessage(errorMessage), + ); + } + if (event === "final") { + if (!isAnswerPayload(data)) { + pendingCompletion = null; + throw makeSearchError("Answer stream returned an invalid final payload.", 502, true); + } + if (pendingCompletion) { + onProgress(pendingCompletion); + pendingCompletion = null; + } + return data; + } + + return null; + } + + while (true) { + const { value, done } = await reader.read(); + if (value && value.length > 0) onActivity?.(); + buffer += decoder.decode(value, { stream: !done }); + + let separator = findSseSeparator(buffer); + while (separator) { + const block = buffer.slice(0, separator.index).trim(); + buffer = buffer.slice(separator.index + separator.length); + const finalPayload = block ? processEvent(block) : null; + if (finalPayload) { + await reader.cancel().catch(() => undefined); + return finalPayload; + } + separator = findSseSeparator(buffer); + } + + if (done) break; + } + + const finalPayload = buffer.trim() ? processEvent(buffer.trim()) : null; + if (finalPayload) return finalPayload; + pendingCompletion = null; + throw makeSearchError("Answer stream ended before a final answer was received.", undefined, true); +} + export function isRetryableStatus(status: number) { return status === 408 || status === 429 || (status >= 500 && status <= 599); } diff --git a/src/lib/answer-contract.ts b/src/lib/answer-contract.ts new file mode 100644 index 000000000..1bc1707a4 --- /dev/null +++ b/src/lib/answer-contract.ts @@ -0,0 +1 @@ +export const documentSummaryQuestion = "Summarize this document for practical psychiatric use."; diff --git a/src/lib/answer-response.ts b/src/lib/answer-response.ts new file mode 100644 index 000000000..7245b69b1 --- /dev/null +++ b/src/lib/answer-response.ts @@ -0,0 +1,83 @@ +import { toClientAnswerPayload } from "@/lib/answer-client-payload"; +import { + hasDangerSourceGovernanceWarning, + sourceGovernanceRefusalAnswer, + sourceGovernanceWarnings, +} from "@/lib/source-governance"; +import type { RagAnswer } from "@/lib/types"; + +export function answerDegradedModeSignal( + answer?: Pick, +) { + if (answer?.degradedMode) return answer.degradedMode; + const active = answer?.answerQualityTier === "source_only"; + return { + active, + reason: active ? (answer?.fallbackReason ?? "source_only") : null, + }; +} + +/** Apply the shared browser-boundary source-governance contract. */ +export function buildGovernedAnswerClientResponse(answer: RagAnswer) { + const warnings = sourceGovernanceWarnings({ + results: answer.sources ?? [], + relevance: answer.relevance ?? answer.smartPanel?.relevance ?? null, + }); + const shouldRefuse = + answer.grounded !== false && + answer.confidence !== "unsupported" && + answer.responseMode !== "evidence_gap" && + hasDangerSourceGovernanceWarning(warnings); + + if (shouldRefuse) { + const routingReason = [answer.routingReason, "source_governance_refusal"].filter(Boolean).join("; "); + const telemetryAnswer = { + ...answer, + answer: sourceGovernanceRefusalAnswer, + grounded: false, + confidence: "unsupported", + citations: [], + sources: [], + responseMode: "evidence_gap", + fallbackReason: "source_governance_refusal", + routingReason, + } satisfies RagAnswer; + + return { + refused: true as const, + warnings, + telemetryAnswer, + payload: { + answer: sourceGovernanceRefusalAnswer, + grounded: false as const, + confidence: "unsupported" as const, + citations: [], + sources: [], + degradedMode: answerDegradedModeSignal(answer), + sourceGovernanceWarnings: warnings, + }, + }; + } + + return { + refused: false as const, + warnings, + telemetryAnswer: answer, + payload: { + ...toClientAnswerPayload(answer), + degradedMode: answerDegradedModeSignal(answer), + sourceGovernanceWarnings: warnings, + }, + }; +} + +/** Apply the governed browser contract while preserving explicit demo/degraded state. */ +export function buildGovernedDemoAnswerClientResponse(answer: RagAnswer, fallbackReason?: string) { + const governedResponse = buildGovernedAnswerClientResponse(answer); + return { + ...governedResponse.payload, + demoMode: true as const, + degradedMode: fallbackReason ? { active: true, reason: fallbackReason } : answerDegradedModeSignal(answer), + ...(fallbackReason ? { fallbackMode: "non_production_demo" as const, fallbackReason } : {}), + }; +} diff --git a/src/lib/australian-source-priority.ts b/src/lib/australian-source-priority.ts index 7ae107c9c..9eba99f6d 100644 --- a/src/lib/australian-source-priority.ts +++ b/src/lib/australian-source-priority.ts @@ -80,11 +80,17 @@ function isSupplementaryPadding(args: { */ export function selectAustralianClinicalContext( results: SearchResult[], - options: { limit?: number; maxPerDocument?: number; sufficientAustralianChunks?: number } = {}, + options: { + limit?: number; + maxPerDocument?: number; + sufficientAustralianChunks?: number; + omitSupplementaryPadding?: boolean; + } = {}, ) { const limit = options.limit ?? 6; const maxPerDocument = options.maxPerDocument ?? 2; const sufficientAustralianChunks = options.sufficientAustralianChunks ?? 4; + const omitSupplementaryPadding = options.omitSupplementaryPadding ?? true; const ranked = results .map((result, index) => ({ result, index, tier: australianSourceTier(result) })) .filter(({ result }) => result.relevance?.verdict !== "none") @@ -93,9 +99,9 @@ export function selectAustralianClinicalContext( const rightRelevance = resultRelevanceRank(right.result); return leftRelevance - rightRelevance || tierRank[left.tier] - tierRank[right.tier] || left.index - right.index; }); - const withoutSupplementaryPadding = ranked.filter( - (candidate) => !isSupplementaryPadding({ candidate, ranked, sufficientAustralianChunks }), - ); + const withoutSupplementaryPadding = omitSupplementaryPadding + ? ranked.filter((candidate) => !isSupplementaryPadding({ candidate, ranked, sufficientAustralianChunks })) + : ranked; const capped = capClinicalContextPerDocument( withoutSupplementaryPadding.map(({ result }) => result), maxPerDocument, diff --git a/src/lib/rag-context-selection.ts b/src/lib/rag-context-selection.ts index d4f05e7d0..3c0309dc3 100644 --- a/src/lib/rag-context-selection.ts +++ b/src/lib/rag-context-selection.ts @@ -31,13 +31,37 @@ export function selectModelContextResults(args: { crossDocument: boolean; results: SearchResult[]; }) { - if (args.queryClass === "medication_dose_risk" || args.queryClass === "table_threshold") { + const highRiskNumericQuery = args.queryClass === "medication_dose_risk" || args.queryClass === "table_threshold"; + if (highRiskNumericQuery) { return selectAustralianClinicalContext(args.results); } - const results = capPerDocumentCrowding(args.results); - if (args.routeMode !== "fast") return results; - if (args.crossDocument || args.queryClass === "comparison" || args.queryClass === "broad_summary") { - return results; - } - return results.slice(0, fastRoutineModelContextLimit); + + const fastRoutineQuery = + args.routeMode === "fast" && + !args.crossDocument && + args.queryClass !== "comparison" && + args.queryClass !== "broad_summary"; + // Preserve the retrieval-ranked fast budget before applying the order-only + // Australian preference. This keeps a stronger/unique supplementary result + // inside the four-chunk budget while still moving equally relevant local + // guidance ahead within that retained set. Apply the existing crowding cap + // first so a fourth chunk from one document cannot hide another document. + const preferenceCandidates = fastRoutineQuery + ? capPerDocumentCrowding( + args.results.filter((result) => result.relevance?.verdict !== "none"), + maxContextChunksPerDocument, + ).slice(0, fastRoutineModelContextLimit) + : args.results; + + // All answer classes should prefer authoritative Australian guidance when it + // is equally relevant. For non-numeric questions this is an order-only + // preference: retain supplementary evidence so a local source cannot hide a + // stronger or uniquely relevant international passage. Numeric/high-risk + // queries keep the stricter bounded Australian-first policy above. + const results = selectAustralianClinicalContext(preferenceCandidates, { + limit: preferenceCandidates.length, + maxPerDocument: maxContextChunksPerDocument, + omitSupplementaryPadding: false, + }); + return results; } diff --git a/src/lib/rag.ts b/src/lib/rag.ts index f6c6d0ce4..0e44cf08e 100644 --- a/src/lib/rag.ts +++ b/src/lib/rag.ts @@ -5699,13 +5699,15 @@ ${qualityRetryInstruction}` } } -/** Summarize document. */ -export async function summarizeDocument(documentId: string, ownerId?: string) { +/** Summarize the committed document context; the route applies the shared client-response governance contract. */ +export async function summarizeDocument(documentId: string, ownerId?: string, options?: { signal?: AbortSignal }) { const supabase = createAdminClient(); let documentQuery = supabase.from("documents").select("id,title,file_name,metadata").eq("id", documentId); if (ownerId) { documentQuery = documentQuery.eq("owner_id", ownerId); + } else { + documentQuery = documentQuery.is("owner_id", null); } const { data: document, error: documentError } = await documentQuery.maybeSingle(); @@ -5747,12 +5749,13 @@ export async function summarizeDocument(documentId: string, ownerId?: string) { })) as SearchResult[]; const summaryInstructions = `Summarize a clinical document for practical psychiatric use in Perth, Australia. -Use only the excerpts provided. Use a layered response: make the answer field a plain high-yield clinical paragraph, usually 1-3 short sentences and 35-75 words, then use answerSections for distinct structured support when it improves scanability. Do not prefix the answer with "Summary", "Key practical points", "Direct answer", or similar labels, and do not use bullets in the answer field. Focus on high-yield actions, thresholds, medication or risk monitoring, exceptions, comparisons, source gaps, and citations. Exclude administrative document-control details unless they change clinical action. +Use only the excerpts provided. Use a layered response: make the answer field a plain high-yield clinical paragraph, +usually 1-3 short sentences and 35-75 words, then use answerSections for distinct structured support when it improves +scanability. Do not prefix the answer with "Summary", "Key practical points", "Direct answer", or similar labels, and +do not use bullets in the answer field. Focus on high-yield actions, thresholds, medication or risk monitoring, +exceptions, comparisons, source gaps, and citations. Exclude administrative document-control details unless they +change clinical action. Return data matching the supplied structured output schema.`; - // Threat model Vectors B/C: document.title is attacker-influenceable (upload - // filename/title metadata) and was reaching the prompt raw here, in the - // highest-trust "Document:" header position — unlike the per-result titles - // inside buildRagSourceBlock, which already neutralize. const summaryInput = `Document: ${neutralizeIdentityField(document.title)} @@ -5767,6 +5770,7 @@ ${buildRagSourceBlock(results)}`; instructions: summaryInstructions, promptCacheKey: "clinical-document-summary-v2", reasoningEffort: env.OPENAI_SUMMARY_REASONING_EFFORT, + signal: options?.signal, }); const answer = parseAnswerJson(generated.text, results, "summary"); answer.answer = cleanClinicalSummaryText(answer.answer); diff --git a/tests/answer-progress.test.ts b/tests/answer-progress.test.ts index eb212a066..4c9741784 100644 --- a/tests/answer-progress.test.ts +++ b/tests/answer-progress.test.ts @@ -5,6 +5,7 @@ import { normalizeAnswerProgressEvent, } from "../src/components/clinical-dashboard/answer-progress"; import { toPublicAnswerProgressEvent } from "../src/lib/answer-progress-public"; +import { readAnswerStream } from "../src/components/clinical-dashboard/search-utils"; describe("answer progress events", () => { it("keeps only safe, normalized Australian source counts at the public boundary", () => { @@ -64,4 +65,40 @@ describe("answer progress events", () => { }), ).toMatchObject({ resultCount: 2, selectedContextCount: undefined }); }); + + it("shares one SSE parser across answer surfaces and commits completion only with a valid final answer", async () => { + const progress: string[] = []; + const tokens: string[] = []; + let revisions = 0; + const body = [ + 'event: progress\ndata: {"stage":"retrieving","message":"private"}', + 'event: token\ndata: {"delta":"Draft"}', + "event: revising\ndata: {}", + 'event: progress\ndata: {"stage":"complete","message":"private","elapsedMs":1200}', + 'event: final\ndata: {"answer":"Grounded answer.","grounded":true,"confidence":"medium","citations":[],"sources":[]}', + "", + ].join("\n\n"); + + const answer = await readAnswerStream( + new Response(body, { headers: { "Content-Type": "text/event-stream" } }), + (event) => progress.push(event.stage), + (delta) => tokens.push(delta), + () => { + revisions += 1; + }, + ); + + expect(progress).toEqual(["retrieving", "complete"]); + expect(tokens).toEqual(["Draft"]); + expect(revisions).toBe(1); + expect(answer.answer).toBe("Grounded answer."); + }); + + it("fails closed when a shared answer stream ends without a valid final payload", async () => { + const response = new Response('event: progress\ndata: {"stage":"complete","message":"private"}\n\n'); + + await expect(readAnswerStream(response, () => undefined)).rejects.toThrow( + "Answer stream ended before a final answer was received.", + ); + }); }); diff --git a/tests/answer-response.test.ts b/tests/answer-response.test.ts new file mode 100644 index 000000000..7e073aa8e --- /dev/null +++ b/tests/answer-response.test.ts @@ -0,0 +1,112 @@ +import { describe, expect, it } from "vitest"; + +import { + answerDegradedModeSignal, + buildGovernedAnswerClientResponse, + buildGovernedDemoAnswerClientResponse, +} from "../src/lib/answer-response"; +import type { RagAnswer, SearchResult } from "../src/lib/types"; + +function source(documentStatus: "current" | "outdated" = "current"): SearchResult { + return { + id: "chunk-1", + document_id: "document-1", + title: "Clinical guideline", + file_name: "clinical-guideline.pdf", + page_number: 1, + chunk_index: 0, + section_heading: "Monitoring", + content: "Use the current monitoring pathway.", + image_ids: [], + images: [], + similarity: 0.9, + source_metadata: { + source_title: "Clinical guideline", + publisher: "WA Health", + publisher_code: "WA_HEALTH", + jurisdiction: "Australia/WA", + version: null, + publication_date: null, + review_date: null, + uploaded_at: null, + indexed_at: null, + uploaded_by: null, + document_status: documentStatus, + clinical_validation_status: "approved", + extraction_quality: "good", + }, + }; +} + +function answer(overrides: Partial = {}): RagAnswer { + return { + answer: "Use the cited monitoring pathway.", + grounded: true, + confidence: "high", + citations: [ + { + chunk_id: "chunk-1", + document_id: "document-1", + title: "Clinical guideline", + file_name: "clinical-guideline.pdf", + page_number: 1, + chunk_index: 0, + }, + ], + sources: [source()], + ...overrides, + }; +} + +describe("governed answer response", () => { + it("keeps a normal grounded answer and derives source-only degradation consistently", () => { + const result = buildGovernedAnswerClientResponse( + answer({ answerQualityTier: "source_only", fallbackReason: "generation_fallback" }), + ); + + expect(result.refused).toBe(false); + expect(result.payload).toMatchObject({ + answer: "Use the cited monitoring pathway.", + degradedMode: { active: true, reason: "generation_fallback" }, + }); + expect(answerDegradedModeSignal()).toEqual({ active: false, reason: null }); + }); + + it("fails closed without leaking answer-only fields when any answer route sees danger governance", () => { + const result = buildGovernedAnswerClientResponse( + answer({ + sources: [source("outdated")], + smartPanel: { query: "monitoring" } as RagAnswer["smartPanel"], + smartApiPlan: { displayMode: "direct" } as unknown as RagAnswer["smartApiPlan"], + }), + ); + + expect(result.refused).toBe(true); + expect(result.payload).toMatchObject({ grounded: false, confidence: "unsupported", citations: [], sources: [] }); + expect(result.payload).not.toHaveProperty("smartPanel"); + expect(result.payload).not.toHaveProperty("smartApiPlan"); + expect(result.telemetryAnswer.routingReason).toContain("source_governance_refusal"); + }); + + it("applies the same governed and degraded contract to demo answers", () => { + const result = buildGovernedDemoAnswerClientResponse( + answer({ + sources: [source("outdated")], + smartPanel: { query: "monitoring" } as RagAnswer["smartPanel"], + }), + "supabase_api_key_configuration", + ); + + expect(result).toMatchObject({ + demoMode: true, + grounded: false, + confidence: "unsupported", + citations: [], + sources: [], + degradedMode: { active: true, reason: "supabase_api_key_configuration" }, + fallbackMode: "non_production_demo", + fallbackReason: "supabase_api_key_configuration", + }); + expect(result).not.toHaveProperty("smartPanel"); + }); +}); diff --git a/tests/private-access-routes.test.ts b/tests/private-access-routes.test.ts index f67cd85d0..f14fd4d5d 100644 --- a/tests/private-access-routes.test.ts +++ b/tests/private-access-routes.test.ts @@ -1,3 +1,4 @@ +import { createHash } from "node:crypto"; import { afterEach, describe, expect, it, vi } from "vitest"; const userId = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa"; @@ -7,6 +8,14 @@ const otherDocumentId = "22222222-2222-4222-8222-222222222222"; const imageId = "33333333-3333-4333-8333-333333333333"; const token = "valid-token"; +function expectFeedbackTokenBoundToAnswer(payload: Record) { + expect(payload.interactionId).toEqual(expect.any(String)); + expect(payload.feedbackToken).toEqual(expect.any(String)); + const [encodedClaims] = String(payload.feedbackToken).split("."); + const claims = JSON.parse(Buffer.from(encodedClaims!, "base64url").toString("utf8")) as { answerHash?: string }; + expect(claims.answerHash).toBe(createHash("sha256").update(String(payload.answer), "utf8").digest("hex")); +} + type QueryError = { message: string }; type QueryResult = { data: unknown; error: QueryError | null }; type QueryFilter = { column: string; value: unknown }; @@ -296,6 +305,7 @@ function mockRuntime( vi.doUnmock("@/lib/openai"); vi.doUnmock("@/lib/document-enrichment"); vi.doUnmock("@/lib/deep-memory"); + vi.doUnmock("@/lib/demo-data"); vi.doMock("@/lib/env", () => ({ env: { MAX_UPLOAD_MB: 150, @@ -3963,6 +3973,166 @@ describe("private document API access", () => { expect(answerQuestionWithScope).not.toHaveBeenCalled(); }); + it("preserves danger-governance warnings when streaming a demo refusal", async () => { + const answerQuestionWithScope = vi.fn(); + const client = createSupabaseMock(); + mockRuntime(client, { answerQuestionWithScope }, { demoMode: true }); + vi.doMock("@/lib/demo-data", () => ({ + demoSummary: vi.fn(), + demoAnswer: vi.fn(() => ({ + answer: "Use the outdated protocol.", + grounded: true, + confidence: "high", + citations: [{ chunk_id: "demo-danger", document_id: documentId, page_number: 1 }], + sources: [ + { + id: "demo-danger", + document_id: documentId, + title: "Outdated demo guideline", + file_name: "outdated-demo.pdf", + page_number: 1, + chunk_index: 0, + section_heading: null, + content: "Use the outdated protocol.", + image_ids: [], + similarity: 0.95, + source_metadata: { + source_title: "Outdated demo guideline", + publisher: "Local WA service", + jurisdiction: "Australia/WA", + version: null, + publication_date: null, + review_date: null, + uploaded_at: null, + indexed_at: null, + uploaded_by: null, + document_status: "outdated", + clinical_validation_status: "approved", + extraction_quality: "good", + }, + images: [], + }, + ], + })), + })); + const { POST } = await import("../src/app/api/answer/stream/route"); + + const response = await POST( + request("/api/answer/stream", { + method: "POST", + body: JSON.stringify({ query: "monitoring" }), + }), + ); + const responseBody = await response.text(); + const finalPayload = ssePayload(responseBody, "final"); + + expect(response.status).toBe(200); + expect(finalPayload).toMatchObject({ + grounded: false, + confidence: "unsupported", + citations: [], + sources: [], + demoMode: true, + sourceGovernanceWarnings: expect.arrayContaining([ + expect.objectContaining({ code: "outdated_source", severity: "danger" }), + ]), + }); + expect(String(finalPayload.answer)).toContain("cannot provide a clinical answer"); + expect(answerQuestionWithScope).not.toHaveBeenCalled(); + }); + + it("streams blank-document summaries through the committed full-document summary path", async () => { + const answerQuestionWithScope = vi.fn(); + const summarizeDocument = vi.fn(async () => ({ + answer: "Full committed document summary.", + grounded: true, + confidence: "high", + citations: [], + sources: [], + })); + const client = createSupabaseMock(); + mockRuntime(client, { answerQuestionWithScope, summarizeDocument }); + const { POST } = await import("../src/app/api/answer/stream/route"); + + const response = await POST( + authenticatedRequest("/api/answer/stream", { + method: "POST", + body: JSON.stringify({ + query: "Summarize this document for practical clinical use.", + documentId, + summaryMode: true, + }), + }), + ); + const responseBody = await response.text(); + + expect(response.status).toBe(200); + expect(responseBody).toContain('"stage":"retrieving"'); + expect(responseBody).toContain('"stage":"generating"'); + expect(ssePayload(responseBody, "final")).toMatchObject({ + answer: "Full committed document summary.", + interactionId: expect.any(String), + feedbackToken: expect.any(String), + }); + expect(summarizeDocument).toHaveBeenCalledWith(documentId, userId, { + signal: expect.any(AbortSignal), + }); + expect(client.rpc).toHaveBeenCalledWith( + "consume_api_rate_limit", + expect.objectContaining({ p_owner_id: userId, p_bucket: "document_summarize" }), + ); + expect(answerQuestionWithScope).not.toHaveBeenCalled(); + }); + + it("rate limits streamed document summaries before provider work", async () => { + const summarizeDocument = vi.fn(async () => ({ + answer: "Expensive streamed summary.", + grounded: true, + confidence: "high", + citations: [], + sources: [], + })); + const client = createSupabaseMock(); + client.rpc.mockImplementation(async (name: string, args?: Record) => + name === "consume_api_rate_limit" && args?.p_bucket === "document_summarize" + ? { data: [rateLimitRow({ limited: true, limit_value: 12, remaining: 0 })], error: null } + : name === "consume_api_rate_limit" + ? { data: [rateLimitRow()], error: null } + : ok([]), + ); + mockRuntime(client, { summarizeDocument }); + const { POST } = await import("../src/app/api/answer/stream/route"); + + const response = await POST( + authenticatedRequest("/api/answer/stream", { + method: "POST", + body: JSON.stringify({ + query: "Summarize this document for practical clinical use.", + documentId, + summaryMode: true, + }), + }), + ); + + expect(response.status).toBe(429); + expect(response.headers.get("Retry-After")).toBe("60"); + expect(await payload(response)).toMatchObject({ + error: "Too many document summary requests. Retry shortly.", + details: { retryAfterSeconds: 60 }, + }); + expect(client.rpc).toHaveBeenNthCalledWith( + 1, + "consume_api_rate_limit", + expect.objectContaining({ p_owner_id: userId, p_bucket: "answer" }), + ); + expect(client.rpc).toHaveBeenNthCalledWith( + 2, + "consume_api_rate_limit", + expect.objectContaining({ p_owner_id: userId, p_bucket: "document_summarize" }), + ); + expect(summarizeDocument).not.toHaveBeenCalled(); + }); + it("completes cached answers after safe cached progress and before final", async () => { const answerQuestionWithScope = vi.fn(async (args: { onProgress?: (event: unknown) => void | Promise }) => { await args.onProgress?.({ @@ -4172,6 +4342,7 @@ describe("private document API access", () => { expect(finalPayload.sourceGovernanceWarnings).toEqual([ expect.objectContaining({ code: "outdated_source", severity: "danger" }), ]); + expectFeedbackTokenBoundToAnswer(finalPayload); }); it("refuses answer responses backed by danger-class source governance warnings", async () => { @@ -4236,6 +4407,7 @@ describe("private document API access", () => { expect(body.sourceGovernanceWarnings).toEqual([ expect.objectContaining({ code: "outdated_source", severity: "danger" }), ]); + expectFeedbackTokenBoundToAnswer(body); }); it("rate limits document summarization before OpenAI work", async () => { @@ -4321,6 +4493,65 @@ describe("private document API access", () => { expect(summarizeDocument).toHaveBeenCalledWith(otherDocumentId, userId); }); + it("applies the shared danger-governance refusal to the legacy document summary endpoint", async () => { + const summarizeDocument = vi.fn(async () => ({ + answer: "Use the old pathway.", + grounded: true, + confidence: "high", + citations: [{ chunk_id: "summary-chunk", document_id: documentId, page_number: 1 }], + smartPanel: { query: "summary" }, + smartApiPlan: { displayMode: "direct" }, + sources: [ + { + id: "summary-chunk", + document_id: documentId, + title: "Outdated summary source", + file_name: "outdated-summary.pdf", + page_number: 1, + chunk_index: 0, + section_heading: "Summary", + content: "Use the old pathway.", + image_ids: [], + images: [], + similarity: 0.9, + source_metadata: { + source_title: "Outdated summary source", + publisher: "WA Health", + publisher_code: "WA_HEALTH", + jurisdiction: "Australia/WA", + version: null, + publication_date: null, + review_date: null, + uploaded_at: null, + indexed_at: null, + uploaded_by: null, + document_status: "outdated", + clinical_validation_status: "approved", + extraction_quality: "good", + }, + }, + ], + })); + const client = createSupabaseMock(); + mockRuntime(client, { summarizeDocument }); + const { POST } = await import("../src/app/api/documents/[id]/summarize/route"); + + const response = await POST(authenticatedRequest(`/api/documents/${documentId}/summarize`, { method: "POST" }), { + params: Promise.resolve({ id: documentId }), + }); + const body = await payload(response); + + expect(response.status).toBe(200); + expect(body).toMatchObject({ grounded: false, confidence: "unsupported", citations: [], sources: [] }); + expect(body.smartPanel).toBeUndefined(); + expect(body.smartApiPlan).toBeUndefined(); + expect(body.sourceGovernanceWarnings).toEqual([ + expect.objectContaining({ code: "outdated_source", severity: "danger" }), + ]); + expect(body.interactionId).toMatch(/^[0-9a-f-]{36}$/i); + expectFeedbackTokenBoundToAnswer(body); + }); + it("passes owner scope into retrieval RPCs", async () => { const client = createSupabaseMock(); mockRuntime(client); diff --git a/tests/rag-context-budget.test.ts b/tests/rag-context-budget.test.ts index ef9e82bbe..2a7f6f203 100644 --- a/tests/rag-context-budget.test.ts +++ b/tests/rag-context-budget.test.ts @@ -330,6 +330,72 @@ describe("RAG model context budgeting", () => { expect(selected.map((result) => result.id)).toEqual(["chunk-2", "chunk-3", "chunk-4", "chunk-5"]); }); + it.each(["document_lookup", "comparison", "broad_summary", "unsupported_or_general"] as const)( + "prefers equally relevant Australian passages for %s answers without dropping supplementary evidence", + (queryClass) => { + const selected = selectModelContextResults({ + routeMode: "strong", + queryClass, + crossDocument: queryClass === "comparison", + results: [ + withRelevance(bmjSupplementarySource(1), "direct"), + withRelevance(waGovernedSource(2, "wa-doc-1"), "direct"), + withRelevance(waGovernedSource(3, "wa-doc-2"), "direct"), + ], + }); + + expect(selected.map((result) => result.id)).toEqual(["chunk-2", "chunk-3", "chunk-1"]); + }, + ); + + it("keeps a more relevant international passage ahead of weaker Australian evidence", () => { + const selected = selectModelContextResults({ + routeMode: "strong", + queryClass: "broad_summary", + crossDocument: false, + results: [ + withRelevance(waGovernedSource(1, "wa-doc-1"), "partial"), + withRelevance(bmjSupplementarySource(2), "direct"), + ], + }); + + expect(selected.map((result) => result.id)).toEqual(["chunk-2", "chunk-1"]); + }); + + it("keeps a higher-ranked supplementary passage inside the routine fast budget", () => { + const selected = selectModelContextResults({ + routeMode: "fast", + queryClass: "document_lookup", + crossDocument: false, + results: [ + withRelevance(bmjSupplementarySource(1), "direct"), + withRelevance(waGovernedSource(2, "wa-doc-1"), "direct"), + withRelevance(waGovernedSource(3, "wa-doc-2"), "direct"), + withRelevance(waGovernedSource(4, "wa-doc-1"), "direct"), + withRelevance(waGovernedSource(5, "wa-doc-2"), "direct"), + ], + }); + + expect(selected.map((result) => result.id)).toEqual(["chunk-2", "chunk-3", "chunk-4", "chunk-1"]); + }); + + it("applies the document crowding cap before fixing the routine fast budget", () => { + const selected = selectModelContextResults({ + routeMode: "fast", + queryClass: "document_lookup", + crossDocument: false, + results: [ + source(1, { document_id: "crowded-doc" }), + source(2, { document_id: "crowded-doc" }), + source(3, { document_id: "crowded-doc" }), + source(4, { document_id: "crowded-doc" }), + source(5, { document_id: "other-doc" }), + ], + }); + + expect(selected.map((result) => result.id)).toEqual(["chunk-1", "chunk-2", "chunk-3", "chunk-5"]); + }); + it("uses a stable context pack cache key for matching retry inputs", () => { const key = packedContextCacheKey(results, "broad_summary", { crossDocument: true }); const sameInputs = packedContextCacheKey([...results], "broad_summary", { crossDocument: true }); diff --git a/tests/rag-document-summary.test.ts b/tests/rag-document-summary.test.ts new file mode 100644 index 000000000..a5046ee8b --- /dev/null +++ b/tests/rag-document-summary.test.ts @@ -0,0 +1,109 @@ +import { afterEach, describe, expect, it, vi } from "vitest"; + +afterEach(() => { + vi.restoreAllMocks(); + vi.resetModules(); + vi.unstubAllEnvs(); +}); + +describe("document summary context", () => { + it("summarizes up to 40 ordered committed chunks instead of normal retrieval results", async () => { + vi.stubEnv("OPENAI_API_KEY", "test-key"); + + const documentId = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa"; + const ownerId = "bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb"; + const chunks = Array.from({ length: 45 }, (_, index) => ({ + id: `summary-chunk-${index + 1}`, + document_id: documentId, + page_number: index + 1, + chunk_index: index, + section_heading: `Section ${index + 1}`, + content: `Committed summary evidence ${index + 1}.`, + retrieval_synopsis: null, + image_ids: [], + index_generation_id: "generation-current", + })); + + const documentQuery = { + select: vi.fn().mockReturnThis(), + eq: vi.fn().mockReturnThis(), + maybeSingle: vi.fn(async () => ({ + data: { + id: documentId, + title: "Long clinical guideline", + file_name: "long-guideline.pdf", + metadata: { + index_generation_id: "generation-current", + publisher: "WA Health", + jurisdiction: "Australia/WA", + document_status: "current", + clinical_validation_status: "approved", + extraction_quality: "good", + }, + }, + error: null, + })), + }; + const chunkQuery = { + select: vi.fn().mockReturnThis(), + eq: vi.fn().mockReturnThis(), + order: vi.fn().mockReturnThis(), + limit: vi.fn(async (limit: number) => ({ data: chunks.slice(0, limit), error: null })), + }; + const rpc = vi.fn(); + const from = vi.fn((table: string) => (table === "documents" ? documentQuery : chunkQuery)); + + vi.doMock("@/lib/supabase/admin", () => ({ + createAdminClient: () => ({ from, rpc }), + })); + + const generateStructuredTextResult = vi.fn( + async (input: string, schema?: Record, options?: { signal?: AbortSignal }) => { + void input; + void schema; + void options; + return { + text: JSON.stringify({ + answer: + "The document provides practical psychiatric guidance across the full indexed source, including the final committed section.", + grounded: true, + confidence: "high", + answerSections: [], + citations: [{ chunk_id: "summary-chunk-40" }], + quoteCards: [], + conflictsOrGaps: [], + }), + model: "gpt-4.1-mini", + operation: "summary", + latencyMs: 12, + requestId: "req_document_summary", + usage: { input_tokens: 400, output_tokens: 60, total_tokens: 460 }, + }; + }, + ); + vi.doMock("@/lib/openai", () => ({ + embedTextWithTelemetry: vi.fn(), + generateStructuredTextResult, + })); + + const { summarizeDocument } = await import("../src/lib/rag"); + const controller = new AbortController(); + const answer = await summarizeDocument(documentId, ownerId, { signal: controller.signal }); + + expect(documentQuery.eq).toHaveBeenCalledWith("owner_id", ownerId); + expect(chunkQuery.order).toHaveBeenCalledWith("chunk_index", { ascending: true }); + expect(chunkQuery.limit).toHaveBeenCalledWith(40); + expect(rpc).not.toHaveBeenCalled(); + expect(generateStructuredTextResult).toHaveBeenCalledTimes(1); + const summaryInput = generateStructuredTextResult.mock.calls[0]?.[0] ?? ""; + expect(generateStructuredTextResult.mock.calls[0]?.[2]).toEqual( + expect.objectContaining({ signal: controller.signal }), + ); + expect(summaryInput).toContain("Committed summary evidence 1."); + expect(summaryInput).toContain("Committed summary evidence 40."); + expect(summaryInput).not.toContain("Committed summary evidence 41."); + expect(answer.citations).toEqual( + expect.arrayContaining([expect.objectContaining({ chunk_id: "summary-chunk-40" })]), + ); + }); +}); diff --git a/tests/ui-smoke.spec.ts b/tests/ui-smoke.spec.ts index a0196925b..c22f2a837 100644 --- a/tests/ui-smoke.spec.ts +++ b/tests/ui-smoke.spec.ts @@ -3,7 +3,8 @@ import { expect, test, type Locator, type Page } from "playwright/test"; import { scrollPrimarySurface } from "./playwright-scroll"; import { recentQueryStorageKey } from "../src/components/clinical-dashboard/dashboard-contracts"; import { answerThreadStorageKey } from "../src/lib/answer-thread-storage"; -import { demoAnswer, demoDocuments, getDemoDocument, getDemoDocumentPayload } from "../src/lib/demo-data"; +import { documentSummaryQuestion } from "../src/lib/answer-contract"; +import { demoAnswer, demoDocuments, demoSummary, getDemoDocument, getDemoDocumentPayload } from "../src/lib/demo-data"; import { deriveGovernanceFromSections } from "../src/lib/medication-records"; import { getMedicationRecord, loadMedicationSnapshot } from "../src/lib/medication-snapshot"; import { medicationToSearchResult, rankMedicationRecords } from "../src/lib/medications"; @@ -178,6 +179,8 @@ async function mockPrivateUnauthenticatedApi(page: Page) { function answerStreamBody(payload: unknown) { return [ `event: progress\ndata: ${JSON.stringify({ stage: "retrieving", message: "Searching indexed documents." })}`, + `event: progress\ndata: ${JSON.stringify({ stage: "ranking", message: "Selecting governed sources." })}`, + `event: progress\ndata: ${JSON.stringify({ stage: "complete", message: "Answer ready.", elapsedMs: 1250 })}`, `event: final\ndata: ${JSON.stringify(payload)}`, "", ].join("\n\n"); @@ -201,7 +204,10 @@ type DemoAnswerOverride = (query: string, documentId?: string, documentIds?: str type MockDemoApiOptions = { answerOverride?: DemoAnswerOverride; answerDelayMs?: number; - onAnswerRequest?: (query: string) => void; + onAnswerRequest?: ( + query: string, + scope: { documentId?: string; documentIds?: string[]; summaryMode?: boolean }, + ) => void; }; async function blockExternalRequests(page: Page) { @@ -297,19 +303,26 @@ async function mockDemoApi(page: Page, options: MockDemoApiOptions = {}) { query?: string; documentId?: string; documentIds?: string[]; + summaryMode?: boolean; }; const query = typeof body.query === "string" ? body.query.trim() : ""; if (!query || query.length > 2000) { await route.fulfill({ status: 400, json: { error: "A query between 1 and 2000 characters is required." } }); return; } - options.onAnswerRequest?.(query); + options.onAnswerRequest?.(query, { + documentId: body.documentId, + documentIds: body.documentIds, + summaryMode: body.summaryMode, + }); if (options.answerDelayMs) { await new Promise((resolve) => setTimeout(resolve, options.answerDelayMs)); } const answer = options.answerOverride?.(query, body.documentId, body.documentIds) ?? - demoAnswer(query, body.documentId, body.documentIds); + (body.summaryMode && body.documentId + ? demoSummary(body.documentId) + : demoAnswer(query, body.documentId, body.documentIds)); await fulfillAnswerResponse(route, { ...answer, demoMode: true, @@ -1670,13 +1683,29 @@ test.describe("Clinical KB UI smoke coverage", () => { // is the whole point: content shorter than the viewport must not scroll. await page.setViewportSize({ width: 390, height: 900 }); await mockDemoApi(page, { - // Strip the section/table-bearing fields so the surface is a few hundred px. - answerOverride: (query, documentId, documentIds) => ({ - ...demoAnswer(query, documentId, documentIds), - answer: "Verify the cited passages before using any clinical numbers.", - answerSections: [], - visualEvidence: [], - }), + // Keep this a genuinely short answer as the shared answer contract grows: + // rich support, safety, and related-document fields are covered elsewhere. + answerOverride: (query, documentId, documentIds) => { + const base = demoAnswer(query, documentId, documentIds); + return { + ...base, + answer: "Verify the cited passages before using any clinical numbers.", + answerSections: [], + visualEvidence: [], + quoteCards: [], + documentBreakdown: [], + evidenceSummary: undefined, + sourceCoverage: undefined, + conflictsOrGaps: [], + bestSource: undefined, + smartPanel: undefined, + relatedDocuments: [], + sources: base.sources.map((source) => ({ + ...source, + content: "This indexed passage directly supports the short answer.", + })), + }; + }, }); await gotoApp(page, "/"); await waitForDemoDashboardReady(page); @@ -2924,21 +2953,41 @@ test.describe("Clinical KB UI smoke coverage", () => { await expect(composer).not.toHaveAttribute("data-scroll-hidden", "true"); }); - test("document summary opens at the top with cleaned bold formatting", async ({ page }) => { + test("document questions use the shared answer stream with progress and cleaned bold formatting", async ({ + page, + }) => { await page.setViewportSize({ width: 390, height: 820 }); - await mockDemoApi(page); + const answerRequests: Array<{ query: string; documentId?: string; summaryMode?: boolean }> = []; + let legacySummaryRequestCount = 0; + page.on("request", (request) => { + if (/\/api\/documents\/[^/]+\/summarize$/.test(new URL(request.url()).pathname)) { + legacySummaryRequestCount += 1; + } + }); + await mockDemoApi(page, { + onAnswerRequest: (query, scope) => + answerRequests.push({ query, documentId: scope.documentId, summaryMode: scope.summaryMode }), + answerOverride: (query, documentId, documentIds) => ({ + ...demoAnswer(query, documentId, documentIds), + answer: + "Key practical points: **clozapine** monitoring requires regular FBC/ANC checks and review of constipation, myocarditis symptoms, metabolic risk, and missed-dose restart rules.", + }), + }); await gotoApp( page, "/documents/11111111-1111-4111-8111-111111111111?page=1&chunk=44444444-4444-4444-8444-444444444442", ); - await page - .getByRole("button", { name: /^Answer from this(?: document)?$/ }) - .first() - .click(); + const composer = page.locator("form.document-viewer-composer"); + await composer + .getByRole("textbox", { name: "Search or answer from this document" }) + .fill("How is clozapine monitored?"); + await composer.getByRole("button", { name: "Answer from this document" }).click(); const generatedSummary = page.getByTestId("generated-clinical-summary"); await expect(generatedSummary).toBeVisible(); + await expect(page.getByTestId("answer-progress-stepper")).toHaveAttribute("data-progress-state", "complete"); + await expect(page.getByText(/Answer ready in 1s/)).toBeVisible(); await expect(generatedSummary).toContainText("clozapine monitoring requires regular FBC/ANC checks"); await expect(generatedSummary).not.toContainText("Key practical points:"); await expect(generatedSummary).not.toContainText("**"); @@ -2949,6 +2998,24 @@ test.describe("Clinical KB UI smoke coverage", () => { expect(summaryBox).not.toBeNull(); expect(previewBox).not.toBeNull(); expect(summaryBox!.y).toBeLessThan(previewBox!.y); + expect(answerRequests).toEqual([ + { + query: "How is clozapine monitored?", + documentId: "11111111-1111-4111-8111-111111111111", + summaryMode: undefined, + }, + ]); + expect(legacySummaryRequestCount).toBe(0); + + await composer.getByRole("textbox", { name: "Search or answer from this document" }).fill(""); + await composer.getByRole("button", { name: "Answer from this document" }).click(); + await expect.poll(() => answerRequests.length).toBe(2); + expect(answerRequests[1]).toEqual({ + query: documentSummaryQuestion, + documentId: "11111111-1111-4111-8111-111111111111", + summaryMode: true, + }); + expect(legacySummaryRequestCount).toBe(0); await expectNoPageHorizontalOverflow(page); });