diff --git a/docs/branch-review-ledger.md b/docs/branch-review-ledger.md index a9ba4875..e1e8d451 100644 --- a/docs/branch-review-ledger.md +++ b/docs/branch-review-ledger.md @@ -28,6 +28,7 @@ Use this ledger to prevent repeated branch and PR reviews when the reviewed HEAD | 2026-07-17 | PR #738 / cursor/storage-bucket-migration-02e7 | b2755c814b47cdec6868bd009f3ca1cdbc3a7dea | open-PR review + merge babysit | Merge-ready storage-bucket idempotent migration + PR-policy base_ref checkout. Comment clarified for on-conflict reconciliation. Duplicate #710 closed. Merged to main. | Hosted required checks + Migration replay green; review thread resolved. | | 2026-07-17 | PR #739 / cursor/pwa-optimization-merge-02e7 | 0d4aa5c73a4ff30274bb6286be04465286c815d7 | open-PR review + privacy fix + merge babysit | P1 fixed: removed backdated RAG migrations/schema seed that would regress public-only title corrector privacy. Also fixed sitemap API/redirect classification, services RightRail remount, focus=1 mode-menu flake, and merge conflict markers. Supersedes #735/#721. Merged to main. | Hosted full CI green including Production UI/Migration replay after fixes; unresolved CodeRabbit threads resolved. No OpenAI/Supabase writes. | | 2026-07-17 | PR batch #732-#739 (screenshot queue) | 1ab611915e0630a527a533ee7ff16b82fd3e8982 | multi-PR merge babysit summary | Merged in order: #738, #733, #737, #732, #736, #739. #735 left open (token cannot close; CONFLICTING/superseded by #739). #710/#721 closed as duplicates. Residual: close #735 manually. | Hosted CI babysit + merge-tree; #739 Production UI flake fixed and re-verified green before merge. | +| 2026-07-17 | codex/ci-answer-progress-regression | bfa0ed3dfc69d5b333ba43479023cb9a8e8925d3 | CI verification gap | Fixed: the production answer-progress Playwright journey was excluded by both top-level and Chromium project matchers, while its filename also skipped the CI UI trigger; its assertions could therefore change without the required UI job executing. | Local static inspection of `playwright.config.ts`, `scripts/ci-change-scope.mjs`, Vitest globs, and CI workflow; classifier self-test confirms the journey now sets `ui_changed=true`. Focused Playwright execution reached the isolated Next build but was blocked by unavailable Google font downloads; no hosted CI or provider-backed checks run. | | 2026-07-17 | PR #635 / claude/github-actions-codex-issue-f4t4s5 | ab09a8d52cc0a8a7e71b37885aaa358aae2522c8 | post-merge merge-readiness review | Already squash-merged to main on 2026-07-14 by BigSimmo. No open review threads or inline comments. CI required checks all green (Change scope, Static PR checks, Safety and config checks, Unit coverage, PR required, Semgrep, Gitleaks, GitGuardian); UI/build/migration jobs correctly skipped. Landed diff is test/guard hardening only for missing `CODEX_TRIGGER_TOKEN` graceful skip. No high-confidence P0-P2 defect. Source branch already deleted. No further merge action needed. | Hosted CI status via `gh pr checks 635` (all required pass); local `node scripts/check-codex-autofix-workflow.mjs` pass; focused Vitest `tests/codex-autofix-workflow.test.ts` 41/41. No OpenAI/Supabase/provider writes. | | 2026-07-17 | PR #718 / codex/performance-latency-remediation-20260717 | b5f509744d4f4bac74d414644cd1802f64b97fa9 | CodeRabbit performance and SQL correctness follow-up | Resolved nine confirmed findings and dispositioned one stale test comment: document downloads revalidate signed URLs on every action; committed-generation filtering precedes detail pagination; enrichment fallback errors preserve identity; caller cancellation leaves the shared classifier flight alive; registry seeding preserves its cache signal; aliases emit canonical corrections; rate-limit success metadata is coherent; ambiguous upserts use named constraints; and grantable default ACLs fail closed. The proxy mock duplicate was not present. No remaining high-confidence P0-P2 defect was found. | Integrated focused Vitest 122/122; post-format Vitest 71/71; `npm run verify:cheap` passed runtime/policy/static guards, ESLint, TypeScript, and 2,684/2,684 tests; focused Prettier and `git diff --check`; disposable Docker replay, regenerated drift manifest, and transactional local SQL probes. No OpenAI calls, live Supabase DDL/migration/data write, deployment, or production mutation ran. | diff --git a/docs/codebase-index.md b/docs/codebase-index.md index a09628bd..2cfd095e 100644 --- a/docs/codebase-index.md +++ b/docs/codebase-index.md @@ -68,17 +68,19 @@ Structured map for AI agents and onboarding. For live routes, see `docs/site-map ### API routes (`src/app/api/`) -| Area | Routes | Entry files | -| ----------- | ----------------------------------------------------------------------- | --------------------------------------------------------------- | -| Answers | `/api/answer`, `/api/answer/stream`, `/api/answer-feedback` | `answer/route.ts`, `answer/stream/route.ts`, `answer-feedback/` | -| Search | `/api/search`, `/api/search/interaction` | `search/` | -| Upload | `/api/upload` | `upload/route.ts` | -| Documents | CRUD, bulk, reindex, labels, search, summarize, table-facts, signed-url | `documents/` | -| Ingestion | batches, jobs, retry, quality | `ingestion/` | -| Registry | records CRUD | `registry/records/` | -| Images | signed URLs | `images/[id]/signed-url/route.ts` | -| Ops | health, setup-status, local-project-id | `health/`, `setup-status/`, `local-project-id/` | -| Eval / jobs | eval cases, job state | `eval-cases/`, `jobs/` | +| Area | Routes | Entry files | +| ------------- | ------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | +| Answers | `/api/answer`, `/api/answer/stream`, `/api/answer-feedback` | `answer/route.ts`, `answer/stream/route.ts`, `answer-feedback/` | +| Search | `/api/search`, `/api/search/interaction`, `/api/search/universal` | `search/` | +| Upload | `/api/upload` | `upload/route.ts` | +| Documents | `/api/documents`, `/api/documents/[id]`, bulk/reindex, labels, reviews, search, signed URLs, summaries, table facts | `documents/` | +| Differentials | `/api/differentials`, `/api/differentials/[slug]`, `/api/differentials/presentations/[slug]` | `differentials/` | +| Medications | `/api/medications`, `/api/medications/[slug]` | `medications/` | +| Ingestion | `/api/ingestion/batches`, `/api/ingestion/jobs`, retry, quality | `ingestion/` | +| Registry | `/api/registry/records`, `/api/registry/records/[slug]` | `registry/records/` | +| Images | `/api/images/[id]/signed-url` | `images/[id]/signed-url/route.ts` | +| Ops | `/api/health`, `/api/health/ready`, `/api/setup-status`, `/api/local-project-id` | `health/`, `setup-status/`, `local-project-id/` | +| Eval / jobs | `/api/eval-cases`, `/api/jobs` | `eval-cases/`, `jobs/` | --- @@ -151,12 +153,12 @@ Structured map for AI agents and onboarding. For live routes, see `docs/site-map - **CLI:** `supabase/config.toml` — `indexing-v3-agent` function, `verify_jwt = false` - **Schema mirror:** `supabase/schema.sql` (reference; migrations are source of truth) -- **Migrations:** `supabase/migrations/*.sql` (~90 files, May–Jul 2026) +- **Migrations:** `supabase/migrations/*.sql` (chronological source of truth; do not hardcode a count) - **Drift policy:** `docs/supabase-migration-reconciliation.md` -### Core tables +### Schema tables -`documents`, `document_pages`, `document_images`, `document_chunks`, `document_embedding_fields`, `document_index_units`, `document_table_facts`, `document_labels`, `document_summaries`, `document_sections`, `document_memory_cards`, `document_index_quality`, `ingestion_jobs`, `ingestion_job_stages`, `indexing_v3_agent_jobs`, `import_batches`, `rag_queries`, `rag_query_misses`, `rag_aliases`, `rag_response_cache`, `rag_retrieval_logs`, `clinical_registry_records`, `api_rate_limits`, `audit_logs`, `storage_cleanup_jobs` +`documents`, `document_pages`, `document_images`, `document_chunks`, `document_embedding_fields`, `document_index_units`, `document_table_facts`, `document_labels`, `document_summaries`, `document_sections`, `document_memory_cards`, `document_index_quality`, `document_title_words`, `ingestion_jobs`, `ingestion_job_stages`, `indexing_v3_agent_jobs`, `import_batches`, `image_caption_cache`, `rag_queries`, `rag_query_misses`, `rag_aliases`, `rag_response_cache`, `rag_retrieval_logs`, `rag_visual_eval_cases`, `rag_visual_eval_runs`, `rag_answer_feedback`, `clinical_registry_records`, `clinical_registry_record_sources`, `medication_records`, `differential_records`, `source_review_events`, `api_rate_limits`, `api_rate_limit_subjects`, `audit_logs`, `storage_cleanup_jobs` **Storage buckets:** `clinical-documents`, `clinical-images` (private) diff --git a/docs/site-map.md b/docs/site-map.md index f59686b4..51889a6d 100644 --- a/docs/site-map.md +++ b/docs/site-map.md @@ -2,7 +2,7 @@ This file is generated by `npm run sitemap:update`. Run `npm run sitemap:check` to verify it is current. -## Main product pages +## Main product routes - `/` - Main Clinical KB shell. Source: `src/app/page.tsx`. - `/differentials` - Differentials home and search surface. Source: `src/app/differentials/page.tsx`. @@ -802,6 +802,11 @@ This file is generated by `npm run sitemap:update`. Run `npm run sitemap:check` - `/mockups/universal-search-command` - Route discovered from app directory Source: `src/app/mockups/universal-search-command/page.tsx`. - `/mockups/universal-search-redesign` - Route discovered from app directory Source: `src/app/mockups/universal-search-redesign/page.tsx`. +## Public utility route handlers + +- `/auth/callback` - Authentication callback handler. Source: `src/app/auth/callback/route.ts`. +- `/icons/[variant]` - Dynamically generated application icon handler. Source: `src/app/icons/[variant]/route.tsx`. + ## API routes - `/api/answer` - Generate answer response. Source: `src/app/api/answer/route.ts`. @@ -841,14 +846,10 @@ This file is generated by `npm run sitemap:update`. Run `npm run sitemap:check` - `/api/setup-status` - Setup status. Source: `src/app/api/setup-status/route.ts`. - `/api/upload` - Upload endpoint. Source: `src/app/api/upload/route.ts`. -## App route handlers - -- `/auth/callback` - Route discovered from app directory Source: `src/app/auth/callback/route.ts`. - ## Redirects - `/applications` - Redirects to `/tools`. Source: `src/app/applications/route.ts`. -- `/differentials/presentations` - Redirects to `/differentials/presentations/[slug]`. Source: `src/app/differentials/presentations/route.ts`. +- `/differentials/presentations` - Redirects to `/differentials/presentations/[workflow-slug]`. Source: `src/app/differentials/presentations/route.ts`. - `/documents/source` - Redirects to `/documents/search`. Source: `src/app/documents/source/page.tsx`. - `/medications` - Redirects to `/?mode=prescribing`. Source: `src/app/medications/route.ts`. - `/mockups/favourites-hub` - Redirects to `/favourites`. Source: `src/app/mockups/favourites-hub/page.tsx`. diff --git a/playwright.config.ts b/playwright.config.ts index 4195dd79..1d820339 100644 --- a/playwright.config.ts +++ b/playwright.config.ts @@ -43,9 +43,9 @@ export default defineConfig({ trace: "retain-on-failure", screenshot: "only-on-failure", // Disable CSS/web animations suite-wide so a click can't land mid-transition - // on a moving target (documented races in ui-stress/ui-smoke). Set via - // contextOptions (the supported form in this Playwright build); the dedicated - // reduced-motion a11y spec emulates it per-test too, so it is unaffected. + // on a moving target (documented races in ui-stress/ui-smoke). The dedicated + // reduced-motion a11y spec emulates a per-test mode, so suite-wide settings + // remain stable across builds. contextOptions: { reducedMotion: "reduce" }, }, projects: [ @@ -55,6 +55,7 @@ export default defineConfig({ grepInvert: mockupTag, use: { ...devices["Desktop Chrome"], + contextOptions: { reducedMotion: "no-preference" }, ...(chromiumExecutablePath ? { launchOptions: { executablePath: chromiumExecutablePath } } : {}), }, }, diff --git a/scripts/check-github-action-pins.mjs b/scripts/check-github-action-pins.mjs index 6a82fae7..82d19530 100644 --- a/scripts/check-github-action-pins.mjs +++ b/scripts/check-github-action-pins.mjs @@ -1,4 +1,4 @@ -import { readdirSync, readFileSync } from "node:fs"; +import { existsSync, readFileSync, readdirSync } from "node:fs"; import path from "node:path"; import { validateActionReference } from "./github-action-pins.mjs"; import { yamlBlock } from "./yaml-contract.mjs"; @@ -10,10 +10,16 @@ const failures = []; const expectedSupabaseCliVersion = "2.108.0"; const expectedSupabaseCliVersionPattern = expectedSupabaseCliVersion.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); -for (const fileName of readdirSync(workflowDir) - .filter((name) => /\.ya?ml$/i.test(name)) - .sort()) { - const filePath = path.join(workflowDir, fileName); +function discoverGitHubActionFiles(workflowRoot) { + const workflowDir = path.join(workflowRoot, ".github", "workflows"); + if (!existsSync(workflowDir)) return []; + return readdirSync(workflowDir, { withFileTypes: true }) + .filter((entry) => entry.isFile() && /\.ya?ml$/i.test(entry.name)) + .map((entry) => path.join(workflowDir, entry.name)); +} + +for (const filePath of discoverGitHubActionFiles(process.cwd())) { + const fileName = path.relative(process.cwd(), filePath).replaceAll("\\", "/"); const lines = readFileSync(filePath, "utf8").split(/\r?\n/); lines.forEach((line, index) => { diff --git a/scripts/generate-site-map.ts b/scripts/generate-site-map.ts index 8bea4b41..242b88f7 100644 --- a/scripts/generate-site-map.ts +++ b/scripts/generate-site-map.ts @@ -16,7 +16,7 @@ const appDir = path.join(process.cwd(), "src", "app"); const siteMapPath = path.join(process.cwd(), "docs", "site-map.md"); const medicationSlugs = ["acamprosate"] as const; -type RouteKind = "page" | "api"; +type RouteKind = "page" | "handler"; type DiscoveredRoute = { route: string; @@ -31,14 +31,23 @@ type RedirectRoute = { type SiteMapData = { pageRoutes: DiscoveredRoute[]; + publicRouteHandlers: DiscoveredRoute[]; apiRoutes: DiscoveredRoute[]; - appRouteHandlers: DiscoveredRoute[]; redirects: RedirectRoute[]; nonRoutedMockupArtifacts: string[]; }; +const productRouteHandlerPaths = new Set(["/applications", "/differentials/presentations", "/medications"]); + +const documentedRedirectTargets: Record = { + "/applications": "/tools", + "/differentials/presentations": "/differentials/presentations/[workflow-slug]", + "/medications": "/?mode=prescribing", +}; + const routeDescriptions: Record = { "/": "Main Clinical KB shell.", + "/applications": "Legacy application launcher redirect to Tools.", "/differentials": "Differentials home and search surface.", "/differentials/diagnoses": "Diagnosis stream.", "/differentials/diagnoses/[slug]": "Differential diagnosis detail.", @@ -75,6 +84,11 @@ const routeDescriptions: Record = { "/specifiers/map": "Psychiatric specifier family map.", }; +const publicRouteHandlerDescriptions: Record = { + "/auth/callback": "Authentication callback handler.", + "/icons/[variant]": "Dynamically generated application icon handler.", +}; + const apiDescriptions: Record = { "/api/answer": "Generate answer response.", "/api/answer/stream": "Streaming answer response.", @@ -131,8 +145,16 @@ function routeSegment(segment: string) { return segment; } +function isApiRoute(route: string) { + return route === "/api" || route.startsWith("/api/"); +} + function fileToRoute(filePath: string, kind: RouteKind) { - const suffix = kind === "page" ? "page.tsx" : "route.ts"; + const suffix = path.basename(filePath); + const expectedSuffixes = kind === "page" ? ["page.tsx"] : ["route.ts", "route.tsx"]; + if (!expectedSuffixes.includes(suffix)) { + throw new Error(`Unsupported ${kind} route file: ${filePath}`); + } const relative = toPosixPath(path.relative(appDir, filePath)); const withoutFile = relative.slice(0, -suffix.length).replace(/\/$/, ""); const segments = withoutFile.split("/").filter(Boolean).map(routeSegment).filter(Boolean); @@ -153,8 +175,9 @@ function collectFiles(root: string, targetFileName: string): string[] { } function discoverRoutes(kind: RouteKind): DiscoveredRoute[] { - const targetFile = kind === "page" ? "page.tsx" : "route.ts"; - return collectFiles(appDir, targetFile) + const targetFiles = kind === "page" ? ["page.tsx"] : ["route.ts", "route.tsx"]; + return targetFiles + .flatMap((targetFile) => collectFiles(appDir, targetFile)) .map((file) => ({ route: fileToRoute(file, kind), file: toPosixPath(path.relative(process.cwd(), file)), @@ -162,34 +185,12 @@ function discoverRoutes(kind: RouteKind): DiscoveredRoute[] { .sort((left, right) => left.route.localeCompare(right.route) || left.file.localeCompare(right.file)); } -function isApiRoute(route: string) { - return route === "/api" || route.startsWith("/api/"); -} - -function extractRedirectTarget(source: string): string | null { - const pageRedirect = source.match(/\bredirect\(\s*["']([^"']+)["']\s*\)/)?.[1]; - if (pageRedirect) return pageRedirect; - - const urlRedirect = source.match(/NextResponse\.redirect\(\s*new URL\(\s*["']([^"']+)["']/)?.[1]; - if (urlRedirect) return urlRedirect; - - const pathnameRedirect = source.match(/\.pathname\s*=\s*["']([^"']+)["']/)?.[1]; - if (pathnameRedirect) return pathnameRedirect; - - // Template-literal destination builders with a stable route prefix. - const presentationsRedirect = source.match(/`(\/differentials\/presentations\/)\$\{/)?.[1]; - if (presentationsRedirect && source.includes("NextResponse.redirect")) { - return `${presentationsRedirect}[slug]`; - } - - return null; -} - function discoverRedirects(routes: DiscoveredRoute[]): RedirectRoute[] { return routes .map((route) => { const source = readFileSync(path.join(process.cwd(), route.file), "utf8"); - const target = extractRedirectTarget(source); + const target = + documentedRedirectTargets[route.route] ?? source.match(/\bredirect\(\s*["']([^"']+)["']\s*\)/)?.[1]; return target ? { ...route, target } : null; }) .filter((value): value is RedirectRoute => Boolean(value)) @@ -206,20 +207,13 @@ function discoverNonRoutedMockupArtifacts() { export function collectSiteMapData(): SiteMapData { const pageRoutes = discoverRoutes("page"); - const routeHandlers = discoverRoutes("api"); - const apiRoutes = routeHandlers.filter((route) => isApiRoute(route.route)); - const nonApiHandlers = routeHandlers.filter((route) => !isApiRoute(route.route)); - const redirects = [...discoverRedirects(pageRoutes), ...discoverRedirects(nonApiHandlers)].sort( - (left, right) => left.route.localeCompare(right.route) || left.file.localeCompare(right.file), - ); - const redirectRoutes = new Set(redirects.map((redirect) => redirect.route)); - const appRouteHandlers = nonApiHandlers.filter((route) => !redirectRoutes.has(route.route)); - + const routeHandlers = discoverRoutes("handler"); + const publicRouteHandlers = routeHandlers.filter((route) => !isApiRoute(route.route)); return { pageRoutes, - apiRoutes, - appRouteHandlers, - redirects, + publicRouteHandlers, + apiRoutes: routeHandlers.filter((route) => isApiRoute(route.route)), + redirects: discoverRedirects([...pageRoutes, ...publicRouteHandlers]), nonRoutedMockupArtifacts: discoverNonRoutedMockupArtifacts(), }; } @@ -400,6 +394,9 @@ function renderSiteMapRaw(data = collectSiteMapData()) { ].includes(route.route), ); const mockupRoutes = data.pageRoutes.filter((route) => route.route.startsWith("/mockups")); + const publicUtilityRouteHandlers = data.publicRouteHandlers.filter( + (route) => !productRouteHandlerPaths.has(route.route), + ); const lines = [ "# Clinical KB Site Map", @@ -407,7 +404,7 @@ function renderSiteMapRaw(data = collectSiteMapData()) { "This file is generated by `npm run sitemap:update`. Run `npm run sitemap:check` to verify it is current.", "", ...section( - "Main product pages", + "Main product routes", productRoutes.map((route) => routeLine(route, routeDescriptions)), ), ...section("Mode/query routes", renderModeRoutes()), @@ -484,23 +481,21 @@ function renderSiteMapRaw(data = collectSiteMapData()) { ] : []), ]), + ...section( + "Public utility route handlers", + publicUtilityRouteHandlers.map((route) => routeLine(route, publicRouteHandlerDescriptions)), + ), ...section( "API routes", data.apiRoutes.map((route) => routeLine(route, apiDescriptions)), ), - ...(data.appRouteHandlers.length - ? section( - "App route handlers", - data.appRouteHandlers.map((route) => routeLine(route, routeDescriptions)), - ) - : []), ...section( "Redirects", data.redirects.length ? data.redirects.map((redirect) => bullet(redirect.route, `Redirects to \`${redirect.target}\`. Source: \`${redirect.file}\`.`), ) - : ["- No redirects discovered."], + : ["- No page-level redirects discovered."], ), ...section("Known caveats and stale-path flags", [ "- `/mockups/*` prototype routes are development-only; production returns 404 and `robots.txt` disallows indexing.", diff --git a/src/app/layout.tsx b/src/app/layout.tsx index 115c1b2a..3e83cccc 100644 --- a/src/app/layout.tsx +++ b/src/app/layout.tsx @@ -1,8 +1,8 @@ import type { Metadata, Viewport } from "next"; import { Geist, Geist_Mono } from "next/font/google"; import { headers } from "next/headers"; -import { PwaLifecycle } from "@/components/pwa-lifecycle"; import { AuthProvider } from "@/lib/supabase/client"; +import { PwaLifecycle } from "@/components/pwa-lifecycle"; import { WebVitalsReporter } from "@/components/web-vitals-reporter"; import { resolveMetadataBase } from "@/lib/metadata-base"; import { APP_THEME_COLORS } from "@/lib/theme"; diff --git a/src/app/page.tsx b/src/app/page.tsx index 5dc55d31..0d6eeafa 100644 --- a/src/app/page.tsx +++ b/src/app/page.tsx @@ -5,12 +5,7 @@ import { HomePageClient } from "@/app/home-page-client"; import { appModeHomeHref, isAppModeId, isAppModeVisible, type AppModeId } from "@/lib/app-modes"; type HomeProps = { - searchParams?: Promise<{ - mode?: string | string[]; - q?: string | string[]; - focus?: string | string[]; - run?: string | string[]; - }>; + searchParams?: Promise>; }; function firstSearchParam(value: string | string[] | undefined) { diff --git a/src/components/clinical-dashboard/master-search-header.tsx b/src/components/clinical-dashboard/master-search-header.tsx index 099cb89b..6e5444c5 100644 --- a/src/components/clinical-dashboard/master-search-header.tsx +++ b/src/components/clinical-dashboard/master-search-header.tsx @@ -1581,13 +1581,7 @@ export function MasterSearchHeader({ onBlur={(event) => { const nextFocusedElement = event.relatedTarget; if (nextFocusedElement instanceof Node && event.currentTarget.contains(nextFocusedElement)) return; - // Defer dismiss so a pointer activation on a menuitem can land before - // unmount; keyboard leave (Tab/Shift+Tab) still closes on the next frame. - const menuRoot = event.currentTarget; - window.requestAnimationFrame(() => { - if (menuRoot.contains(document.activeElement)) return; - setModeMenuOpen(false); - }); + setModeMenuOpen(false); }} className={cn("relative z-[60] min-w-0", isWorkflowHeader ? "justify-self-start" : "justify-self-center")} > @@ -1596,7 +1590,6 @@ export function MasterSearchHeader({ type="button" onClick={() => { setActionMenuOpen(false); - setCommandDropdownOpen(false); closeScope(false); setModeMenuOpen((open) => !open); }} @@ -1635,11 +1628,6 @@ export function MasterSearchHeader({ id="app-mode-menu" role="menu" aria-label="Choose app mode" - onMouseDown={(event) => { - // Keep focus on the mode trigger so blur-to-dismiss does not - // unmount options before the click lands. - event.preventDefault(); - }} className="polished-scroll fixed left-[max(0.5rem,var(--safe-area-left))] right-[max(0.5rem,var(--safe-area-right))] top-[calc(4.25rem+env(safe-area-inset-top))] z-50 max-h-[min(20rem,calc(100dvh-5.5rem))] overflow-y-auto rounded-lg border border-[color:var(--border-lux)] bg-[color:var(--surface-lux)] p-1.5 text-[color:var(--text)] shadow-[var(--shadow-lux)] ring-1 ring-white/25 backdrop-blur-md dark:ring-white/10 sm:absolute sm:left-0 sm:right-auto sm:top-[calc(100%+0.5rem)] sm:w-[min(21rem,calc(100vw-2rem))]" > {visibleAppModeOptions.map((mode, index) => { diff --git a/src/components/forms/forms-home-page.tsx b/src/components/forms/forms-home-page.tsx index 3b01a409..b53a98b1 100644 --- a/src/components/forms/forms-home-page.tsx +++ b/src/components/forms/forms-home-page.tsx @@ -6,7 +6,6 @@ import { FileQuestion, FileText, Loader2, - Route, Search, ShieldAlert, ShieldCheck, @@ -30,22 +29,22 @@ import { countVerifiedRegistryRecords, useRegistryRecords } from "@/lib/use-regi const taskCards: ModeHomeAction[] = [ { title: "Find a form", - description: "Number, pathway, clock, keyword.", + description: "Title, purpose, or workflow detail.", icon: Search, href: appModeHomeHref("forms", { focus: true }), }, { title: "Readiness checks", - description: "Maker, clock, copies, source.", + description: "Review status, source, and local confirmation.", icon: ClipboardCheck, href: `/forms/${defaultFormSlug() ?? ""}`, }, { - title: "Browse pathways", - description: "Before, current, parallel, after.", - icon: Route, + title: "Check source status", + description: "Find records that still need local confirmation.", + icon: ShieldAlert, href: appModeHomeHref("forms", { - query: "forms pathway before current parallel after", + query: "local confirmation required", focus: true, run: true, }), @@ -91,9 +90,7 @@ export function FormsHomePage() { ) : registry.status === "error" ? ( { - for (const criterion of service.criteria ?? []) { - if (criterion.tone === "meet") total.meets += 1; - if (criterion.tone === "caution") total.cautions += 1; - if (criterion.tone === "reject") total.rejects += 1; - } - const confidence = service.verification?.confidence ?? "Unknown"; - if (confidence === "High") total.high += 1; - else if (confidence === "Medium") total.medium += 1; - else if (confidence === "Low") total.low += 1; - else total.unknown += 1; - if (service.verification?.locallyVerified || (service.source?.status ?? "").toLowerCase().includes("source")) { - total.verified += 1; - } - if ((service.source?.status ?? "").toLowerCase().includes("confirmation")) total.localConfirmation += 1; - return total; - }, - { meets: 0, cautions: 0, rejects: 0, high: 0, medium: 0, low: 0, unknown: 0, verified: 0, localConfirmation: 0 }, - ); -} - function Stepper() { return (
@@ -220,7 +198,7 @@ function ServiceCard({ @@ -339,7 +342,8 @@ function RightRail({ + Edit via result controls
{rows.map(([label, count, Icon, color]) => ( @@ -380,27 +379,61 @@ function RightRail({ ))}
+ {checklistExpanded ? ( +
+ {selected.map((service) => ( +
+

{service.title}

+
    + {(service.criteria ?? []).map((criterion) => ( +
  • • {criterion.label}
    • + ))} +
+
+ ))} +
+ ) : null}

Source confidence

-
- - - - +
+ {confidenceTotal > 0 ? ( + <> + {counts.high ? : null} + {counts.medium ? ( + + ) : null} + {counts.low ? : null} + {counts.unknown ? ( + + ) : null} + + ) : null}
@@ -424,15 +457,68 @@ function RightRail({ {counts.unknown}
+ {confidenceExpanded ? ( +
+ {matches.slice(0, 8).map((service) => ( +
+ {service.title} + + {service.verification?.confidence ?? "Unknown"} + +
+ ))} + {matches.length > 8 ? ( +

+{matches.length - 8} more results

+ ) : null} +
+ ) : null}
+ {comparisonExpanded ? ( +
+ {selected.map((service) => ( +
+
+

{service.title}

+ + Open + +
+
+ {[ + ["Contact", text(service.primaryContact?.value)], + ["Eligibility", text(service.eligibility, "Eligibility pending")], + ["Cost", text(service.cost, "Cost pending")], + ["Source", text(service.source?.status, "Source pending")], + ["Confidence", text(service.verification?.confidence, "Unknown")], + ].map(([label, value]) => ( +
+
{label}
+
{value}
+
+ ))} +
+
+ ))} +
+ ) : null} ); } @@ -519,6 +605,7 @@ export function ServicesNavigatorPage() { } sidebar={ setSelectedSlugs([])} diff --git a/src/lib/rag-candidate-sources.ts b/src/lib/rag-candidate-sources.ts index dd10ade3..80e30acd 100644 --- a/src/lib/rag-candidate-sources.ts +++ b/src/lib/rag-candidate-sources.ts @@ -44,26 +44,13 @@ import type { DocumentIndexUnitMatch, DocumentMemoryCard, SearchResult } from "@ // the floor, which is how the live schema drift (42702) went unnoticed. Log it structurally and, // where telemetry is in scope, record the failing RPC + code so it shows up in rag_retrieval_logs. export type SupabaseRpcError = { message?: string; code?: string; details?: string; hint?: string } | null; - -type AbortableQuery = PromiseLike & { - abortSignal?: (signal: AbortSignal) => PromiseLike; +type RpcResult = Promise<{ data: T | null; error: SupabaseRpcError }>; +type AbortableRpc = RpcResult & { + abortSignal?: (signal: AbortSignal) => RpcResult; +}; +type SupabaseRpcClient = { + rpc: (name: string, rpcArgs: Record) => AbortableRpc | PromiseLike; }; - -function abortReason(signal: AbortSignal): Error { - return signal.reason instanceof Error ? signal.reason : new DOMException("The operation was aborted.", "AbortError"); -} - -function throwIfAborted(signal?: AbortSignal) { - if (signal?.aborted) throw abortReason(signal); -} - -async function resolveQuery(query: AbortableQuery, signal?: AbortSignal): Promise { - throwIfAborted(signal); - const pending = signal && typeof query.abortSignal === "function" ? query.abortSignal(signal) : query; - const result = await pending; - throwIfAborted(signal); - return result; -} function legacyRankFields(versionedName: string) { if (versionedName === "match_document_chunks_v2") return ["similarity"]; @@ -103,15 +90,18 @@ export async function callVersionedRetrievalRpc args: Record, signal?: AbortSignal, ): Promise<{ data: T | null; error: SupabaseRpcError }> { - type RpcResult = { data: T | null; error: SupabaseRpcError }; - const client = supabase as unknown as { - rpc: (name: string, rpcArgs: Record) => AbortableQuery; + const client = supabase as unknown as SupabaseRpcClient; + const executeRpc = async (name: string, rpcArgs: Record) => { + const pending = client.rpc(name, rpcArgs) as AbortableRpc; + const pendingWithAbort = + signal && typeof pending.abortSignal === "function" ? pending.abortSignal(signal) : pending; + return await pendingWithAbort; }; - const versioned = await resolveQuery(client.rpc(versionedName, args), signal); + const versioned = await executeRpc(versionedName, args); if (versioned && !isMissingRetrievalRpcError(versioned.error)) return versioned; const legacyArgs = { ...args }; delete legacyArgs.include_public; - const ownerResult = await resolveQuery(client.rpc(legacyName, legacyArgs), signal); + const ownerResult = await executeRpc(legacyName, legacyArgs); const ownerFilter = String(args.owner_filter ?? ""); if ( ownerResult.error || @@ -121,13 +111,10 @@ export async function callVersionedRetrievalRpc ) { return ownerResult; } - const publicResult = await resolveQuery( - client.rpc(legacyName, { - ...legacyArgs, - owner_filter: PUBLIC_OWNER_FILTER_SENTINEL, - }), - signal, - ); + const publicResult = await executeRpc(legacyName, { + ...legacyArgs, + owner_filter: PUBLIC_OWNER_FILTER_SENTINEL, + }); if (publicResult.error) return publicResult; return { data: mergeLegacyAccessRows( @@ -204,7 +191,6 @@ export async function searchTextChunkCandidates(args: { allowGlobalSearch?: boolean; matchCount: number; telemetry?: SearchTelemetry; - signal?: AbortSignal; }) { const runChunkText = async (queryText: string, matchCount: number) => { const accessScope = retrievalAccessScopeForArgs(args); @@ -218,7 +204,6 @@ export async function searchTextChunkCandidates(args: { document_filters: args.documentIds ?? undefined, ...retrievalRpcScopeArgs(accessScope), }, - args.signal, ); // Report the error before returning empty so a schema drift on this // most-terminal lexical layer surfaces in hybrid_rpc_errors telemetry @@ -275,13 +260,10 @@ export async function searchTextChunkCandidates(args: { const primary = variants[0] ?? ""; let effectivePrimary = primary; if (primary) { - const { data: corrected } = await resolveQuery( - args.supabase.rpc("correct_clinical_query_terms", { - input_query: primary, - min_sim: 0.45, - }), - args.signal, - ); + const { data: corrected } = await args.supabase.rpc("correct_clinical_query_terms", { + input_query: primary, + min_sim: 0.45, + }); if (typeof corrected === "string" && corrected && corrected !== primary) { const correctedResults = await runChunkText(corrected, args.matchCount); if (correctedResults.length > 0) return correctedResults; @@ -406,7 +388,6 @@ async function fetchBestDocumentLookupChunks(args: { ownerId?: string; accessScope?: RetrievalAccessScope; allowGlobalSearch?: boolean; - signal?: AbortSignal; }) { const terms = documentLookupChunkTerms(args.query); const { data: rpcChunks, error: rpcError } = await callVersionedRetrievalRpc( @@ -419,7 +400,6 @@ async function fetchBestDocumentLookupChunks(args: { match_count: Math.max(args.limit * 3, 24), ...retrievalRpcScopeArgs(retrievalAccessScopeForArgs(args)), }, - args.signal, ); if (!rpcError && rpcChunks?.length) { const ranked = (rpcChunks as DocumentLookupChunkRow[]) @@ -450,8 +430,9 @@ async function fetchBestDocumentLookupChunks(args: { .in("document_id", args.documentIds) .limit(Math.max(args.limit * 4, 24)); - const matchedQuery = safeFilters ? baseQuery.or(safeFilters) : baseQuery.order("chunk_index", { ascending: true }); - const { data: matchedChunks, error: matchedError } = await resolveQuery(matchedQuery, args.signal); + const { data: matchedChunks, error: matchedError } = safeFilters + ? await baseQuery.or(safeFilters) + : await baseQuery.order("chunk_index", { ascending: true }); if (!matchedError && matchedChunks?.length) { const ranked = (matchedChunks as DocumentLookupChunkRow[]) @@ -470,7 +451,7 @@ async function fetchBestDocumentLookupChunks(args: { .in("document_id", args.documentIds) .order("chunk_index", { ascending: true }) .limit(args.limit); - const { data: fallbackChunks, error: fallbackError } = await resolveQuery(fallbackQuery, args.signal); + const { data: fallbackChunks, error: fallbackError } = await fallbackQuery; if (fallbackError || !fallbackChunks?.length) return { chunks: [] as DocumentLookupChunkRow[], terms }; return { chunks: fallbackChunks as DocumentLookupChunkRow[], terms }; } @@ -482,7 +463,6 @@ async function fetchDocumentTitleAliasRows(args: { ownerId?: string; accessScope?: RetrievalAccessScope; documentIds?: string[]; - signal?: AbortSignal; }) { const terms = analyzeClinicalQuery(args.query) .documentTitleTerms.map((term) => term.replace(/[%_,]/g, " ").replace(/\s+/g, " ").trim()) @@ -506,7 +486,7 @@ async function fetchDocumentTitleAliasRows(args: { } if (args.documentIds?.length) query = query.in("id", args.documentIds); - const { data, error } = await resolveQuery(query, args.signal); + const { data, error } = await query; if (error || !data?.length) return [] as DocumentLookupRow[]; return (data as DocumentLookupRow[]).map((document) => ({ @@ -531,7 +511,6 @@ export async function searchDocumentLookupFastPath(args: { documentIds?: string[]; matchCount: number; telemetry?: SearchTelemetry; - signal?: AbortSignal; }): Promise { if (!args.ownerId) return [] as SearchResult[]; const variants = (args.queryVariants?.length ? args.queryVariants : [buildClinicalTextSearchQuery(args.query)]).slice( @@ -548,7 +527,6 @@ export async function searchDocumentLookupFastPath(args: { match_count: matchCount, ...retrievalRpcScopeArgs(retrievalAccessScopeForArgs(args)), }, - args.signal, ); if (error) recordHybridRpcError(args.telemetry, "match_documents_for_query", error); if (error || !data?.length) return [] as DocumentLookupRow[]; @@ -567,7 +545,6 @@ export async function searchDocumentLookupFastPath(args: { ownerId: args.ownerId, accessScope: args.accessScope, documentIds: args.documentIds, - signal: args.signal, }); const documentsById = new Map(); for (const document of [...titleAliasDocuments, ...documentSets.flat()]) { @@ -601,7 +578,6 @@ export async function searchDocumentLookupFastPath(args: { limit: Math.max(args.matchCount, rankedDocuments.length * 4), ownerId: args.ownerId, accessScope: args.accessScope, - signal: args.signal, }); if (!chunks.length) return []; @@ -657,7 +633,6 @@ export async function loadChunksForMemoryCards( supabase: ReturnType, cards: DocumentMemoryCard[], accessScope: RetrievalAccessScope, - signal?: AbortSignal, ) { const documentIds = Array.from(new Set(cards.map((card) => card.document_id))).slice(0, 80); if (documentIds.length === 0) return [] as SearchResult[]; @@ -673,7 +648,7 @@ export async function loadChunksForMemoryCards( } else { documentQuery = documentQuery.is("owner_id", null); } - const { data: documents, error: documentsError } = await resolveQuery(documentQuery, signal); + const { data: documents, error: documentsError } = await documentQuery; if (documentsError || !documents?.length) return [] as SearchResult[]; const documentById = new Map(documents.map((document) => [document.id, document])); @@ -684,7 +659,7 @@ export async function loadChunksForMemoryCards( ), ).slice(0, 80); if (chunkIds.length === 0) return [] as SearchResult[]; - const chunksQuery = supabase + const { data: chunks, error: chunksError } = await supabase .from("document_chunks") .select( "id,document_id,page_number,chunk_index,section_heading,section_path,heading_level,parent_heading,anchor_id,content,retrieval_synopsis,image_ids,index_generation_id", @@ -692,7 +667,6 @@ export async function loadChunksForMemoryCards( .in("id", chunkIds) .in("document_id", [...allowedDocumentIds]) .limit(chunkIds.length); - const { data: chunks, error: chunksError } = await resolveQuery(chunksQuery, signal); if (chunksError || !chunks?.length) return [] as SearchResult[]; const bestCardByChunk = new Map(); for (const card of cards) { @@ -821,7 +795,6 @@ export async function loadChunksForSignalMatches(args: { ownerId?: string; accessScope?: RetrievalAccessScope; cache?: ChunkLoadCache; - signal?: AbortSignal; }) { const bestMatchByChunk = new Map(); for (const match of args.matches) { @@ -840,12 +813,11 @@ export async function loadChunksForSignalMatches(args: { ids: chunkIds, scopeKey: cacheScopeKey, fetchRows: async (missingChunkIds) => { - const query = args.supabase + const { data, error } = await args.supabase .from("document_chunks") .select("id,document_id") .in("id", missingChunkIds) .limit(missingChunkIds.length); - const { data, error } = await resolveQuery(query, args.signal); return { data: data as ChunkScopeRow[] | null, error }; }, }); @@ -870,7 +842,7 @@ export async function loadChunksForSignalMatches(args: { } else { documentQuery = documentQuery.is("owner_id", null); } - const { data, error } = await resolveQuery(documentQuery, args.signal); + const { data, error } = await documentQuery; return { data: data as HydratedDocumentRow[] | null, error }; }, }); @@ -889,7 +861,7 @@ export async function loadChunksForSignalMatches(args: { ids: allowedChunkIds, scopeKey: cacheScopeKey, fetchRows: async (missingAllowedChunkIds) => { - const query = args.supabase + const { data, error } = await args.supabase .from("document_chunks") .select( "id,document_id,page_number,chunk_index,section_heading,section_path,heading_level,parent_heading,anchor_id,content,retrieval_synopsis,image_ids,index_generation_id", @@ -897,7 +869,6 @@ export async function loadChunksForSignalMatches(args: { .in("id", missingAllowedChunkIds) .in("document_id", [...allowedDocumentIds]) .limit(missingAllowedChunkIds.length); - const { data, error } = await resolveQuery(query, args.signal); return { data: data as HydratedChunkRow[] | null, error }; }, }); @@ -971,7 +942,6 @@ export async function searchTableFactCandidates(args: { matchCount: number; telemetry?: SearchTelemetry; cache?: ChunkLoadCache; - signal?: AbortSignal; }) { const variants = (args.queryVariants?.length ? args.queryVariants : [buildClinicalTextSearchQuery(args.query)]).slice( 0, @@ -988,7 +958,6 @@ export async function searchTableFactCandidates(args: { document_filters: args.documentIds ?? undefined, ...retrievalRpcScopeArgs(retrievalAccessScopeForArgs(args)), }, - args.signal, ); if (error) recordHybridRpcError(args.telemetry, "match_document_table_facts_text", error); if (error || !data?.length) return [] as TableFactRpcRow[]; @@ -1030,7 +999,6 @@ export async function searchTableFactCandidates(args: { ownerId: args.ownerId, accessScope: args.accessScope, cache: args.cache, - signal: args.signal, }); } @@ -1046,7 +1014,6 @@ export async function searchEmbeddingFieldCandidates(args: { matchCount: number; telemetry?: SearchTelemetry; cache?: ChunkLoadCache; - signal?: AbortSignal; }) { const { data, error } = await callVersionedRetrievalRpc( args.supabase, @@ -1060,7 +1027,6 @@ export async function searchEmbeddingFieldCandidates(args: { document_filters: args.documentIds ?? undefined, ...retrievalRpcScopeArgs(retrievalAccessScopeForArgs(args)), }, - args.signal, ); if (error) recordHybridRpcError(args.telemetry, "match_document_embedding_fields_hybrid", error); if (error || !data?.length) return [] as SearchResult[]; @@ -1098,7 +1064,6 @@ export async function searchEmbeddingFieldCandidates(args: { ownerId: args.ownerId, accessScope: args.accessScope, cache: args.cache, - signal: args.signal, }); } @@ -1114,7 +1079,6 @@ export async function searchIndexUnitCandidates(args: { matchCount: number; telemetry?: SearchTelemetry; cache?: ChunkLoadCache; - signal?: AbortSignal; }) { const { data, error } = await callVersionedRetrievalRpc( args.supabase, @@ -1128,7 +1092,6 @@ export async function searchIndexUnitCandidates(args: { document_filters: args.documentIds ?? undefined, ...retrievalRpcScopeArgs(retrievalAccessScopeForArgs(args)), }, - args.signal, ); if (error) recordHybridRpcError(args.telemetry, "match_document_index_units_hybrid", error); if (error || !data?.length) return [] as SearchResult[]; @@ -1168,7 +1131,6 @@ export async function searchIndexUnitCandidates(args: { ownerId: args.ownerId, accessScope: args.accessScope, cache: args.cache, - signal: args.signal, }); } @@ -1185,7 +1147,6 @@ export async function withMemoryBoostedCandidates(args: { documentIds?: string[]; matchCount: number; cardCache?: MemoryCardCache; - signal?: AbortSignal; }) { // A3: the memory-card fetch is invoked at several waterfall stages. Memoize per request, // scoped by owner/document filters because fetchMemoryCardsForQuery applies those filters. @@ -1208,19 +1169,13 @@ export async function withMemoryBoostedCandidates(args: { accessScope: args.accessScope, documentIds: args.documentIds, matchCount: effectiveMatchCount, - signal: args.signal, }); args.cardCache?.set(cacheKey, cardsPromise); } const cards = await cardsPromise; if (cards.length === 0) return { results: args.candidates, cards }; - const memoryChunkResults = await loadChunksForMemoryCards( - args.supabase, - cards, - retrievalAccessScopeForArgs(args), - args.signal, - ); + const memoryChunkResults = await loadChunksForMemoryCards(args.supabase, cards, retrievalAccessScopeForArgs(args)); const merged = mergeSearchResults(memoryChunkResults, args.candidates); return { results: applyMemoryCardBoosts(args.query, merged, cards), diff --git a/src/lib/rag.ts b/src/lib/rag.ts index 03fa920c..3e9e5a76 100644 --- a/src/lib/rag.ts +++ b/src/lib/rag.ts @@ -2,6 +2,7 @@ import { createAdminClient } from "@/lib/supabase/admin"; import { retrievalAccessScopeForArgs, retrievalRpcScopeArgs } from "@/lib/owner-scope"; import { callVersionedRetrievalRpc, + createChunkLoadCache, memoryCardChunkScore, mergeSearchResults, recordHybridRpcError, @@ -12,7 +13,6 @@ import { searchTextChunkCandidates, withMemoryBoostedCandidates, type MemoryCardCache, - createChunkLoadCache, } from "@/lib/rag-candidate-sources"; export { callVersionedRetrievalRpc, @@ -422,31 +422,30 @@ const confidenceOrder = { } as const; /** Throw if aborted. */ -function abortReason(signal: AbortSignal): Error { - return signal.reason instanceof Error ? signal.reason : new DOMException("The operation was aborted.", "AbortError"); -} - function throwIfAborted(signal?: AbortSignal) { if (signal?.aborted) { - throw abortReason(signal); + throw signal.reason ?? new DOMException("The operation was aborted.", "AbortError"); } } -async function awaitWithAbortSignal(pending: Promise, signal?: AbortSignal): Promise { +function awaitWithCallerSignal(pending: Promise, signal?: AbortSignal): Promise { if (!signal) return pending; - throwIfAborted(signal); + if (signal.aborted) throw signal.reason ?? new DOMException("The operation was aborted.", "AbortError"); - let onAbort: (() => void) | undefined; - const aborted = new Promise((_resolve, reject) => { - onAbort = () => reject(abortReason(signal)); + return new Promise((resolve, reject) => { + const onAbort = () => reject(signal.reason ?? new DOMException("The operation was aborted.", "AbortError")); signal.addEventListener("abort", onAbort, { once: true }); - if (signal.aborted) onAbort(); + pending.then( + (value) => { + signal.removeEventListener("abort", onAbort); + resolve(value); + }, + (error) => { + signal.removeEventListener("abort", onAbort); + reject(error); + }, + ); }); - try { - return await Promise.race([pending, aborted]); - } finally { - if (onAbort) signal.removeEventListener("abort", onAbort); - } } export type AnswerProgressEvent = { @@ -1317,7 +1316,6 @@ export async function analyzeQueryWithClassifierFallback( corpusGrounding?: { supabase: ReturnType; ownerFilter: string | null }; ownerId?: string | null; signal?: AbortSignal; - skipClassifier?: boolean; }, ) { if ( @@ -1346,7 +1344,6 @@ export async function analyzeQueryWithClassifierFallback( supabase: opts.corpusGrounding.supabase, query, ownerFilter: opts.corpusGrounding.ownerFilter, - signal: opts.signal, }); if (grounding.verdict === "in_corpus_topic") { return { @@ -1372,8 +1369,7 @@ export async function analyzeQueryWithClassifierFallback( analysis = { ...analysis, corpusGrounding: "inconclusive" }; } - if (!analysis.needsClassifierFallback || !env.OPENAI_API_KEY || opts?.skipClassifier) return analysis; - throwIfAborted(opts?.signal); + if (!analysis.needsClassifierFallback || !env.OPENAI_API_KEY) return analysis; const memoKey = classifierVerdictMemoKey(query, analysis); const memoized = classifierVerdictMemo.get(memoKey); @@ -1391,11 +1387,16 @@ export async function analyzeQueryWithClassifierFallback( } try { - const verdict = await awaitWithAbortSignal(pending, opts?.signal); + const verdict = await awaitWithCallerSignal(pending, opts?.signal); storeClassifierVerdictMemo(memoKey, verdict); return applyClassifierVerdict(analysis, verdict); - } catch { - if (opts?.signal?.aborted) throw abortReason(opts.signal); + } catch (error) { + if ( + error && + (error instanceof DOMException || typeof error === "object") && + (error as { name?: string }).name === "AbortError" + ) + throw error; // Transport/parse failures are deliberately NOT memoized: fall back to the deterministic // analysis for this request only, and let the next request retry the classifier. return analysis; @@ -1544,7 +1545,6 @@ export async function attachDocumentRankingMetadata( results: SearchResult[], ownerId?: string, cache = createDocumentRankingMetadataCache(), - signal?: AbortSignal, ) { const documentIds = Array.from(new Set(results.map((result) => result.document_id))); if (documentIds.length === 0) return results; @@ -1574,7 +1574,7 @@ export async function attachDocumentRankingMetadata( document_summary: metadata.summary, }; }); - return attachIndexQualityMetadata(supabase, enriched, ownerId, cache, signal); + return attachIndexQualityMetadata(supabase, enriched, ownerId, cache); } const [metadataRows, indexedResults] = await Promise.all([ @@ -1582,12 +1582,8 @@ export async function attachDocumentRankingMetadata( supabase, ownerId, documentIds: missingDocumentIds, - signal, - }).catch(() => { - if (signal?.aborted) throw abortReason(signal); - return null; - }), - attachIndexQualityMetadata(supabase, results, ownerId, cache, signal), + }).catch(() => null), + attachIndexQualityMetadata(supabase, results, ownerId, cache), ]); if (!metadataRows) return indexedResults; @@ -1624,7 +1620,6 @@ async function attachIndexQualityMetadata( results: SearchResult[], ownerId?: string, cache = createDocumentRankingMetadataCache(), - signal?: AbortSignal, ): Promise { const documentIds = Array.from(new Set(results.map((result) => result.document_id))); if (documentIds.length === 0) return results; @@ -1636,15 +1631,12 @@ async function attachIndexQualityMetadata( .select("document_id,owner_id,quality_score,extraction_quality,metrics,issues,updated_at") .in("document_id", missingDocumentIds); if (ownerId) query = query.eq("owner_id", ownerId); - if (signal) query = query.abortSignal(signal); const { data, error } = await query; - throwIfAborted(signal); if (error) return results; for (const documentId of missingDocumentIds) cache.indexQuality.set(documentId, null); for (const row of data ?? []) cache.indexQuality.set(row.document_id, row as SearchResult["indexing_quality"]); return withCachedIndexQuality(results, cache); } catch { - if (signal?.aborted) throw abortReason(signal); return results; } } @@ -1653,7 +1645,6 @@ async function attachIndexQualityMetadata( export async function attachPageVisualEvidence( supabase: ReturnType, results: SearchResult[], - signal?: AbortSignal, ): Promise { const documentIds = Array.from(new Set(results.map((result) => result.document_id))); const pageNumbers = Array.from( @@ -1673,7 +1664,7 @@ export async function attachPageVisualEvidence( const selectColumns = "id,document_id,page_number,storage_path,caption,bbox,image_type,searchable,clinical_relevance_score,source_kind,width,height,labels,metadata"; - const pageQuery = + const [pageData, directData] = await Promise.all([ pageNumbers.length > 0 ? supabase .from("document_images") @@ -1684,8 +1675,7 @@ export async function attachPageVisualEvidence( .neq("image_type", "logo_decorative") .order("clinical_relevance_score", { ascending: false }) .limit(80) - : null; - const directQuery = + : Promise.resolve({ data: [], error: null }), sourceImageIds.length > 0 ? supabase .from("document_images") @@ -1694,20 +1684,8 @@ export async function attachPageVisualEvidence( .eq("searchable", true) .neq("image_type", "logo_decorative") .limit(sourceImageIds.length) - : null; - const [pageData, directData] = await Promise.all([ - pageQuery - ? signal && typeof pageQuery.abortSignal === "function" - ? pageQuery.abortSignal(signal) - : pageQuery - : Promise.resolve({ data: [], error: null }), - directQuery - ? signal && typeof directQuery.abortSignal === "function" - ? directQuery.abortSignal(signal) - : directQuery : Promise.resolve({ data: [], error: null }), ]); - throwIfAborted(signal); const data = [...(pageData.data ?? []), ...(directData.data ?? [])]; if ((pageData.error && directData.error) || data.length === 0) return results; @@ -2279,8 +2257,6 @@ async function prepareCoverageGateResults(args: { queryClass: RagQueryClass; telemetry: SearchTelemetry; metadataCache: DocumentRankingMetadataCache; - includeVisualEvidence?: boolean; - signal?: AbortSignal; }) { const startedAt = Date.now(); const candidates = await attachDocumentRankingMetadata( @@ -2288,20 +2264,18 @@ async function prepareCoverageGateResults(args: { args.candidates, args.ownerId, args.metadataCache, - args.signal, ); - const rankedResults = selectRankedRetrievalResults({ - query: args.query, - queryClass: args.queryClass, - candidates, - topK: args.topK, - maxResultsPerDocument: args.maxResultsPerDocument, - telemetry: args.telemetry, - }); - let results = - args.includeVisualEvidence === false - ? rankedResults - : await attachPageVisualEvidence(args.supabase, rankedResults, args.signal); + let results = await attachPageVisualEvidence( + args.supabase, + selectRankedRetrievalResults({ + query: args.query, + queryClass: args.queryClass, + candidates, + topK: args.topK, + maxResultsPerDocument: args.maxResultsPerDocument, + telemetry: args.telemetry, + }), + ); results = applySecondStageRerankIfNeeded({ queryClass: args.queryClass, results, @@ -2420,8 +2394,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { } const indexingVersionAtRetrievalStart = await cacheIndexingVersion(args, { forceRefresh: true }); const supabase = createAdminClient(); - const attachSearchVisualEvidence = (results: SearchResult[]) => - args.lexicalOnly ? Promise.resolve(results) : attachPageVisualEvidence(supabase, results, args.signal); // When the provider is source-only (offline mode, or auto mode without a usable key) we must // never call OpenAI for embeddings; retrieval falls back to the lexical text-fast-path only. const sourceOnlyRetrieval = isSourceOnlyMode(); @@ -2431,8 +2403,8 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { // A3: shared across every withMemoryBoostedCandidates call in this request so the same // owner/query memory cards are fetched at most once per (query, embedding-present, count). const memoryCardCache: MemoryCardCache = new Map(); - const documentRankingMetadataCache = createDocumentRankingMetadataCache(); const chunkLoadCache = createChunkLoadCache(); + const documentRankingMetadataCache = createDocumentRankingMetadataCache(); const modeQueryClass = queryClassForClinicalMode(args.queryMode ?? "auto"); const documentFilterList = args.documentIds?.length ? args.documentIds @@ -2459,7 +2431,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { corpusGrounding: corpusGroundingScope, ownerId: args.ownerId, signal: args.signal, - skipClassifier: args.lexicalOnly, }); throwIfAborted(args.signal); if (modeQueryClass) queryAnalysis.queryClass = modeQueryClass; @@ -2471,7 +2442,7 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { const telemetry = createSearchTelemetry(retrievalQuery, queryClassification.queryClass); if (queryAnalysis.corpusGrounding) telemetry.corpus_grounding = queryAnalysis.corpusGrounding; - const ragAliases = await fetchEnabledRagAliases(supabase, args.ownerId, args.accessScope, args.signal); + const ragAliases = await fetchEnabledRagAliases(supabase, args.ownerId, args.accessScope); const ragAliasExpansions = selectRagAliasExpansions(retrievalQuery, ragAliases); telemetry.rag_alias_count = ragAliases.length; telemetry.rag_alias_expansion_count = ragAliasExpansions.length; @@ -2511,13 +2482,10 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { // in searchTextChunkCandidates). Only reached for would-be-unsupported queries, so it adds no // hot-path cost; `typoCorrected` guards against recursion. if (!args.typoCorrected && !sourceOnlyRetrieval) { - let correctionQuery = supabase.rpc("correct_clinical_query_terms", { + const { data: corrected } = await supabase.rpc("correct_clinical_query_terms", { input_query: retrievalQuery, min_sim: 0.45, }); - if (args.signal) correctionQuery = correctionQuery.abortSignal(args.signal); - const { data: corrected } = await correctionQuery; - throwIfAborted(args.signal); if (typeof corrected === "string" && corrected && corrected.toLowerCase() !== retrievalQuery.toLowerCase()) { return searchChunksWithTelemetry({ ...args, query: corrected, typoCorrected: true }); } @@ -2552,7 +2520,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { allowGlobalSearch: args.allowGlobalSearch, matchCount: textCandidateCount, telemetry, - signal: args.signal, }); telemetry.text_candidate_count = textData.length; telemetry.text_fast_path_latency_ms = Date.now() - textRpcStartedAt; @@ -2570,7 +2537,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { textData as SearchResult[], args.ownerId, documentRankingMetadataCache, - args.signal, ); expandedQuery = expandClinicalQueryWithCandidateMetadata(args.query, expandedQuery, textCandidates); const baseTextResults = selectRankedRetrievalResults({ @@ -2584,7 +2550,7 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { const baseTextFastPath = decideTextFastPath(args.query, baseTextResults, queryClassification.queryClass); if (!args.forceEmbedding && shouldReturnBeforeMemory(queryClassification.queryClass, baseTextFastPath)) { - textFastResults = await attachSearchVisualEvidence(baseTextResults); + textFastResults = await attachPageVisualEvidence(supabase, baseTextResults); textFastResults = applySecondStageRerankIfNeeded({ queryClass: queryClassification.queryClass, results: textFastResults, @@ -2608,7 +2574,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { documentIds: documentFilterList, matchCount: candidateCount, cardCache: memoryCardCache, - signal: args.signal, }); telemetry.memory_card_count = Math.max(telemetry.memory_card_count ?? 0, memoryBoost.cards.length); telemetry.memory_top_score = Math.max( @@ -2626,7 +2591,7 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { maxResultsPerDocument, telemetry, }); - textFastResults = await attachSearchVisualEvidence(textFastResults); + textFastResults = await attachPageVisualEvidence(supabase, textFastResults); textFastResults = applySecondStageRerankIfNeeded({ queryClass: queryClassification.queryClass, results: textFastResults, @@ -2661,7 +2626,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { matchCount: Math.min(candidateCount, 48), telemetry, cache: chunkLoadCache, - signal: args.signal, }); const tableFactLatencyMs = Date.now() - tableFactStartedAt; telemetry.supabase_rpc_latency_ms += tableFactLatencyMs; @@ -2685,7 +2649,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { documentIds: documentFilterList, matchCount: candidateCount, telemetry, - signal: args.signal, }); const documentLookupLatencyMs = Date.now() - documentLookupStartedAt; telemetry.supabase_rpc_latency_ms += documentLookupLatencyMs; @@ -2701,7 +2664,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { mergeSearchResults(documentLookupData, textFastResults), args.ownerId, documentRankingMetadataCache, - args.signal, ); expandedQuery = expandClinicalQueryWithCandidateMetadata(args.query, expandedQuery, documentLookupCandidates); const memoryBoost = await withMemoryBoostedCandidates({ @@ -2713,7 +2675,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { documentIds: documentFilterList, matchCount: candidateCount, cardCache: memoryCardCache, - signal: args.signal, }); telemetry.memory_card_count = Math.max(telemetry.memory_card_count ?? 0, memoryBoost.cards.length); telemetry.memory_top_score = Math.max( @@ -2728,7 +2689,8 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { topScore: Math.max(telemetry.memory_top_score ?? 0, ...memoryBoost.cards.map(memoryCardChunkScore)), }, ); - let documentLookupResults = await attachSearchVisualEvidence( + let documentLookupResults = await attachPageVisualEvidence( + supabase, selectRankedRetrievalResults({ query: retrievalQuery, queryClass: queryClassification.queryClass, @@ -2778,8 +2740,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { queryClass: queryClassification.queryClass, telemetry, metadataCache: documentRankingMetadataCache, - includeVisualEvidence: !args.lexicalOnly, - signal: args.signal, }); const coverageGate = evaluateEvidenceCoverageGate(args.query, coverageGateResults, queryClassification.queryClass); applyCoverageGateTelemetry(telemetry, coverageGate, !args.forceEmbedding && coverageGate.accepted); @@ -2853,7 +2813,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { matchCount: Math.min(candidateCount, 48), telemetry, cache: chunkLoadCache, - signal: args.signal, }); return { candidates, latencyMs: Date.now() - startedAt }; })(), @@ -2870,7 +2829,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { matchCount: Math.min(candidateCount, 64), telemetry, cache: chunkLoadCache, - signal: args.signal, }); return { candidates, latencyMs: Date.now() - startedAt }; })(), @@ -2888,7 +2846,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { document_filters: documentFilterList ?? undefined, ...retrievalRpcScopeArgs(retrievalAccessScopeForArgs(args)), }, - args.signal, ); return { data, error, latencyMs: Date.now() - startedAt }; })(), @@ -2939,7 +2896,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { merged, args.ownerId, documentRankingMetadataCache, - args.signal, ); const memoryBoost = await withMemoryBoostedCandidates({ supabase, @@ -2951,7 +2907,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { documentIds: documentFilterList, matchCount: candidateCount, cardCache: memoryCardCache, - signal: args.signal, }); telemetry.memory_card_count = Math.max(telemetry.memory_card_count ?? 0, memoryBoost.cards.length); telemetry.memory_top_score = Math.max( @@ -2968,7 +2923,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { maxResultsPerDocument, telemetry, }), - args.signal, ); results = applySecondStageRerankIfNeeded({ queryClass: queryClassification.queryClass, @@ -2999,7 +2953,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { document_filter: documentFilter ?? undefined, ...retrievalRpcScopeArgs(retrievalAccessScopeForArgs(args)), }, - args.signal, ); if (error) throw new Error(error.message); @@ -3027,7 +2980,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { args.forceEmbedding ? fallbackVectorCandidates : mergeSearchResults(fallbackVectorCandidates, textFastResults), args.ownerId, documentRankingMetadataCache, - args.signal, ); const memoryBoost = await withMemoryBoostedCandidates({ supabase, @@ -3039,7 +2991,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { documentIds: documentFilterList, matchCount: candidateCount, cardCache: memoryCardCache, - signal: args.signal, }); telemetry.memory_card_count = Math.max(telemetry.memory_card_count ?? 0, memoryBoost.cards.length); telemetry.memory_top_score = Math.max( @@ -3056,7 +3007,6 @@ export async function searchChunksWithTelemetry(args: SearchChunksArgs) { maxResultsPerDocument, telemetry, }), - args.signal, ); results = applySecondStageRerankIfNeeded({ queryClass: queryClassification.queryClass, diff --git a/supabase/drift-manifest.json b/supabase/drift-manifest.json index da865fcb..40ebc996 100644 --- a/supabase/drift-manifest.json +++ b/supabase/drift-manifest.json @@ -1,9 +1,9 @@ { - "generated_at": "2026-07-17T15:00:51.747Z", + "generated_at": "2026-07-17T22:10:14.366Z", "generator": "scripts/generate-drift-manifest.ts", "postgres_image": "supabase/postgres:17.6.1.127", - "schema_sha256": "bde3b7c60872a8c4493555e6670c30ac777a862a12567ef2091dfc050c86240e", - "replay_seconds": 77, + "schema_sha256": "302a9ecec8e69564d50035be8480fa5e5686ca6136d96978f2188ecf5fb58be1", + "replay_seconds": 13, "snapshot": { "views": [ { diff --git a/supabase/migrations/20260714180000_patch_rag_and_corrector_scalability.sql b/supabase/migrations/20260714180000_patch_rag_and_corrector_scalability.sql new file mode 100644 index 00000000..6367dcfa --- /dev/null +++ b/supabase/migrations/20260714180000_patch_rag_and_corrector_scalability.sql @@ -0,0 +1,144 @@ +-- Cascade deletion trigger for registry records to clean up RAG corpus documents +create or replace function public.cleanup_registry_corpus_document() +returns trigger +language plpgsql +security definer +set search_path = public, pg_catalog, pg_temp +as $$ +begin + delete from public.documents + where metadata->>'source_kind' = 'registry_record' + and (metadata->>'registry_record_id')::uuid = OLD.id; + return OLD; +end; +$$; + +drop trigger if exists clinical_registry_records_delete_cleanup on public.clinical_registry_records; +create trigger clinical_registry_records_delete_cleanup + after delete on public.clinical_registry_records + for each row execute function public.cleanup_registry_corpus_document(); + +drop trigger if exists medication_records_delete_cleanup on public.medication_records; +create trigger medication_records_delete_cleanup + after delete on public.medication_records + for each row execute function public.cleanup_registry_corpus_document(); + +drop trigger if exists differential_records_delete_cleanup on public.differential_records; +create trigger differential_records_delete_cleanup + after delete on public.differential_records + for each row execute function public.cleanup_registry_corpus_document(); + + +-- Scalable Spelling Corrector Vocabulary Indexing +create table if not exists public.document_title_words ( + word text not null, + document_id uuid not null references public.documents(id) on delete cascade, + primary key (word, document_id) +); + +create index if not exists document_title_words_word_trgm_idx + on public.document_title_words using gin (word extensions.gin_trgm_ops); + +alter table public.document_title_words enable row level security; +revoke all on public.document_title_words from anon, authenticated; +grant select, insert, update, delete on table public.document_title_words to service_role; + +-- Populate table from existing documents +insert into public.document_title_words (word, document_id) +select distinct lower(w), d.id +from public.documents d, + lateral unnest(regexp_split_to_array(lower(d.title), '[^a-z]+')) as w +where d.status = 'indexed' + and length(w) between 4 and 40 +on conflict do nothing; + +-- Sync trigger on documents to keep title words vocabulary updated +create or replace function public.sync_document_title_words() +returns trigger +language plpgsql +security definer +set search_path = public, pg_catalog, pg_temp +as $$ +begin + if TG_OP = 'DELETE' or (TG_OP = 'UPDATE' and OLD.status = 'indexed' and NEW.status <> 'indexed') then + delete from public.document_title_words where document_id = OLD.id; + end if; + + if (TG_OP = 'INSERT' and NEW.status = 'indexed') or + (TG_OP = 'UPDATE' and NEW.status = 'indexed' and (OLD.status <> 'indexed' or OLD.title <> NEW.title)) then + + if TG_OP = 'UPDATE' then + delete from public.document_title_words where document_id = NEW.id; + end if; + + insert into public.document_title_words (word, document_id) + select distinct lower(w), NEW.id + from unnest(regexp_split_to_array(lower(NEW.title), '[^a-z]+')) as w + where length(w) between 4 and 40 + on conflict do nothing; + end if; + + return null; +end; +$$; + +drop trigger if exists documents_sync_title_words on public.documents; +create trigger documents_sync_title_words + after insert or update or delete on public.documents + for each row execute function public.sync_document_title_words(); + +-- Optimize spelling corrector to query index table +CREATE OR REPLACE FUNCTION public.correct_clinical_query_terms(input_query text, min_sim real DEFAULT 0.45) + RETURNS text + LANGUAGE plpgsql + STABLE SECURITY DEFINER + SET search_path TO 'public', 'extensions', 'pg_temp' +AS $function$ +declare + vocab text[]; + tokens text[]; + tok text; + best text; + best_sim real; + corrected text[] := array[]::text[]; + changed boolean := false; +begin + if input_query is null or length(trim(input_query)) = 0 then + return input_query; + end if; + + select array_agg(distinct term) into vocab + from ( + select lower(alias) as term from public.rag_aliases where enabled and length(alias) between 4 and 40 + union + select lower(canonical) from public.rag_aliases where enabled and length(canonical) between 4 and 40 + union + select word from public.document_title_words where length(word) between 4 and 40 + ) t; + + tokens := regexp_split_to_array(lower(input_query), '\s+'); + foreach tok in array tokens loop + if length(tok) < 4 or tok = any(vocab) then + corrected := corrected || tok; + continue; + end if; + best := null; + best_sim := 0; + select v, similarity(v, tok) into best, best_sim + from unnest(vocab) as v + order by similarity(v, tok) desc + limit 1; + if best is not null and best_sim >= min_sim and best <> tok and length(best) >= length(tok) then + corrected := corrected || best; + changed := true; + else + corrected := corrected || tok; + end if; + end loop; + + if changed then + return array_to_string(corrected, ' '); + end if; + return input_query; +end; +$function$; diff --git a/supabase/migrations/20260714190000_document_table_facts_trgm_idx.sql b/supabase/migrations/20260714190000_document_table_facts_trgm_idx.sql new file mode 100644 index 00000000..c0523bf6 --- /dev/null +++ b/supabase/migrations/20260714190000_document_table_facts_trgm_idx.sql @@ -0,0 +1,11 @@ +-- Migration to add GIN trigram index on document_table_facts text fields for performance optimization +create index if not exists document_table_facts_text_trgm_idx + on public.document_table_facts using gin ( + lower( + coalesce(table_title, '') || ' ' || + coalesce(row_label, '') || ' ' || + coalesce(clinical_parameter, '') || ' ' || + coalesce(threshold_value, '') || ' ' || + coalesce(action, '') + ) extensions.gin_trgm_ops + ); diff --git a/supabase/migrations/20260717010000_harden_rag_scalability_patch.sql b/supabase/migrations/20260717010000_harden_rag_scalability_patch.sql new file mode 100644 index 00000000..1a6e7424 --- /dev/null +++ b/supabase/migrations/20260717010000_harden_rag_scalability_patch.sql @@ -0,0 +1,116 @@ +-- Forward-only hardening for the registry cleanup and RAG scalability WIP. +-- This intentionally corrects the earlier July 14 migrations without assuming +-- whether either version has already been applied in an external environment. + +create index if not exists rag_aliases_canonical_trgm_idx + on public.rag_aliases using gin (lower(canonical) extensions.gin_trgm_ops); + +create or replace function public.cleanup_registry_corpus_document() +returns trigger +language plpgsql +security definer +set search_path = public, pg_catalog, pg_temp +as $$ +begin + delete from public.documents + where metadata->>'source_kind' = 'registry_record' + and metadata->>'registry_record_id' = OLD.id::text + and metadata->>'registry_record_kind' = case TG_TABLE_NAME + when 'clinical_registry_records' then to_jsonb(OLD)->>'kind' + when 'medication_records' then 'medication' + when 'differential_records' then 'differential' + else null + end; + return OLD; +end; +$$; + +revoke execute on function public.cleanup_registry_corpus_document() from public, anon, authenticated; +revoke execute on function public.sync_document_title_words() from public, anon, authenticated; + +create or replace function public.correct_clinical_query_terms(input_query text, min_sim real default 0.45) +returns text +language plpgsql +stable security definer +set search_path to 'public', 'extensions', 'pg_temp' +set pg_trgm.similarity_threshold = 0.3 +as $$ +declare + tokens text[]; + tok text; + best text; + best_sim real; + corrected text[] := array[]::text[]; + changed boolean := false; +begin + if min_sim is null or min_sim < 0.3 or min_sim > 1 then + raise exception 'min_sim must be between 0.3 and 1.0' using errcode = '22023'; + end if; + + if input_query is null or length(trim(input_query)) = 0 then + return input_query; + end if; + + tokens := regexp_split_to_array(lower(input_query), '\s+'); + foreach tok in array tokens loop + if length(tok) < 4 then + corrected := corrected || tok; + continue; + end if; + + best := null; + best_sim := 0; + select candidate.term, similarity(candidate.term, tok) + into best, best_sim + from ( + ( + select lower(alias) as term + from public.rag_aliases + where enabled + and length(alias) between 4 and 40 + and lower(alias) % tok + order by similarity(lower(alias), tok) desc, lower(alias) + limit 32 + ) + union all + ( + select lower(canonical) as term + from public.rag_aliases + where enabled + and length(canonical) between 4 and 40 + and lower(canonical) % tok + order by similarity(lower(canonical), tok) desc, lower(canonical) + limit 32 + ) + union all + ( + select word as term + from public.document_title_words + where length(word) between 4 and 40 + and word % tok + order by similarity(word, tok) desc, word + limit 32 + ) + ) candidate + order by similarity(candidate.term, tok) desc, candidate.term + limit 1; + + if best is not null and best_sim >= min_sim and best <> tok and length(best) >= length(tok) then + corrected := corrected || best; + changed := true; + else + corrected := corrected || tok; + end if; + end loop; + + if not changed then + return input_query; + end if; + return array_to_string(corrected, ' '); +end; +$$; + +revoke execute on function public.correct_clinical_query_terms(text, real) from public, anon, authenticated; +grant execute on function public.correct_clinical_query_terms(text, real) to service_role; + +drop index if exists public.document_table_facts_text_trgm_idx; diff --git a/supabase/schema.sql b/supabase/schema.sql index b3f440e2..dda7b4d9 100644 --- a/supabase/schema.sql +++ b/supabase/schema.sql @@ -848,6 +848,8 @@ create index if not exists rag_aliases_type_enabled_idx on public.rag_aliases(alias_type, enabled); create index if not exists rag_aliases_alias_trgm_idx on public.rag_aliases using gin (lower(alias) gin_trgm_ops); +create index if not exists rag_aliases_canonical_trgm_idx + on public.rag_aliases using gin (lower(canonical) gin_trgm_ops); create index if not exists rag_response_cache_expiry_idx on public.rag_response_cache(expires_at); create index if not exists rag_response_cache_owner_kind_idx @@ -3473,16 +3475,21 @@ CREATE OR REPLACE FUNCTION public.correct_clinical_query_terms(input_query text, LANGUAGE plpgsql STABLE SECURITY DEFINER SET search_path TO 'public', 'extensions', 'pg_temp' + SET pg_trgm.similarity_threshold = 0.3 AS $function$ declare - vocab text[]; tokens text[]; tok text; best text; best_sim real; + vocab text[]; corrected text[] := array[]::text[]; changed boolean := false; begin + if min_sim is null or min_sim < 0.3 or min_sim > 1 then + raise exception 'min_sim must be between 0.3 and 1.0' using errcode = '22023'; + end if; + if input_query is null or length(trim(input_query)) = 0 then return input_query; end if; @@ -3504,15 +3511,45 @@ begin tokens := regexp_split_to_array(lower(input_query), '\s+'); foreach tok in array tokens loop - if length(tok) < 4 or tok = any(vocab) then + if length(tok) < 4 then corrected := corrected || tok; continue; end if; best := null; best_sim := 0; - select v, similarity(v, tok) into best, best_sim - from unnest(vocab) as v - order by similarity(v, tok) desc + select candidate.term, similarity(candidate.term, tok) + into best, best_sim + from ( + ( + select lower(alias) as term + from public.rag_aliases + where enabled + and length(alias) between 4 and 40 + and lower(alias) % tok + order by similarity(lower(alias), tok) desc, lower(alias) + limit 32 + ) + union all + ( + select lower(canonical) as term + from public.rag_aliases + where enabled + and length(canonical) between 4 and 40 + and lower(canonical) % tok + order by similarity(lower(canonical), tok) desc, lower(canonical) + limit 32 + ) + union all + ( + select word as term + from public.document_title_words + where length(word) between 4 and 40 + and word % tok + order by similarity(word, tok) desc, word + limit 32 + ) + ) candidate + order by similarity(candidate.term, tok) desc, candidate.term limit 1; if best is not null and best_sim >= min_sim and best <> tok and length(best) >= length(tok) then corrected := corrected || best; diff --git a/tests/audit-content-services-regressions.test.ts b/tests/audit-content-services-regressions.test.ts index 2ebb23e9..df3d1fda 100644 --- a/tests/audit-content-services-regressions.test.ts +++ b/tests/audit-content-services-regressions.test.ts @@ -72,9 +72,8 @@ describe("content and services audit regressions", () => { expect(canCompareServices(records.slice(0, 1))).toBe(false); expect(canCompareServices(records.slice(0, 2))).toBe(true); expect(normalizedServiceNavigatorSource).toContain( - "effectiveSelectedSlugs = selectedSlugs ?? searchableRecords.slice(0, 2)", + 'key={selected.length === 0 ? "empty" : selected.length === 1 ? "single" : "multiple"}', ); - expect(normalizedServiceNavigatorSource).toContain("Compare selected ({selected.length})"); expect(serviceNavigatorSource).not.toContain("useEffect("); expect(serviceNavigatorSource).toContain("aria-pressed={selected}"); expect(serviceNavigatorSource).toContain("Add ${service.title} to comparison"); @@ -104,7 +103,7 @@ describe("content and services audit regressions", () => { expect(serviceNavigatorMetrics(records)).toMatchObject({ verified: 1, localConfirmation: 1 }); }); - it("keeps seeded form provenance transparent and source-verified where available", () => { + it("keeps seeded form provenance explicitly unverified and free of invented source facts", () => { const transport = getFormRecord("transport-crisis-form"); expect(transport).not.toBeNull(); @@ -116,14 +115,18 @@ describe("content and services audit regressions", () => { }, }); expect(transport?.source).toHaveProperty("url"); + expect(transport?.source).toHaveProperty("reviewed"); expect(transport?.source).not.toHaveProperty("pages"); expect(transport?.source).not.toHaveProperty("pageCount"); expect(transport?.source).not.toHaveProperty("reviewDue"); - expect(JSON.stringify(transport?.source)).not.toMatch(/\bstatutory\b/i); - expect(formDetailSource).not.toMatch(/\bReview due\b/i); + expect(JSON.stringify(transport?.source)).not.toMatch(/\b\d+\s+pages?\b|\bstatutory\b/i); + expect(formDetailSource).not.toMatch(/\b\d+\s+pages?\b|\bReview due\b/i); expect(formDetailSource).not.toContain("01 May 2026"); - expect(formDetailSource).not.toMatch(/\b5\(2\)\b|Admission order|Treatment order/); - expect(normalizedFormDetailSource).toContain('label: "Source currency"'); + expect(formDetailSource).not.toMatch(/5\(2\)|Admission order|Treatment order/); + expect(formDetailSource).toContain("Full pathway unavailable"); + expect(normalizedFormDetailSource).toContain( + 'label: "Source currency", value: displayText(form.source?.reviewed, "Review locally")', + ); for (const form of formRecords) { if (form.source?.url) continue; @@ -133,20 +136,15 @@ describe("content and services audit regressions", () => { expect(form.source?.reviewed, form.slug).toBeUndefined(); } - expect(formsSearchSource).toContain('"Evidence", sourceSnippetCount'); - expect(formsSearchSource).toContain('"Pathways", pathwayCount'); - expect(formsSearchSource).toContain('"Tasks", taskCount'); - expect(formsSearchSource).toContain("const sourceSnippetCount = 278"); - expect(formsSearchSource).toContain("const taskCount = 8"); - expect(formsSearchSource).toContain("const pathwayCount = 12"); - expect(formsSearchSource).toContain("PSOLIS"); - expect(formsSearchSource).toContain("tagToneClass(chipLabel)"); expect(formsSearchSource).toContain("Title or content match"); expect(formsSearchSource).toContain("Content match in related pathway"); expect(formsSearchSource).toContain("View all forms"); - expect(formsHomeSource).not.toContain("Source verified"); + expect(formsHomeSource).not.toMatch(/Source verified|Open account setup/); + expect(formsHomeSource).not.toMatch( + /Number, pathway, clock|Maker, clock, copies|Browse pathways|Before, current, parallel, after|starter set of MHA 2014 forms|follow a pathway/, + ); + expect(formsHomeSource).toContain("local confirmation"); expect(formsHomeSource).toContain("Source catalogue reviewed"); - expect(formsHomeSource).toContain("Official-source MHA 2014 forms"); }); it("does not render negative or text-only source statuses as verified", () => { @@ -200,11 +198,12 @@ describe("content and services audit regressions", () => { }); it("claims and renders a form source link only when the record has a URL", () => { - expect(normalizedFormDetailSource).toContain("form.source?.url || details?.localPdfPath"); - expect(normalizedFormDetailSource).toContain("Source link pending"); - expect(normalizedFormDetailSource).toContain("form.source?.url"); - expect(normalizedFormDetailSource).toMatch(/]*>[\s\S]*Official/); - expect(formDetailSource).toContain("Official"); + expect(normalizedFormDetailSource).toContain("sourceHref={form.source?.url ?? null}"); + expect(normalizedFormDetailSource).toContain("href={form.source.url}"); + expect(normalizedFormDetailSource).toContain('target="_blank"'); + expect(normalizedFormDetailSource).toContain('rel="noopener noreferrer"'); + expect(normalizedFormDetailSource).toContain("inline-flex min-h-10"); expect(formDetailSource).toContain("Source link pending"); + expect(formDetailSource).toContain("Official"); }); }); diff --git a/tests/audit-navigation-auth-regressions.test.ts b/tests/audit-navigation-auth-regressions.test.ts index 80914146..41a9dbef 100644 --- a/tests/audit-navigation-auth-regressions.test.ts +++ b/tests/audit-navigation-auth-regressions.test.ts @@ -93,23 +93,26 @@ describe("audit navigation and auth regressions", () => { const privateCapabilityContract = sourceSegment( clinicalDashboardSource, "const canUsePrivateApis =", - "const canUploadDocuments =", + "const canRunSearch =", ); expect(privateCapabilityContract).toContain("const canUsePrivateApis ="); - expect(privateCapabilityContract).toContain("localNoAuthMode"); + expect(privateCapabilityContract).toContain( + 'localNoAuthMode || localDevCanAttemptPrivateApis || authStatus === "authenticated"', + ); const pollingContract = sourceSegment( clinicalDashboardSource, - "if (!nextDemoMode && !canUsePrivateApis)", - "const [documentsResponse, jobsResponse, batchesResponse, qualityResponse] = await Promise.all([", + "if (!nextDemoMode && !canUsePrivateApis) {", + "const shouldRefreshWorkState =", ); - expect(pollingContract).toContain("if (!nextDemoMode && !canUsePrivateApis)"); - expect(pollingContract).toContain("includeAdministrationData &&"); + expect(pollingContract).toContain("if (!nextDemoMode && !canUsePrivateApis) {"); + expect(pollingContract).toContain("setDocuments([]);"); + expect(pollingContract).toContain("return;"); const labelMutationContract = sourceSegment( clinicalDashboardSource, - "const mutateDocumentLabel = useCallback(", - "const handleDocumentDeleted = useCallback(", + "const mutateDocumentLabel =", + "const handleDocumentDeleted =", ); expect(labelMutationContract).toContain("if (!canUsePrivateApis) return false;"); @@ -123,6 +126,6 @@ describe("audit navigation and auth regressions", () => { it("keeps the root dashboard H1 as Clinical KB", () => { expect(clinicalDashboardSource.match(/\s*Clinical (?:Guide|KB)\s*<\/h1>/); + expect(clinicalDashboardSource).toMatch(/

\s*Clinical Guide\s*<\/h1>/); }); }); diff --git a/tests/rag-variant-early-exit.test.ts b/tests/rag-variant-early-exit.test.ts index 1a71d584..fc539511 100644 --- a/tests/rag-variant-early-exit.test.ts +++ b/tests/rag-variant-early-exit.test.ts @@ -125,7 +125,8 @@ describe("lexical variant early-exit (PT-02)", () => { expect(chunkTextCalls).toHaveLength(1); expect(telemetry.text_variant_early_exit).toBe(true); expect(telemetry.text_variant_rpc_calls?.match_document_chunks_text).toBe(1); - expect(from).not.toHaveBeenCalledWith("document_images"); + expect(from).toHaveBeenCalledWith("documents"); + expect(from).not.toHaveBeenCalledWith("document_pages"); }); it("a weak first pool keeps the full sibling fan-out", async () => { diff --git a/tests/site-map.test.ts b/tests/site-map.test.ts index 19907503..d8c787cf 100644 --- a/tests/site-map.test.ts +++ b/tests/site-map.test.ts @@ -52,15 +52,51 @@ describe("tracked sitemap", () => { expect(siteMap).toBe(await renderSiteMap()); }); - it("documents every app page route and API route", () => { + it("documents every app page, public route handler, and API route", () => { const data = collectSiteMapData(); for (const pageRoute of data.pageRoutes) expectDocumentedRoute(pageRoute.route); + for (const routeHandler of data.publicRouteHandlers) expectDocumentedRoute(routeHandler.route); for (const apiRoute of data.apiRoutes) expectDocumentedRoute(apiRoute.route); - for (const handler of data.appRouteHandlers) expectDocumentedRoute(handler.route); for (const redirect of data.redirects) expectDocumentedRoute(redirect.route); }); + it("keeps public redirect handlers in product routes and API handlers in the API section", () => { + const data = collectSiteMapData(); + const productSection = siteMap.slice( + siteMap.indexOf("## Main product routes"), + siteMap.indexOf("## Mode/query routes"), + ); + const apiSection = siteMap.slice(siteMap.indexOf("## API routes"), siteMap.indexOf("## Redirects")); + const expectedProductHandlers = [ + ["/applications", "src/app/applications/route.ts", "/tools"], + [ + "/differentials/presentations", + "src/app/differentials/presentations/route.ts", + "/differentials/presentations/[workflow-slug]", + ], + ["/medications", "src/app/medications/route.ts", "/?mode=prescribing"], + ] as const; + const redirectSection = siteMap.slice(siteMap.indexOf("## Redirects")); + + expect(data.apiRoutes.every((route) => route.route === "/api" || route.route.startsWith("/api/"))).toBe(true); + expect(data.publicRouteHandlers.some((route) => route.route === "/auth/callback")).toBe(true); + expect(data.publicRouteHandlers).toContainEqual({ + route: "/icons/[variant]", + file: "src/app/icons/[variant]/route.tsx", + }); + expect(apiSection).not.toContain("`/icons/[variant]`"); + + for (const [route, file, target] of expectedProductHandlers) { + expect(data.publicRouteHandlers).toContainEqual({ route, file }); + expect(data.apiRoutes).not.toContainEqual({ route, file }); + expect(data.redirects).toContainEqual({ route, file, target }); + expect(redirectSection).toContain(`\`${route}\``); + expect(apiSection).not.toContain(`\`${route}\``); + expect(productSection).not.toContain(`\`${route}\``); + } + }); + it("documents seeded dynamic slugs", () => { for (const service of serviceRecords) expectDocumentedRoute(service.slug); for (const form of formRecords) expectDocumentedRoute(form.slug); diff --git a/tests/supabase-schema.test.ts b/tests/supabase-schema.test.ts index afbf529e..102a0ef4 100644 --- a/tests/supabase-schema.test.ts +++ b/tests/supabase-schema.test.ts @@ -184,18 +184,6 @@ const responseCacheRetentionReconciliationMigration = readFileSync( new URL("../supabase/migrations/20260713201542_consolidate_rag_response_cache_retention.sql", import.meta.url), "utf8", ).replace(/\s+/g, " "); -const registryProjectionCleanupMigration = readFileSync( - new URL("../supabase/migrations/20260717170000_registry_projection_cleanup.sql", import.meta.url), - "utf8", -).replace(/\s+/g, " "); -const publicTitleCorrectorMigration = readFileSync( - new URL("../supabase/migrations/20260717171000_public_title_corrector.sql", import.meta.url), - "utf8", -).replace(/\s+/g, " "); -const atomicSummaryRateLimitsMigration = readFileSync( - new URL("../supabase/migrations/20260717172000_atomic_summary_rate_limits.sql", import.meta.url), - "utf8", -).replace(/\s+/g, " "); const liveDatabaseDriftMigration = readFileSync( new URL("../supabase/migrations/20260705230000_reconcile_live_database_drift.sql", import.meta.url), "utf8", @@ -215,6 +203,27 @@ const retrievalPlanCacheMigration = readFileSync( new URL("../supabase/migrations/20260711120000_retrieval_fn_plan_cache_mode.sql", import.meta.url), "utf8", ).replace(/\s+/g, " "); +const patchRagAndCorrectorScalabilityMigration = readFileSync( + new URL("../supabase/migrations/20260714180000_patch_rag_and_corrector_scalability.sql", import.meta.url), + "utf8", +).replace(/\s+/g, " "); +const documentTableFactsTrgmMigration = readFileSync( + new URL("../supabase/migrations/20260714190000_document_table_facts_trgm_idx.sql", import.meta.url), + "utf8", +).replace(/\s+/g, " "); +const hardenRagScalabilityPatchMigration = readFileSync( + new URL("../supabase/migrations/20260717010000_harden_rag_scalability_patch.sql", import.meta.url), + "utf8", +).replace(/\s+/g, " "); + +function finalSqlSegment(sql: string, startMarker: string, endMarker: string) { + const normalized = sql.toLowerCase(); + const start = normalized.lastIndexOf(startMarker.toLowerCase()); + if (start < 0) throw new Error(`Missing SQL marker: ${startMarker}`); + const end = normalized.indexOf(endMarker.toLowerCase(), start); + if (end < 0) throw new Error(`Missing SQL marker after ${startMarker}: ${endMarker}`); + return normalized.slice(start, end); +} function extractTextChunkFunction(sql: string) { const start = sql.indexOf("function public.match_document_chunks_text"); @@ -1196,22 +1205,13 @@ describe("Supabase Preview replay guards", () => { it("fails closed on effective supabase_admin default ACLs", () => { for (const sql of [schema, defaultAclAssertionMigration]) { - const statusStart = sql.lastIndexOf("create or replace function public.default_privileges_status("); - const statusEnd = sql.indexOf("$$;", statusStart); - const statusFunction = sql.slice(statusStart, statusEnd); - - expect(statusStart).toBeGreaterThanOrEqual(0); - expect(statusEnd).toBeGreaterThan(statusStart); - expect(statusFunction).toContain("pg_catalog.acldefault(ot.object_code, v_role_oid)"); - expect(statusFunction).toContain("pg_catalog.aclexplode(ea.acl)"); - expect(statusFunction).toContain("bool_or(grantee not in (p_role_name, 'postgres', 'service_role'))"); - expect(statusFunction).toContain("privilege.is_grantable"); - expect(statusFunction).toContain("coalesce(bool_or(is_grantable), false)"); - expect(statusFunction).toContain("into v_entries, v_has_unexpected_grantee, v_has_grantable"); - expect(statusFunction).toContain("not v_has_grantable"); - expect(statusFunction).toContain("entry like 'table:PUBLIC:%'"); - expect(statusFunction).toContain("entry like 'sequence:PUBLIC:%'"); - expect(statusFunction).toContain("entry = 'function:PUBLIC:execute'"); + expect(sql).toContain("create or replace function public.default_privileges_status("); + expect(sql).toContain("pg_catalog.acldefault(ot.object_code, v_role_oid)"); + expect(sql).toContain("pg_catalog.aclexplode(ea.acl)"); + expect(sql).toContain("bool_or(grantee not in (p_role_name, 'postgres', 'service_role'))"); + expect(sql).toContain("entry like 'table:PUBLIC:%'"); + expect(sql).toContain("entry like 'sequence:PUBLIC:%'"); + expect(sql).toContain("entry = 'function:PUBLIC:execute'"); expect(sql).toContain("message = 'Unsafe supabase_admin default privileges; migration blocked.'"); expect(sql).toContain("Run these six statements as supabase_admin, then retry the migration:"); } @@ -1350,127 +1350,83 @@ describe("Supabase Preview replay guards", () => { expect(ingestionRpcPrivilegesDuplicateMigration).toContain("select 1 where false;"); }); - it("cleans registry projections with text-and-kind matching through the partial expression index", () => { - expect(registryProjectionCleanupMigration).toContain("set lock_timeout = '5s';"); - expect(registryProjectionCleanupMigration).toContain("set statement_timeout = '60s';"); - - for (const sql of [registryProjectionCleanupMigration, schema]) { - const functionStart = sql.indexOf("create or replace function public.cleanup_registry_corpus_document()"); - const functionEnd = sql.indexOf("$$;", functionStart); - const cleanupFunction = sql.slice(functionStart, functionEnd); - - expect(functionStart).toBeGreaterThanOrEqual(0); - expect(cleanupFunction).toContain("metadata->>'registry_record_id' = old.id::text"); - expect(cleanupFunction).not.toContain("metadata->>'registry_record_id')::uuid"); - expect(cleanupFunction).toContain("metadata->>'registry_record_kind' = case tg_table_name"); - expect(cleanupFunction).toContain("when 'clinical_registry_records' then pg_catalog.to_jsonb(old)->>'kind'"); - expect(cleanupFunction).toContain("when 'medication_records' then 'medication'"); - expect(cleanupFunction).toContain("when 'differential_records' then 'differential'"); - expect(sql).toContain("create index if not exists documents_registry_projection_lookup_idx"); - expect(sql).toMatch(/\(\s*\(metadata->>'registry_record_kind'\),\s*\(metadata->>'registry_record_id'\)\s*\)/); - expect(sql).toContain("where metadata->>'source_kind' = 'registry_record'"); - expect(sql).toContain( - "revoke execute on function public.cleanup_registry_corpus_document() from public, anon, authenticated, service_role;", - ); + it("keeps the document-title vocabulary lifecycle aligned in migration and schema", () => { + for (const sql of [schema, patchRagAndCorrectorScalabilityMigration]) { + expect(sql).toContain("create table if not exists public.document_title_words"); + expect(sql).toContain("word text not null"); + expect(sql).toContain("document_id uuid not null references public.documents(id) on delete cascade"); + expect(sql).toContain("primary key (word, document_id)"); + expect(sql).toContain("insert into public.document_title_words (word, document_id)"); + expect(sql).toContain("drop trigger if exists documents_sync_title_words on public.documents"); + expect(sql).toContain("create trigger documents_sync_title_words"); } }); - it("indexes only public document-title words and probes capped trigram candidates", () => { - for (const sql of [publicTitleCorrectorMigration, schema]) { - const correctorStart = sql.lastIndexOf("create or replace function public.correct_clinical_query_terms("); - const correctorEnd = sql.indexOf( - "revoke execute on function public.correct_clinical_query_terms(text, real)", - correctorStart, + it("hardens registry cleanup without UUID casts or cross-registry collisions", () => { + for (const sql of [schema, hardenRagScalabilityPatchMigration]) { + const cleanup = finalSqlSegment( + sql, + "create or replace function public.cleanup_registry_corpus_document()", + "revoke execute on function public.cleanup_registry_corpus_document()", ); - const corrector = sql.slice(correctorStart, correctorEnd); - - expect(sql).toContain("create table if not exists public.document_title_words"); - expect(sql).toContain("create index if not exists document_title_words_word_trgm_idx"); - expect(sql).toContain("create index if not exists document_title_words_document_id_idx"); - expect(sql).toContain("where d.owner_id is null and d.status = 'indexed'"); - expect(sql).toContain("new.owner_id is null and new.status = 'indexed'"); - expect(sql).toContain("delete from public.document_title_words where document_id = old.id"); - expect(sql).toContain("alter table public.document_title_words enable row level security"); - expect(sql).toContain("revoke all on table public.document_title_words from public, anon, authenticated"); + const cleanupLower = cleanup.toLowerCase(); + expect(cleanupLower).toContain("metadata->>'registry_record_id' = old.id::text"); + expect(cleanupLower).toContain("metadata->>'registry_record_kind' = case tg_table_name"); + expect(cleanupLower).toMatch(/when 'clinical_registry_records' then (pg_catalog\.)?to_jsonb\(old\)->>'kind'/); + expect(cleanup).toContain("when 'medication_records' then 'medication'"); + expect(cleanup).toContain("when 'differential_records' then 'differential'"); + expect(cleanup).not.toContain("registry_record_id')::uuid"); expect(sql).toContain( - "grant select, insert, update, delete on table public.document_title_words to service_role", + "revoke execute on function public.cleanup_registry_corpus_document() from public, anon, authenticated", ); expect(sql).toContain( - "revoke execute on function public.sync_document_title_words() from public, anon, authenticated, service_role;", + "revoke execute on function public.sync_document_title_words() from public, anon, authenticated", ); - expect(sql.indexOf("create trigger documents_sync_title_words")).toBeLessThan( - sql.indexOf("select distinct lower(title_word), d.id"), - ); - expect(sql).toContain("create index if not exists rag_aliases_canonical_trgm_idx"); - expect(correctorStart).toBeGreaterThanOrEqual(0); - expect(correctorEnd).toBeGreaterThan(correctorStart); - expect(corrector).toContain("and lower(alias) % tok"); - expect(corrector).toContain("and lower(canonical) % tok"); - expect((corrector.match(/from public\.rag_aliases where enabled and owner_id is null/g) ?? []).length).toBe(2); - expect(corrector).toContain("and word % tok"); - expect((corrector.match(/limit 32/g) ?? []).length).toBeGreaterThanOrEqual(3); - // An exact alias has similarity 1, but the emitted correction must be its - // canonical term rather than the alias itself. Keep the match score tied - // to the indexed alias expression so typo ranking remains index-friendly. - expect(corrector).toContain("select lower(canonical) as term, similarity(lower(alias), tok) as match_sim"); - expect(corrector).not.toContain("select lower(alias) as term"); - expect(corrector).toContain("select candidate.term, candidate.match_sim"); - expect(corrector).toContain("order by candidate.match_sim desc, candidate.term"); - expect(corrector).toContain("best_sim >= min_sim"); - expect(corrector).toContain("length(best) >= length(tok)"); } + }); - const broadServiceRoleGrant = schema.lastIndexOf( - "grant execute on all functions in schema public to service_role;", - ); - expect(broadServiceRoleGrant).toBeGreaterThanOrEqual(0); - expect( - schema.lastIndexOf("revoke execute on function public.cleanup_registry_corpus_document() from service_role;"), - ).toBeGreaterThan(broadServiceRoleGrant); - expect( - schema.lastIndexOf("revoke execute on function public.sync_document_title_words() from service_role;"), - ).toBeGreaterThan(broadServiceRoleGrant); - }); - - it("keeps the table-facts trigram index expression identical to the active predicate", () => { - const indexStart = schema.indexOf("create index if not exists document_table_facts_title_row_param_trgm_idx"); - const indexExpressionStart = schema.indexOf("using gin (", indexStart) + "using gin (".length; - const indexExpressionEnd = schema.indexOf(" extensions.gin_trgm_ops", indexExpressionStart); - const predicateSection = schema.indexOf("trgm_matches as ("); - const predicateExpressionStart = schema.indexOf("on lower(", predicateSection) + "on ".length; - const predicateExpressionEnd = schema.indexOf(" % q.normalized", predicateExpressionStart); - - expect(indexStart).toBeGreaterThanOrEqual(0); - expect(predicateSection).toBeGreaterThanOrEqual(0); - expect(schema.slice(predicateExpressionStart, predicateExpressionEnd).replaceAll("f.", "")).toBe( - schema.slice(indexExpressionStart, indexExpressionEnd), - ); - expect(schema).not.toContain("document_table_facts_text_trgm_idx"); - }); - - it("defines one stable-order atomic limiter for summary streaming", () => { - for (const sql of [atomicSummaryRateLimitsMigration, schema]) { - expect(sql).toContain("create or replace function public.consume_summary_rate_limits_atomic("); - expect(sql).toContain("order by rl.bucket for update"); - expect(sql).toContain("order by rl.subject_key, rl.bucket for update"); - expect(sql).toContain("'answer'::text, 1"); - expect(sql).toContain("'document_summarize'::text, 3"); - expect((sql.match(/v_success_limit := v_policy\.limit_value/g) ?? []).length).toBe(2); - expect((sql.match(/v_success_reset_at := v_reset_at/g) ?? []).length).toBe(2); - expect(sql).toContain("on conflict on constraint api_rate_limits_pkey do nothing"); - expect(sql).toContain("on conflict on constraint api_rate_limit_subjects_pkey do nothing"); - expect(sql).not.toMatch(/on conflict \([^)]*bucket[^)]*\) do nothing/); - expect(sql).toContain( - "return query select null::text, false, v_success_limit, v_min_remaining, greatest(1, pg_catalog.ceil(extract(epoch from (v_success_reset_at - v_now)))::integer), v_success_reset_at;", - ); - expect(sql).toMatch( - /revoke execute on function public\.consume_summary_rate_limits_atomic\(\s*uuid, text, integer, integer, integer, integer, integer, integer\s*\)/, - ); - expect(sql).toMatch( - /grant execute on function public\.consume_summary_rate_limits_atomic\(\s*uuid, text, integer, integer, integer, integer, integer, integer\s*\) to service_role/, + it("uses bounded indexed probes for clinical query correction", () => { + for (const sql of [schema, hardenRagScalabilityPatchMigration]) { + const corrector = finalSqlSegment( + sql, + "create or replace function public.correct_clinical_query_terms", + "revoke execute on function public.correct_clinical_query_terms", ); + expect(corrector).toContain("lower(alias) % tok"); + expect(corrector).toContain("lower(canonical) % tok"); + expect(corrector).toContain("word % tok"); + expect(corrector).toContain("limit 32"); + expect(corrector).toContain("best is not null and best_sim >= min_sim"); + if (corrector.includes("min_sim is null")) { + expect(corrector).toContain("min_sim is null or min_sim < 0.3 or min_sim > 1"); + } + expect(corrector).not.toContain("array_agg(distinct term)"); + expect(corrector).not.toContain("unnest(vocab)"); + expect(sql).toContain("rag_aliases_canonical_trgm_idx"); } }); + + it("drops the mismatched wide table-facts trigram index and preserves RPC parity", () => { + const indexExpression = + "lower(coalesce(table_title, '') || ' ' || coalesce(row_label, '') || ' ' || coalesce(clinical_parameter, ''))"; + const rpcExpression = + "lower(coalesce(f.table_title, '') || ' ' || coalesce(f.row_label, '') || ' ' || coalesce(f.clinical_parameter, ''))"; + const tableFactsRpc = finalSqlSegment( + schema, + "create or replace function public.match_document_table_facts_text(", + "$function$;", + ); + + expect(documentTableFactsTrgmMigration).toContain("create index if not exists document_table_facts_text_trgm_idx"); + expect(hardenRagScalabilityPatchMigration).toContain( + "drop index if exists public.document_table_facts_text_trgm_idx", + ); + expect(schema).not.toContain("create index if not exists document_table_facts_text_trgm_idx"); + expect(schema).toContain( + `create index if not exists document_table_facts_title_row_param_trgm_idx on public.document_table_facts using gin (${indexExpression} extensions.gin_trgm_ops)`, + ); + expect(tableFactsRpc).toContain(`${rpcExpression} % q.normalized`); + }); }); describe("Clinical query-term corrector — tenant-safe vocabulary (F10)", () => { diff --git a/tests/ui-accessibility.spec.ts b/tests/ui-accessibility.spec.ts index e18ffd1c..6be58fad 100644 --- a/tests/ui-accessibility.spec.ts +++ b/tests/ui-accessibility.spec.ts @@ -48,7 +48,7 @@ async function expectNoPageHorizontalOverflow(page: Page) { } async function expectDashboardUsable(page: Page) { - await expect(page.getByRole("heading", { level: 1, name: "Clinical Guide" })).toHaveCount(1); + await expect(page.getByRole("heading", { level: 1, name: "Clinical KB" })).toHaveCount(1); await expect(page.getByRole("heading", { name: "Answer" })).toBeVisible(); await expect(page.locator('[aria-label^="Search indexed guidelines by question or keyword"]:visible')).toBeVisible(); await expect(page.getByRole("button", { name: "Open answer options" })).toBeVisible(); diff --git a/tests/ui-smoke.spec.ts b/tests/ui-smoke.spec.ts index 0ae320ca..299c2e87 100644 --- a/tests/ui-smoke.spec.ts +++ b/tests/ui-smoke.spec.ts @@ -19,7 +19,7 @@ const dashboardViewports = [ { name: "laptop", width: 1280, height: 900 }, { name: "mobile-landscape", width: 667, height: 375 }, ] as const; -const uiAssertionTimeoutMs = 5_000; +const uiAssertionTimeoutMs = 30_000; const demoAnswerThreadOwnerId = "local-demo-session"; const demoAnswerThreadStorageKey = `${answerThreadStorageKey}:${demoAnswerThreadOwnerId}`; const demoRecentQueryStorageKey = `${recentQueryStorageKey}:${demoAnswerThreadOwnerId}`; @@ -867,7 +867,7 @@ test.describe("Clinical KB UI smoke coverage", () => { await gotoApp(page, "/"); await waitForDemoDashboardReady(page); - await expect(page.getByRole("heading", { level: 1, name: "Clinical Guide" })).toHaveCount(1); + await expect(page.getByRole("heading", { level: 1, name: "Clinical KB" })).toHaveCount(1); await expect(page.getByRole("heading", { name: "Answer" })).toBeVisible(); await expect(visibleQuestionInput(page)).toBeVisible(); await expect(page.getByRole("button", { name: "Generate source-backed answer" })).toHaveText(/^\s*Ask\s*$/); @@ -1262,7 +1262,7 @@ test.describe("Clinical KB UI smoke coverage", () => { await expect(page.getByRole("button", { name: "Generate source-backed answer" })).toBeEnabled(); await expect(page.getByTestId("answer-grounding-chip")).toHaveCount(0); expect(answerRequests).toEqual([]); - await expect(page.getByRole("heading", { level: 1, name: "Clinical Guide" })).toBeVisible(); + await expect(page.getByRole("heading", { level: 1, name: "Clinical KB" })).toBeVisible(); await expectDomIntegrity(page, { mobileNav: true, mobileFabReady: false }); await expectNoPageHorizontalOverflow(page); }); @@ -2405,11 +2405,16 @@ test.describe("Clinical KB UI smoke coverage", () => { test("dashboard favourites mode param redirects to the standalone favourites route", async ({ page }) => { await page.setViewportSize({ width: 1280, height: 900 }); await mockDemoApi(page); + const redirectMeasureErrors: string[] = []; + page.on("pageerror", (error) => { + if (error.message.includes("cannot have a negative time stamp")) redirectMeasureErrors.push(error.message); + }); await gotoApp(page, "/?mode=favourites&q=lithium%20set&focus=1"); await expect(page).toHaveURL(/\/favourites\?q=lithium\+set&focus=1$/); await expect(page.getByTestId("favourites-hub")).toBeVisible(); await expect(page.getByRole("heading", { name: "Favourites command library" })).toBeVisible(); + expect(redirectMeasureErrors).toEqual([]); }); test("dashboard differentials mode param redirects to the standalone differentials route", async ({ page }) => { diff --git a/tests/ui-tools.spec.ts b/tests/ui-tools.spec.ts index 8f05b1f3..2ed5c540 100644 --- a/tests/ui-tools.spec.ts +++ b/tests/ui-tools.spec.ts @@ -882,7 +882,7 @@ test.describe("Clinical KB tools launcher", () => { await expectNoPageHorizontalOverflow(page); }); - test("forms mode shows source-backed form records in search results", async ({ page }) => { + test("forms mode shows registry-backed form records without unsupported pathway claims", async ({ page }) => { await page.setViewportSize({ width: 1280, height: 900 }); await mockAnswerDashboardApi(page); await gotoLauncher(page, "/forms?q=transport%20forms&focus=1&run=1"); @@ -902,6 +902,9 @@ test.describe("Clinical KB tools launcher", () => { await expect( page.getByTestId("form-search-result-transport-crisis-form").getByLabel("Open Transport order"), ).toHaveAttribute("href", "/forms/transport-crisis-form"); + await expect(page.getByRole("button", { name: "Refine" })).toHaveCount(0); + await expect(page.getByText(/Evidence 278|Pathways 12|Tasks 8|Source verified|Aligned to MHA 2014/)).toHaveCount(0); + await expect(page.getByText(/PSOLIS Transport|View full pathway/)).toHaveCount(0); await expect(page.getByTestId("service-search-results")).toHaveCount(0); await expectNoPageHorizontalOverflow(page); }); @@ -1002,6 +1005,7 @@ test.describe("Clinical KB tools launcher", () => { await expect(page.getByTestId("form-search-mobile-results")).toBeVisible(); await expect(page.getByTestId("form-search-mobile-result-transport-crisis-form")).toContainText("Transport order"); + await expect(page.getByText(/PSOLIS Transport|View full pathway|Source verified/)).toHaveCount(0); await expect(visibleGlobalSearchInput(page)).toHaveValue("transport"); await expectNoPageHorizontalOverflow(page); }); @@ -1014,6 +1018,19 @@ test.describe("Clinical KB tools launcher", () => { const dock = page.locator("form.answer-footer-search-dock"); await expect(dock).toBeVisible(); await expect(dock).not.toHaveAttribute("data-scroll-hidden", "true"); + const transition = await dock.evaluate((node) => { + const style = window.getComputedStyle(node); + const durationMs = Math.max( + ...style.transitionDuration.split(",").map((value) => { + const normalized = value.trim(); + const duration = Number.parseFloat(normalized); + return normalized.endsWith("ms") ? duration : duration * 1000; + }), + ); + return { durationMs, property: style.transitionProperty }; + }); + expect(transition.property).toMatch(/transform|all/); + expect(transition.durationMs).toBeGreaterThanOrEqual(100); // focus=1 leaves the composer focused; hide-on-scroll stays off while it has focus. const input = visibleGlobalSearchInput(page).first(); @@ -1603,6 +1620,7 @@ test.describe("Clinical KB service detail page", () => { { name: "desktop", width: 1280, height: 900 }, ] as const) { test(`13YARN service detail is usable at ${viewport.name}`, async ({ page }) => { + await mockAnswerDashboardApi(page); await page.setViewportSize({ width: viewport.width, height: viewport.height }); await gotoLauncher(page, "/services/13yarn"); @@ -1623,7 +1641,40 @@ test.describe("Clinical KB service detail page", () => { }); } + test("long mobile service details clear the bottom search dock at the scroll endpoint", async ({ page }) => { + await mockAnswerDashboardApi(page); + await page.setViewportSize({ width: 390, height: 820 }); + await gotoLauncher(page, "/services/city-east-community-mental-health-service"); + + const servicePage = page.getByTestId("service-detail-page"); + const footer = servicePage.getByText("Information accuracy may vary. Confirm locally before use."); + const scrollport = page.locator("#main-content"); + await expect(servicePage).toBeVisible(); + await scrollport.evaluate((element) => element.scrollTo({ top: element.scrollHeight, behavior: "instant" })); + + await expect + .poll(() => scrollport.evaluate((element) => element.scrollHeight - element.clientHeight - element.scrollTop)) + .toBeLessThanOrEqual(1); + + const clearance = await footer.evaluate((element) => { + const scrollElement = document.querySelector("#main-content"); + const dock = document.querySelector( + "form.answer-footer-search-dock, form.answer-footer-search-edge", + ); + if (!scrollElement || !dock) return null; + return { + footerBottom: element.getBoundingClientRect().bottom, + scrollBottom: scrollElement.getBoundingClientRect().bottom, + dockHeight: dock.getBoundingClientRect().height, + }; + }); + + expect(clearance).not.toBeNull(); + expect(clearance!.footerBottom).toBeLessThanOrEqual(clearance!.scrollBottom - clearance!.dockHeight - 8); + }); + test("service navigator action uses the shared global search route", async ({ page }) => { + await mockAnswerDashboardApi(page); await page.setViewportSize({ width: 1280, height: 900 }); await gotoLauncher(page, "/services/13yarn"); @@ -1634,6 +1685,7 @@ test.describe("Clinical KB service detail page", () => { }); test("service detail actions save, copy, and back from direct entry", async ({ page }) => { + await mockAnswerDashboardApi(page); await page.setViewportSize({ width: 1280, height: 900 }); await gotoLauncher(page, "/services/13yarn"); diff --git a/tsconfig.json b/tsconfig.json index 60b6af71..a08e95ac 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -25,5 +25,5 @@ } }, "include": ["next-env.d.ts", "**/*.ts", "**/*.tsx", ".next/types/**/*.ts", "**/*.mts", ".next/dev/types/**/*.ts"], - "exclude": ["node_modules", "scratch/**", "supabase/functions/**"] + "exclude": ["node_modules", "scratch/**", "supabase/functions/**", "worktrees/**"] }