perf(ui): speed up search and interactive catalogue surfaces

docs/branch-review-ledger.md

@@ -715,6 +715,12 @@ This file is append-only. Never rewrite or delete an existing review record; app
 | 2026-07-21 | claude/clinical-kb-pwa-review-asi3wb (Option A wave verdict — no code change; #1040 merged as cde6c5c) | canary run 29827012719 (#61, main cde6c5c) vs banked #60 (29800029819) | OPTION A WAVE ADOPTED — FIRST FULLY-GREEN 44-CASE CANARY IN PROGRAM HISTORY (Blocking failures: None). (1) Option A payoff EXCEEDED: citation_failure_rate 0.0227→0; the neuroleptic-side-effect-escalation case flipped from wrong-doc→failed-generation→1-citation-fallback to **strong route, successful gpt-5.6-sol generation, passed in 15.4s with no fallback marker** — the rescued S3 retrieval fixed generation itself, not just the citation count; expected_source_hit 0.6364→0.6591. (2) Golden held exactly as the blast-radius analysis promised: 36/36 PASS, content_recall 1.0, mrr@10 0.8921 BYTE-IDENTICAL to the pre-wave baseline (zero ordering movement — no golden case fires the predicate), irrelevant@10 0.1083→0.0917 (slightly better). (3) Parity payoff PARTIAL: monitoring targeting 1/5→2/5 (olanzapine-lai flipped — previously called a retrieval-depth residual; quetiapine-dose also flipped on the dose side); lithium-range (232ch) + metabolic (73ch, byte-identical answer to #60) did NOT flip despite offline-proven fixes — their live chunk sets evidently contain no admissible schedule sentence even under the widened gate → reclassified as retrieval-depth/live-content residuals joining adhd; below the ≥3/5 target but strictly improved, no regression anywhere. Dose 2/5 vs 2/4: same passing count, applicable set grew (new quality-metformin-renal-dosing miss = eval-set churn, not regression). (4) No-worse EXCEEDED: relevance 0.5333→0.6 (the two-step watch-item slide FULLY REVERSED to the #58 level), targeting_rate 0.6667→0.6957, fail_closed 0.9 held, readability/artifact_leaks 1.0, route ceilings 0, grounded 1.0, unsupported_correct 1.0, numeric 0, p95 22.8s, red_result 3/3. Adoption per the measured-gain rule: primary goal achieved, three case flips, relevance recovered, zero regressions. Residual queue: monitoring retrieval-depth trio (lithium-range/metabolic/adhd), E-3d H2 discards, weekly ANSWER_CASE_LIMIT 8→44 raise now unblocked (gate would be green), comparison-class coverage. Wave spend +~$2-4 → Phase E + Option A total ~$12-20 of ≤$20. | Evidence: run #61 job log read (Threshold Status: None; Answer Metrics; neuroleptic diagnostics row; targeting metric_rates + 6-miss list; golden 36 PASS lines + summary). Revert drill NOT triggered. |
 | 2026-07-21 | claude/clinical-kb-pwa-review-asi3wb (PR: I9 weekly coverage raise) | see PR head | ADDENDUM 5 post-green item I9 (plan-authorized "after reds fixed"): weekly scheduled canary ANSWER_CASE_LIMIT default 8→44 — the Sunday 18:00 UTC cron now guards the FULL answer-quality case set instead of the first 8 (both #57 blocking reds historically lived OUTSIDE the first 8, leaving the weekly gate blind to them). Unblocked by run #61 proving the citation gate green on the full 44. Cost: est +$1-2/week (user-authorized in the plan). Contract test pin updated in lockstep (eval-canary-workflow.test.ts). Dispatch shapes unchanged (input override still wins); operational-risk diff, plain-revert rollback. | check:github-actions PASS; check:ci-scope PASS; check:gate-manifest PASS (20/20); eval-canary-workflow contract 4/4; prettier clean; no provider calls |
 | 2026-07-21 | claude/database-governance-audit-10b6ed (PR #1051: source-governance audit — safe subset) | cee396730 | Governance-metadata observability + UI display + provenance flow test; no ranking/retrieval/generation surface touched. | IMPLEMENTED + handed off (not a review of prior work). Resolved audit #1 (logger.warn on unrecognized enum values; return value unchanged), #2 (review_due_source added to frontendVisibleWarningCodes → answer-level badge; warning-severity, no refusal impact), #9 (source_metadata retained on safety-finding citations + governance pill in SafetyFindingsListContent), #13 (new tests/provenance-flow.test.ts: DB-normalize→governance→client payload sources+safety citations→render policy). Deferred #4/5/6/8/10 (RAG-protected ranking/selection/LLM-context/cache — need live eval-canary+approval), #11/#5 flag debt (D5/D4), #3 (is_public schema/RLS), #7 (conflict-detection scope), #12 (canary automation). Rebased onto origin/main (was 18 behind; conflict-free — none of the 18 commits touched the 8 files). PR-policy CI green (confirmed no ragRankingPatterns match). | verify:pr-local exit 0 (351 files/3129 tests, production build, client-bundle secret scan, offline RAG fixtures 36/36); typecheck + lint + prettier green. verify:ui NOT run locally: pre-existing globals.css Tailwind/Turbopack dev-compile error (git-clean, unrelated; prod build passed) — CI Production UI job covers it. check:production-readiness deferred (offline env/config validator; PR changes no env/secret/config inputs; secretless worktree). No provider calls. |
+| 2026-07-24 | cursor/search-interactive-perf-af54 (PR #1138) | 46597a9b | Explicit performance + frontend-ui review of search/interactive surfaces; low-risk client deferral/cache/abort/progressive-reveal pass | Prior document/universal search latency work retained (NDJSON stream, LRU, lazy PDF, content-first detail). New work: differential debounce+abort+LRU; useDeferredValue on catalogue ranking; document results Show more window; RelatedDocumentsPanel memo; universal LRU 100+TTL; deferred registry search extracted from ClinicalDashboard. No RAG/retrieval/ranking edits. No high-confidence P0–P2 defect found in the shipped scope; residual risk = deferred paint lag on large catalogues and progressive reveal missing deep cards until Show more. | Focused Vitest 10/10 (differential + universal + performance boundaries); verify:cheap exit 0 (3262 tests); typecheck clean; verify:ui exit 0 (Chromium). No provider calls. |
+| 2026-07-24 | cursor/search-interactive-perf-af54 (PR #1138 follow-up) | 7e2ccee0 | Bugfix pass on search interactive performance diff | Fixed P1 auth-stale differential matches; P2 progressive-reveal hiding selected card; P2 deferred empty/full-catalogue flash on services/forms/formulation/therapy-compass; Prettier CI failure on universal-search test. No remaining high-confidence P0–P2 in scoped diff. Residual: differential debounce skeleton flicker; RelatedDocumentsPanel memo limited by unstable callbacks. | Focused Vitest 11/11; typecheck; format:check; maintainability budgets. No provider calls. |
+| 2026-07-24 | cursor/search-interactive-perf-af54 (PR #1138 babysit) | e27df0fa | Bugbot + merge babysit | Merged origin/main. Fixed Bugbot medium findings: deferred clear left stale service/registry matches. No unresolved review threads. | Focused Vitest (deferred registry + differential + boundaries); typecheck; maintainability budgets. Watching hosted CI. |
+| 2026-07-24 | cursor/search-interactive-perf-af54 (PR #1138 merge-ready) | 36d86fd0 | Babysit closeout | CodeRabbit: stable RelatedDocuments callbacks, identity-based progressive reveal, clear differential LRU on 401, Sheet unmount focus-restore via layout flag, formulation/therapy clear. Bugbot: live therapy filters with deferred query text. CI PR required green; 0 unresolved threads. | Focused Vitest; hosted CI PR required PASS. |
+| 2026-07-24 | cursor/search-interactive-perf-af54 (PR #1138 bugbot) | 8d712183 | Bugbot babysit | Fixed medium: formulation builder/home cleared live query still ranked against lagging deferredQuery. Merged main (#1137 search-chrome). No unresolved review threads. | Focused Vitest deferred registry; typecheck pending in CI. |
+| 2026-07-24 | cursor/search-interactive-perf-af54 (PR #1138 babysit close) | 2f8509cc | Babysit to merge-ready | Merged main; fixed Bugbot deferred-clear findings; fixed Sheet focus-restore teardown flake that failed Unit coverage; no unresolved review threads. Hosted PR required green. | Hosted CI: Static PR, Unit coverage, Build, Production UI, Advisory UI, PR required PASS. No provider calls. |
 | 2026-07-24 | codex/reindex-agent-enrichment-guard (PR #1143) | f82cf9cfa | Run PR sweep: CI fix + threads + drift | Before: PR required green, 1 unresolved reindex/agent-claim serialization thread, branch behind main. After: remote branch already had atomic RPC fix 2bb0470d plus merge-main d2d57d7c; validated pushed head, resolved thread via GraphQL; reply mutation 403 noted in commit f82cf9cf. | node scripts/run-vitest.mjs run --reporter=dot tests/reindex-enrichment-lease.test.ts tests/private-access-routes.test.ts tests/supabase-schema.test.ts tests/drift-detection.test.ts PASS (218/218); npm run check:migration-role PASS; npm run typecheck PASS; npm run check:production-readiness PASS with non-blocking warnings; no live eval gates run. |
 
 ## 2026-07-24 — work search chrome behaviour review
@@ -724,3 +730,4 @@ This file is append-only. Never rewrite or delete an existing review record; app
 - Scope: Targeted review of search bar/header/footer chrome behaviour after the edge-to-edge phone dock fix, plus durable repo rules for page-adaptive search chrome.
 - Outcome: No new P0/P1 search chrome defect found in the static review. Fixed one regression hazard: a stale ClinicalDashboard comment still instructed a 0.75rem hidden dock pad despite the implementation/tests requiring 0rem. Added durable search chrome behaviour rules in AGENTS.md and docs/search-chrome-behaviour.md, with a static guard tying the remembered rules to the hidden-reserve contract.
 - Checks: dependency shortcut section count; git diff --check; targeted rg for stale 0.75rem hidden-pad source wording (only negative test assertions remain); targeted Vitest command attempted but blocked by missing node_modules/vitest under Node 20.20.2 in this container. No provider-backed checks run.
+| 2026-07-24 | cursor/search-interactive-perf-af54 (PR #1138) | ff4b293d95f922e70ebf5ee9b0c156c41a8bff3b | Run PR sweep: CI fix + threads + drift | Before: CONFLICTING, CI green, 0 threads. RAG impact: no retrieval behaviour change — PR is client deferred-search/UI only (no src/lib/rag/**). After: merged origin/main; conflict resolved in src/components/ui/sheet.tsx by keeping main restoreTimersRef/unmountingRef focus-restore fix; pushed ff4b293d9. Threads: none. | merge origin/main only; no provider-backed checks run |

src/components/ClinicalDashboard.tsx

@@ -196,9 +196,7 @@ import {
 import { persistPrivateSearchScope, restorePrivateSearchScope } from "@/lib/private-search-scope";
 import { parseApiErrorResponse } from "@/lib/api-client-error";
 import { answerLifecycleReducer, initialAnswerLifecycle } from "@/lib/answer-lifecycle";
-import { rankFormRecords } from "@/lib/form-ranker";
-import { rankServiceRecords } from "@/lib/service-ranker";
-import { useRegistryRecords } from "@/lib/use-registry-records";
+import { useDeferredRegistrySearch } from "@/components/clinical-dashboard/use-deferred-registry-search";
 import { buildAnswerFollowUpQuery, buildAnswerFollowUpSuggestions } from "@/lib/answer-follow-up";
 import {
   clearPersistedAnswerThread,
@@ -431,24 +429,9 @@ export function ClinicalDashboard({
   const [restoredPrivateScopeRef, setRestoredPrivateScopeRef] = useState<string | null>(null);
 
   // Record matches come from the owner-scoped registry API (mock fixtures in
-  // demo mode); ranking stays client-side so live-typing behaviour is
-  // unchanged and the registry is fetched once per active mode.
-  const registryRecords = useRegistryRecords(searchMode === "forms" ? "form" : "service", {
-    enabled: searchMode === "services" || searchMode === "forms",
-  });
-  const serviceSearchMatches = useMemo(
-    () => (searchMode === "services" ? rankServiceRecords(registryRecords.records, query) : []),
-    [query, searchMode, registryRecords.records],
-  );
-  const formSearchMatches = useMemo(
-    () => (searchMode === "forms" ? rankFormRecords(registryRecords.records, query) : []),
-    [query, searchMode, registryRecords.records],
-  );
-  const recordSearchMatches = useMemo(
-    () => (searchMode === "forms" ? formSearchMatches : searchMode === "services" ? serviceSearchMatches : []),
-    [searchMode, formSearchMatches, serviceSearchMatches],
-  );
-  const recordSearchMode = searchMode === "forms" ? "forms" : "services";
+  // demo mode); ranking stays client-side (deferred) so live-typing stays
+  // responsive and the registry is fetched once per active mode.
+  const { recordSearchMatches, recordSearchMode, recordStatus } = useDeferredRegistrySearch(searchMode, query);
   // The thread mirror ref must never outlive the answer it describes: every
   // reset path nulls `answer`, so clearing here covers them all (mode
   // switches, new chat, differentials/services clears) without each caller
@@ -3750,7 +3733,7 @@ export function ClinicalDashboard({
                         matches={documentMatches}
                         recordMatches={recordSearchMatches}
                         recordMode={recordSearchMode}
-                        recordStatus={registryRecords.status}
+                        recordStatus={recordStatus}
                         showRecordMatches={searchMode === "services" || searchMode === "forms"}
                         query={query}
                         loading={loading}
@@ -3849,8 +3832,8 @@ export function ClinicalDashboard({
               {activeModeResultKind === "answer" && answer && (
                 <RelatedDocumentsPanel
                   documents={relatedDocuments}
-                  onScopeDocument={scopeOnlyDocument}
-                  onTagSearch={handleTagSearch}
+                  onScopeDocument={handleScopeDocument}
+                  onTagSearch={handleDocumentTagSearch}
                 />
               )}
               {(documentsDrawerOpen || uploadDrawerOpen) && (

src/components/clinical-dashboard/document-results.tsx

@@ -1,6 +1,7 @@
 "use client";
 
 import Link from "next/link";
+import { memo } from "react";
 import { BookOpen, FileImage, Filter, ListChecks } from "lucide-react";
 
 import { DocumentOrganizationBadges, documentDisplayTitle } from "@/components/DocumentOrganizationBadges";
@@ -105,7 +106,7 @@ function RelatedDocumentCard({
   );
 }
 
-export function RelatedDocumentsPanel({
+function RelatedDocumentsPanelImpl({
   documents,
   onScopeDocument,
   onTagSearch,
@@ -136,3 +137,7 @@ export function RelatedDocumentsPanel({
     </UtilityDrawer>
   );
 }
+
+// Memoized so answer SSE progress in ClinicalDashboard does not re-render this
+// subtree when documents and callbacks are unchanged.
+export const RelatedDocumentsPanel = memo(RelatedDocumentsPanelImpl);

src/components/clinical-dashboard/document-search-results.tsx

@@ -68,6 +68,10 @@ import { documentRelevancePercent } from "./relevance-score";
 
 type SearchFacet = { value: string; count: number };
 type ResultTypeFilter = "all" | "tables" | "images" | "pdfs";
+
+/** Initial DOM budget for document result cards; further rows reveal on demand. */
+const DOCUMENT_RESULTS_INITIAL_WINDOW = 25;
+const DOCUMENT_RESULTS_PAGE_SIZE = 25;
 export type SearchFacets = {
   status?: SearchFacet[];
   validation?: SearchFacet[];
@@ -873,6 +877,33 @@ function DocumentSearchResultsPanelImpl({
     () => sortResultItems(displayedMatches, sortValue, documentDisplayTitle),
     [displayedMatches, sortValue],
   );
+  // Progressive reveal so large libraries do not mount every card on first paint.
+  // Reset the window whenever the sorted result set identity changes (query/filter/sort),
+  // but expand far enough that an explicit selection stays visible in the list.
+  const resultsSignature = [
+    trimmedQuery,
+    sortValue,
+    effectiveResultType,
+    activeFacetKeys.join(","),
+    sortedMatches.map((document) => document.document_id).join(","),
+  ].join("\0");
+  const selectedIndex = selectedDocumentId
+    ? sortedMatches.findIndex((document) => document.document_id === selectedDocumentId)
+    : -1;
+  const minimumVisibleForSelection =
+    selectedIndex >= 0 ? Math.max(DOCUMENT_RESULTS_INITIAL_WINDOW, selectedIndex + 1) : DOCUMENT_RESULTS_INITIAL_WINDOW;
+  const [visibleCountState, setVisibleCountState] = useState({
+    signature: resultsSignature,
+    count: minimumVisibleForSelection,
+  });
+  if (visibleCountState.signature !== resultsSignature) {
+    setVisibleCountState({ signature: resultsSignature, count: minimumVisibleForSelection });
+  } else if (selectedIndex >= visibleCountState.count) {
+    setVisibleCountState({ signature: resultsSignature, count: selectedIndex + 1 });
+  }
+  const visibleCount = Math.min(visibleCountState.count, sortedMatches.length);
+  const renderedMatches = sortedMatches.slice(0, visibleCount);
+  const hasMoreMatches = visibleCount < sortedMatches.length;
   const selectedDocument =
     sortedMatches.find((document) => document.document_id === selectedDocumentId) ?? sortedMatches[0] ?? null;
   const recordMatchCount = recordMatches.length;
@@ -1012,7 +1043,7 @@ function DocumentSearchResultsPanelImpl({
                   No document matches include all selected filters.
                 </div>
               ) : null}
-              {sortedMatches.map((document, index) => {
+              {renderedMatches.map((document, index) => {
                 const relevanceDisplay = relevanceTone(document);
                 const fileKind = documentFileKind(document.file_name, "DOC");
                 const relevanceVariant = relevanceDisplay.short === "High relevance" ? "high" : "relevant";
@@ -1134,6 +1165,24 @@ function DocumentSearchResultsPanelImpl({
                   </article>
                 );
               })}
+              {hasMoreMatches ? (
+                <button
+                  type="button"
+                  className={cn(
+                    floatingControl,
+                    "min-h-tap w-full justify-center rounded-xl px-4 text-sm font-semibold",
+                  )}
+                  onClick={() =>
+                    setVisibleCountState((current) => ({
+                      signature: resultsSignature,
+                      count: Math.min(current.count + DOCUMENT_RESULTS_PAGE_SIZE, sortedMatches.length),
+                    }))
+                  }
+                  data-testid="document-search-show-more"
+                >
+                  Show more ({sortedMatches.length - visibleCount} remaining)
+                </button>
+              ) : null}
             </div>
             {selectedDocument ? (
               <SelectedDocumentEvidencePanel