Skip to content

Audit fixes + Node 24 toolchain bump#76

Merged
BigSimmo merged 3 commits into
mainfrom
claude/practical-pare-f2b20b
Jun 27, 2026
Merged

Audit fixes + Node 24 toolchain bump#76
BigSimmo merged 3 commits into
mainfrom
claude/practical-pare-f2b20b

Conversation

@BigSimmo

Copy link
Copy Markdown
Owner

Summary

Fixes from a repo-wide bug/flaw/logic/CSS/JS audit, plus the Node 24 toolchain bump that unblocks the upgraded runtime. Two commits:

  • chore(runtime) — the exact-equality Node 22 / npm 10 guards rejected the upgraded Node 24 runtime, blocking npm install (preinstall), the dev server, npm run ensure, and check:runtime. Bumps engines, packageManager, .nvmrc/.node-version, all runtime guards, the release gate (+ its test), and CI to Node 24 / npm 11.
  • fix — audit findings:
    • search-scope (H1): deterministic .order("id") before .range() so scoped document paging can no longer silently skip/duplicate documents past the 1000-row page.
    • eval-cases route (M3, privacy): store the normalized query + privacy metadata and gate verbatim answer/note behind RAG_PERSIST_RAW_QUERY_TEXT, matching every other rag_query_misses writer (RET-H4). Closes a PHI-at-rest gap.
    • rag-answer-text (M1): truncation now fires only on a quoted JSON key, never on ordinary prose words like confidence: / body: (was silently dropping the rest of real answers).
    • rag extractive (M7): grounded/confidence gated on an actually-extracted answer; guarded a quote-card find() non-null assertion that could throw.
    • worker (L2): evict progressUpdateState per job (slow leak). docx extractor (L3): map real image MIME types instead of defaulting EMF/WMF/TIFF/BMP to image/png. signed-url routes (L6): UUID-validate the id (→ 400 not 500). DashboardFloatingFab (L7): clear copy-notice timeout on unmount. evidence (L8): snippet ellipsis keyed on real truncation. globals.css (L12/L13): consolidate radius tokens to @theme (Tailwind utilities and var() were drifting); drop dead duplicate --duration-*.

Deferred (need live-app verification or carry ingestion-pipeline risk): M2 (reindex resets index before extract), M4 (chunk dedup over-collapse), M5 (relevancePercent), M6 (PDF viewer remount), M8 (search race). Tracked for follow-up.

Verification

  • npm run verify:cheap — ran components individually: lint ✅, typecheck ✅, vitest 502/502
  • npm run verify:uinot run. CSS token consolidation + a timer-cleanup touch the UI; low visual risk, but a Chromium pass is recommended before release.
  • npm run verify:release — not run.
  • npm run format:checkpre-existing repo-wide failures (34 files, most untouched here; the pre-edit versions of the files this PR touches were already non-compliant). No new formatting introduced by this PR.
  • npm run check:production-readinessNode runtime check passes (24.15.0). The remaining FAILs are missing local env vars (NEXT_PUBLIC_SUPABASE_URL, SUPABASE_SERVICE_ROLE_KEY, OPENAI_API_KEY) in the sandbox, not introduced by this change.

Clinical Governance Preflight

  • Source-backed claims still require linked source verification before clinical use — M7 strengthens this (no grounded:true without an extracted, cited sentence)
  • No patient-identifiable document workflow was introduced or expanded — M3 reduces PHI retention (raw query/answer no longer stored unless explicitly enabled)
  • Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy) — unchanged
  • Service-role keys and private document access remain server-only — signed-url routes stay runtime = "nodejs"; L6 only adds id validation, no change to the access model
  • Demo/synthetic content remains clearly separated from real clinical sources — unchanged (eval capture still blocks demo mode)
  • Source metadata, review status, and outdated/unknown-source behavior remain conservative — unchanged
  • Deployment classification/TGA SaMD impact checked — answer-generation changes (M1, M7) make output more conservative (preserve real text, avoid false grounding); no new decision-support capability added

Notes

  • Two separable commits (toolchain vs. fixes) for independent review/revert.
  • Full audit (including deferred items and 9 investigated-and-cleared findings) is documented outside the repo in the planning notes.

🤖 Generated with Claude Code

BigSimmo and others added 2 commits June 27, 2026 13:12
The exact-equality Node 22 / npm 10 guards rejected the upgraded Node 24 runtime, blocking npm install (preinstall), the dev server, npm run ensure, and check:runtime. Bump engines, packageManager, .nvmrc/.node-version, all runtime guards, the release gate (and its test), and CI to Node 24 / npm 11.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- search-scope: deterministic .order(id) before .range() so scoped document paging cannot silently skip/duplicate docs past the 1000-row page (H1).
- eval-cases route: store normalized query + privacy metadata, gate verbatim answer/note behind RAG_PERSIST_RAW_QUERY_TEXT (PHI at rest, M3).
- rag-answer-text: only truncate at a quoted JSON key, never at prose words like 'confidence:'/'body:' (M1).
- rag extractive: gate grounded/confidence on an actually-extracted answer; guard the quote-card find() non-null assertion (M7).
- worker: evict progressUpdateState per job (L2).
- docx extractor: map real image MIME types instead of defaulting to png (L3).
- signed-url routes: UUID-validate the id -> 400 not 500 (L6).
- DashboardFloatingFab: clear copy-notice timeout on unmount (L7).
- evidence: snippet ellipsis keyed on real truncation (L8).
- globals.css: consolidate radius tokens to @theme; drop dead duplicate --duration-* (L12/L13).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings June 27, 2026 05:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the project’s runtime/tooling baseline and applies a set of audit-driven fixes across ingestion, retrieval/RAG, API routes, and UI/CSS to improve correctness, privacy handling, and operational stability.

Changes:

  • Bump runtime/toolchain targeting to Node 24 / npm 11 across engines, local scripts, runtime guards, CI, and the release gate test.
  • Apply audit fixes to search scope paging determinism, RAG answer sanitization/extractive grounding, and eval-cases privacy retention behavior.
  • Improve reliability/UX via small fixes (worker state cleanup, signed-url UUID validation, evidence snippet truncation, FAB timer cleanup, CSS token consolidation).

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
worker/main.ts Cleans up per-job progress throttle state to prevent long-lived map growth.
tests/eval-cases-route.test.ts Updates eval-cases tests to validate privacy-gated query/answer retention behavior.
tests/check-runtime.test.ts Updates runtime gate tests for Node 24 / npm 11 targets.
src/lib/search-scope.ts Adds deterministic ordering before paging to prevent skip/dup across page queries.
src/lib/rag.ts Hardens extractive answer citation lookup and gates grounded/confidence on actual extraction.
src/lib/rag-answer-text.ts Tightens mid-text truncation to only trigger on quoted JSON keys (avoids cutting prose).
src/lib/extractors/document.ts Improves DOCX embedded image MIME typing for downstream storage/processing.
src/lib/evidence.ts Fixes snippet ellipsis logic to reflect real truncation instead of post-trim length.
src/components/DashboardFloatingFab.tsx Ensures pending timeout cleanup on unmount to avoid stray setState.
src/app/globals.css Consolidates radius/motion tokens to avoid drift and dead/duplicated definitions.
src/app/api/images/[id]/signed-url/route.ts Validates route id as UUID and returns a public 400 error on invalid ids.
src/app/api/eval-cases/route.ts Stores normalized query + privacy metadata; gates verbatim note/answer by env flag.
src/app/api/documents/[id]/signed-url/route.ts Validates route id as UUID and returns a public 400 error on invalid ids.
scripts/ensure-local-server.mjs Updates local server guard to require Node 24.x.
scripts/dev-free-port.mjs Updates local server guard to require Node 24.x.
scripts/check-runtime.ts Updates default expected majors to Node 24 / npm 11 for release verification.
scripts/check-node-engine.cjs Updates install-time engine guard to require Node 24 / npm 11.
package.json Bumps engines + packageManager to Node 24.x / npm 11.x and locks npm version.
.nvmrc Updates nvm target to Node 24.
.node-version Updates node-version target to Node 24.
.github/workflows/ci.yml Updates CI Node version to 24.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/lib/extractors/document.ts

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b2d702b7e5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread package.json
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@BigSimmo
BigSimmo merged commit 3d9c6c7 into main Jun 27, 2026
4 checks passed
@BigSimmo
BigSimmo deleted the claude/practical-pare-f2b20b branch June 27, 2026 05:31
cursor Bot pushed a commit that referenced this pull request Jul 24, 2026
Extend find_tables candidates using contiguous cell drawings so straddling
final rows are cropped to the page edge, recover structured remnant text when
possible, and emit an incompleteness warning when content continues past the
page. Flip the #76 red contracts green and archive the ledger item.

Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>
cursor Bot pushed a commit that referenced this pull request Jul 24, 2026
* test(pdf): red repro for page-edge table crop truncation (#76)

Add a minimal authorised synthetic fixture and deliberately-red Python/Vitest
contracts that name the failing pymupdf_find_tables stage and expected clip
geometry before any retention/padding change from PR #1129.

Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>

* fix(pdf): retain on-page remnant for page-edge table crops (#76)

Extend find_tables candidates using contiguous cell drawings so straddling
final rows are cropped to the page edge, recover structured remnant text when
possible, and emit an incompleteness warning when content continues past the
page. Flip the #76 red contracts green and archive the ledger item.

Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>

* docs: record PR #1176 page-edge crop review in branch ledger

Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>

* docs: supersede #1176 review ledger row for merged tip

Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>

* fix(ci): format PDF crop follow-up

* docs: record PR 1176 maintenance

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants