fix(db): reconcile live database drift and harden security without RAG regression by BigSimmo · Pull Request #300 · BigSimmo/Database

BigSimmo · 2026-07-05T18:44:38Z

Summary

Reconciles live Supabase drift on sjrfecxgysukkwxsowpy, hardens security boundaries, and fixes CI blockers — without changing RAG scoring, ranking, or chunking behavior.

Changes

Migrations: 20260705133000 (search_document_chunks owner scope), 20260705220000 (jobs table, RPCs, embedding-fields grants, eval RLS)
Edge: indexing-v3-agent JSONB status RPC parsing fix
App: health deep probe gating, owner-scope retrieval tightening, document label governance advisory
Tests/docs: schema mirror tests, public access deep checks, migration reconciliation docs

Verification

npm run format:check — PASS
CI verify job — PASS (run https://github.com/BigSimmo/Database/actions/runs/28750309235)
CI ui-smoke — PASS (same run)
npm run check:edge:functions — PASS locally
Live migrations applied through 20260705220000

CI fixes in this branch

Commit	Fix
`6f1d525e3`	Prettier on 4 files blocking format check
`09b3567c8`	TS2339 — add `allowGlobalSearch?` to `fetchBestDocumentLookupChunks` args type

Clinical Governance Preflight

Source-backed claims still require linked source verification before clinical use
No patient-identifiable document workflow introduced
Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
Service-role keys and private document access remain server-only
No RAG scoring/ranking/chunking changes in this PR

Merge note

Branch may conflict with main where overlapping access-rollout work landed via other PRs (#291, #297). Resolve at merge without importing unrelated UI commits onto this branch.

Operator follow-ups (not in this PR)

Set HEALTH_DEEP_PROBE_SECRET in production
Publishable key rotation
Auth server connection cap (Supabase dashboard)

- Degrade invalid bearer tokens to anonymous scope instead of 401 - Allow public document read routes (list, detail, signed-url, search, images) - Align registry routes with medications/differentials auth-signal short-circuit - Let DocumentViewer load public sources without requiring sign-in - Add regression tests and update access-control expectations

Create missing retrieval-support indexes (trgm, composite btree, partial miss log) that were absent or only present under legacy names on live. Update search_schema_health() to accept verified functional equivalents during rollout. Set search_path for pg_trgm gin_trgm_ops in extensions. Verified on linked project: search_schema_health() ok=true, missing=[].

- Re-apply indexing_v3_agent_jobs table and claim/update RPCs on live - Codify match_document_embedding_fields_text with service_role-only execute - Enable RLS on rag_visual_eval_* tables - Fix edge function JSONB status RPC parsing - Harden owner-scope and health deep-probe gating - Restore .env.example; remove unused postgres npm dep - Make gold-label governance advisory-only

… parsing

- Re-apply indexing_v3_agent_jobs table and claim/update RPCs on live - Codify match_document_embedding_fields_text with service_role-only execute - Enable RLS on rag_visual_eval_* tables - Fix edge function JSONB status RPC parsing - Harden owner-scope and health deep-probe gating - Restore .env.example; remove unused postgres npm dep - Make gold-label governance advisory-only

supabase · 2026-07-05T18:44:49Z

Updates to Preview Branch (cursor/fix-all-db-issues-5f13) ↗︎

Deployments	Status	Updated
Database	✅	Mon, 06 Jul 2026 04:35:03 UTC
Services	✅	Mon, 06 Jul 2026 04:35:03 UTC
APIs	✅	Mon, 06 Jul 2026 04:35:03 UTC

Tasks are run on every commit but only new migration files are pushed.
Close and reopen this PR if you want to apply changes from existing seed or migration files.

Tasks	Status	Updated
Configurations	✅	Mon, 06 Jul 2026 04:35:05 UTC
Migrations	✅	Mon, 06 Jul 2026 04:35:09 UTC
Seeding	✅	Mon, 06 Jul 2026 04:35:10 UTC
Edge Functions	✅	Mon, 06 Jul 2026 04:35:13 UTC

View logs for this Workflow Run ↗︎.
Learn more about Supabase for Git ↗︎.

Co-authored-by: Cursor <cursoragent@cursor.com>

Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>

chatgpt-codex-connector · 2026-07-06T02:34:13Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Copilot

Pull request overview

This PR reconciles observed drift between the repository and the live Supabase project schema, tightens owner-scoped retrieval boundaries, and hardens operational endpoints (health + indexing agent RPC parsing) while aiming to avoid any RAG scoring/ranking/chunking changes.

Changes:

Codifies live database drift via new/idempotent migrations and updates supabase/schema.sql to match (jobs table/RPCs, embedding-fields RPC grants, visual eval tables + RLS, search_document_chunks scope).
Hardens /api/health?deep=1 by requiring an operator secret header for the deep Supabase probe, and updates deep public-access tests accordingly.
Fixes indexing-v3-agent edge function handling of JSONB-returning status RPCs via robust result parsing.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/supabase-schema.test.ts	Adds assertions to ensure schema + key migrations mirror live drift fixes and owner-scope tightening.
tests/public-access-deep.test.ts	Updates health deep-probe expectations to reflect new secret-gated behavior.
supabase/schema.sql	Mirrors drift reconciliation (drop/recreate function signature, adds search_document_chunks, embedding-fields RPC, visual eval tables + RLS/policies, grants).
supabase/migrations/20260705220000_reconcile_live_database_drift.sql	New idempotent migration to re-apply missing live objects and lock down permissions/RLS.
supabase/migrations/20260705210000_retrieval_owner_filter_sentinel.sql	Adds a defensive DROP to handle signature/OUT-column drift before CREATE OR REPLACE.
supabase/migrations/20260705133000_tighten_search_document_chunks_owner_scope.sql	Documents/codifies live owner-scope tightening and revokes public/anon/auth execute.
supabase/functions/indexing-v3-agent/index.ts	Uses typed RPC rows + parsing helper for JSONB-returning job-status RPCs.
src/lib/env.ts	Adds SUPABASE_DB_URL and HEALTH_DEEP_PROBE_SECRET to the validated env schema.
src/lib/document-label-governance.ts	Treats missing gold-label rows as non-blocking (advisory) in the overall pass/fail result.
src/app/api/health/route.ts	Adds constant-time header token check for deep health probe authorization; returns 503 on unauthorized deep probe.
scripts/check-document-label-governance.ts	Clarifies CLI output that missing gold-label rows are advisory.
package.json	Removes the `postgres` dependency.
docs/supabase-migration-reconciliation.md	Documents the newly codified live drift and how it’s reconciled.
docs/process-hardening.md	Adds a dated log entry describing drift reconciliation and follow-ups.
.env.example	Adds SUPABASE_DB_URL and HEALTH_DEEP_PROBE_SECRET examples for operators/devs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

BigSimmo · 2026-07-06T04:16:16Z

@copilot resolve the merge conflicts in this pull request

…ope and commit-gen RPC test cases

Copilot · 2026-07-06T04:18:16Z

@copilot resolve the merge conflicts in this pull request

Done — merged origin/main into this branch. The only conflict was in tests/supabase-schema.test.ts where both branches added a test in the same location. I kept both:

This branch: "mirrors tightened search_document_chunks owner scope in schema and migration"
main: "surfaces stale commit generation RPCs through search_schema_health"

Resolved in merge commit 3fc7de8.

…ollision

BigSimmo added 10 commits July 5, 2026 20:02

chore(db): codify live search_document_chunks owner-scope migration

976c287

chore(db): mirror search_document_chunks in schema and fix edge JSONB…

8275263

… parsing

fix(edge): use JobStatusRpcResult for JSONB status RPC parsing

f7b0edb

style: fix Prettier formatting for CI verify gate

6f1d525

fix(rag): add allowGlobalSearch to document lookup chunk args type

09b3567

fix(edge): use JobStatusRpcResult for JSONB status RPC parsing

3179742

BigSimmo and others added 4 commits July 6, 2026 02:47

style: fix Prettier formatting for CI verify gate

aee99a7

fix(test): remove duplicate closing brace in supabase-schema spec

6c69cb1

Co-authored-by: Cursor <cursoragent@cursor.com>

fix: resolve edge JSONB typing and owner-scope schema test import

34370ee

Co-authored-by: Cursor <cursoragent@cursor.com>

chore(db): mirror search_document_chunks owner scope in schema.sql

8ab94ea

Co-authored-by: Cursor <cursoragent@cursor.com>

BigSimmo force-pushed the cursor/fix-all-db-issues-5f13 branch from 09b3567 to 8ab94ea Compare July 5, 2026 19:11

cursoragent and others added 3 commits July 5, 2026 19:32

merge: sync main into fix-all-db-issues branch

2e31212

Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>

merge: integrate remote branch updates

c0b5f74

Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>

fix(edge): guard null parseJobStatusRpcResult in updateAgentJobStatus

f4809e1

Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>

BigSimmo marked this pull request as ready for review July 6, 2026 02:34

BigSimmo requested a review from Copilot July 6, 2026 02:34

Copilot started reviewing on behalf of BigSimmo July 6, 2026 02:34 View session

Copilot AI reviewed Jul 6, 2026

View reviewed changes

Comment thread src/app/api/health/route.ts

Comment thread .env.example Outdated

BigSimmo and others added 2 commits July 6, 2026 12:15

Potential fix for pull request finding

c66df0e

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

e04da81

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot started work on behalf of BigSimmo July 6, 2026 04:16 View session

merge: resolve conflict in supabase-schema tests — keep both owner-sc…

3fc7de8

…ope and commit-gen RPC test cases

Copilot finished work on behalf of BigSimmo July 6, 2026 04:18

fix(db): rename reconcile migration to avoid 20260705220000 version c…

7f4e709

…ollision

BigSimmo merged commit ea569a7 into main Jul 6, 2026
5 checks passed

BigSimmo added a commit that referenced this pull request Jul 6, 2026

merge: sync desktop-answer-popups with main after #300

6ac1d5c

BigSimmo mentioned this pull request Jul 6, 2026

fix: address review findings across recent PR churn (governance caveat, skeleton hang, owner sentinel, port probe) #321

Merged

15 tasks

BigSimmo deleted the cursor/fix-all-db-issues-5f13 branch July 8, 2026 16:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(db): reconcile live database drift and harden security without RAG regression#300

fix(db): reconcile live database drift and harden security without RAG regression#300
BigSimmo merged 21 commits into
mainfrom
cursor/fix-all-db-issues-5f13

BigSimmo commented Jul 5, 2026

Uh oh!

supabase Bot commented Jul 5, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot commented Jul 6, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

BigSimmo commented Jul 6, 2026

Uh oh!

Copilot AI commented Jul 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

BigSimmo commented Jul 5, 2026

Summary

Changes

Verification

CI fixes in this branch

Clinical Governance Preflight

Merge note

Operator follow-ups (not in this PR)

Uh oh!

supabase Bot commented Jul 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Jul 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

BigSimmo commented Jul 6, 2026

Uh oh!

Copilot AI commented Jul 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

supabase Bot commented Jul 5, 2026 •

edited

Loading