Skip to content

fix(ingestion): R5 — deep-merge document metadata instead of full replace#408

Merged
BigSimmo merged 11 commits into
mainfrom
claude/r5-worker-metadata-clobber
Jul 8, 2026
Merged

fix(ingestion): R5 — deep-merge document metadata instead of full replace#408
BigSimmo merged 11 commits into
mainfrom
claude/r5-worker-metadata-clobber

Conversation

@BigSimmo

@BigSimmo BigSimmo commented Jul 8, 2026

Copy link
Copy Markdown
Owner

Summary

  • R5 from docs/ingestion-concurrency-fix-workorder.md: commit_document_index_generation and worker updateDocument no longer full-replace documents.metadata.
  • Adds jsonb_merge_deep + apply_document_metadata_patch; commit RPC merges p_metadata onto live metadata.
  • Worker sends worker-owned key deltas only (drops stale claim-time metadata spread).
  • Focused offline merge-semantics tests in tests/document-metadata-merge.test.ts.

Deploy order

  1. Apply migration first (signature unchanged; merge is backward-compatible for old full-object callers).
  2. Deploy worker second so it emits deltas only.

Known gap — do not merge yet

supabase/drift-manifest.json is not regenerated here. Docker Desktop on this host fails with Docker Desktop is unable to start. tests/drift-detection.test.ts will fail CI until npm run drift:manifest is run against this schema.sql from an environment with a working Docker daemon.

Intentionally not included

Test plan

  • npx vitest run tests/document-metadata-merge.test.ts
  • npm run drift:manifest once Docker works, then re-push
  • npm run check:drift
  • Apply migration to live after CI green; smoke worker:once commit preserves a concurrent rename key

BigSimmo and others added 2 commits July 9, 2026 00:00
…lace

Stop worker/commit paths from clobbering concurrent renames and agent
patches by merging worker-owned metadata deltas onto live JSONB.

KNOWN GAP: supabase/drift-manifest.json not regenerated — Docker Desktop
cannot start on this host. Do not merge until npm run drift:manifest is green.

Co-authored-by: Cursor <cursoragent@cursor.com>
…ct tests

JSON null in jsonb_merge_deep now clears sticky error/gate fields that full-replace used to wipe. Worker falls back to a shallow live merge when the R5 RPC is missing, and tests lock the owned-delta contract.

Co-authored-by: Cursor <cursoragent@cursor.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

BigSimmo and others added 4 commits July 9, 2026 00:22
CI verify failed solely on format:check for the new R5 test file.

Co-authored-by: Cursor <cursoragent@cursor.com>
Local Docker Desktop/WSL cannot start on this host, so regenerate
supabase/drift-manifest.json on ubuntu-latest and download the artifact.

Co-authored-by: Cursor <cursoragent@cursor.com>
Local Docker Desktop cannot start (WSL service unavailable). Push a
path-filtered regen workflow and marker file so ubuntu-latest can
produce supabase/drift-manifest.json as an artifact.

Co-authored-by: Cursor <cursoragent@cursor.com>
Regenerated via CI (ubuntu Docker) because local Docker Desktop/WSL
cannot start. schema_sha256 now matches supabase/schema.sql after the
jsonb_merge_deep / apply_document_metadata_patch additions.

Co-authored-by: Cursor <cursoragent@cursor.com>
@BigSimmo BigSimmo closed this Jul 8, 2026
@BigSimmo BigSimmo reopened this Jul 8, 2026
@supabase

supabase Bot commented Jul 8, 2026

Copy link
Copy Markdown

Updates to Preview Branch (claude/r5-worker-metadata-clobber) ↗︎

Deployments Status Updated
Database Wed, 08 Jul 2026 17:14:18 UTC
Services Wed, 08 Jul 2026 17:14:18 UTC
APIs Wed, 08 Jul 2026 17:14:18 UTC

Tasks are run on every commit but only new migration files are pushed.
Close and reopen this PR if you want to apply changes from existing seed or migration files.

Tasks Status Updated
Configurations Wed, 08 Jul 2026 17:14:20 UTC
Migrations Wed, 08 Jul 2026 17:14:22 UTC
Seeding ⏸️ Wed, 08 Jul 2026 17:14:11 UTC
Edge Functions ⏸️ Wed, 08 Jul 2026 17:14:11 UTC

❌ Branch Error • Wed, 08 Jul 2026 17:14:23 UTC

ERROR: duplicate key value violates unique constraint "schema_migrations_pkey" (SQLSTATE 23505)
Key (version)=(20260708160000) already exists.
At statement: 3
INSERT INTO supabase_migrations.schema_migrations(version, name, statements) VALUES($1, $2, $3)

View logs for this Workflow Run ↗︎.
Learn more about Supabase for Git ↗︎.

Co-authored-by: Cursor <cursoragent@cursor.com>
@BigSimmo
BigSimmo enabled auto-merge (squash) July 8, 2026 17:04
BigSimmo and others added 4 commits July 9, 2026 01:06
Co-authored-by: Cursor <cursoragent@cursor.com>
Drift manifest is refreshed; remove the path-triggered helper workflow so
standard pull_request CI runs again on this PR.

Co-authored-by: Cursor <cursoragent@cursor.com>
Resolve branch conflicts with main (post-#405/#406). Drift manifest will
be regenerated via CI because local Docker Desktop/WSL is unavailable.

Co-authored-by: Cursor <cursoragent@cursor.com>
Regenerated from merged schema.sql (R5 metadata merge + R17/R406 main).

Co-authored-by: Cursor <cursoragent@cursor.com>
@BigSimmo
BigSimmo merged commit a5d54a4 into main Jul 8, 2026
5 of 6 checks passed
@BigSimmo BigSimmo mentioned this pull request Jul 9, 2026
@BigSimmo
BigSimmo deleted the claude/r5-worker-metadata-clobber branch July 9, 2026 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant