fix(worker): preserve distinct image placements in cheap image de-duplication

[BigSimmo]

Summary

• Cheap image de-duplication now keys on placement (hash + source kind + page + bbox) so identical bytes on different pages are retained for classification/captioning.
• Enrich-documents and the worker both use the placement key; missing/malformed bboxes no longer collapse into a shared sentinel key.
• Supersedes fix(worker): preserve distinct image placements in cheap image de-duplication #881 with a policy-compliant description.

Verification

• npm run test -- --run tests/image-filtering.test.ts (21 passed)
• Focused unit coverage for placement-aware de-dupe and unknown-bbox uniqueness
• Verification not run: full npm run verify:pr-local deferred; offline unit proof covers the dedupe contract and CI Static/Unit/Build gates apply on this PR.

Risk and rollout

• Risk: medium; changes which visual crops survive cheap pre-filter before captioning, which can change ingestion outputs for multi-page duplicate-byte images.
• Rollback: revert the merge commit on main; re-run affected enrichment/worker jobs for documents ingested under the new key if needed.
• Provider or production effects: None for live OpenAI/Supabase calls in this PR; worker behavior changes on next deploy/run.

Clinical Governance Preflight

• Source-backed claims still require linked source verification before clinical use
• No patient-identifiable document workflow was introduced or expanded without explicit governance approval
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
• Service-role keys and private document access remain server-only
• Demo/synthetic content remains clearly separated from real clinical sources
• Source metadata, review status, and outdated/unknown-source behavior remain conservative
• Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed

Notes

• Addresses Codex/CodeRabbit findings from fix(worker): preserve distinct image placements in cheap image de-duplication #881 (enrich-documents key parity and unknown-bbox collision).

  

Summary by CodeRabbit

• Bug Fixes
  • Improved image duplicate detection by considering placement details, such as page and position.
  • Prevented distinct images with identical content from being incorrectly skipped.
  • Preserved images with missing or invalid placement information instead of merging them incorrectly.
  • Standardized equivalent image positions to ensure true duplicates are still filtered consistently.

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 8aa4970a-d408-4e53-958e-18f56fefe04c

📥 Commits

Reviewing files that changed from the base of the PR and between 44a4c51 and 8c2f95c.

📒 Files selected for processing (4)

• scripts/enrich-documents.ts
• src/lib/image-filtering.ts
• tests/image-filtering.test.ts
• worker/main.ts

---

📝 Walkthrough

Walkthrough

Image deduplication now uses normalized image placement alongside hash, source kind, and page number. Unknown or malformed bounding boxes do not produce dedupe keys. Existing-image classification and worker uploads use the placement-aware behavior.

Changes

Image placement deduplication

Layer / File(s)	Summary
Placement key and filtering behavior src/lib/image-filtering.ts, tests/image-filtering.test.ts	The filter input accepts optional page numbers, imagePlacementDedupeKey normalizes comparable bounding boxes, and duplicate detection uses placement-aware keys. Tests cover differing pages, malformed boxes, and rounded-equivalent placements.
Caller integration scripts/enrich-documents.ts, worker/main.ts	Existing-image classification and uploadAndCaptionImages construct placement-aware inputs and track non-null placement keys instead of raw image hashes.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• BigSimmo/Database#881: Contains the same placement-aware deduplication changes across filtering, worker integration, enrichment, and tests.

Suggested labels: codex

Suggested reviewers: claude, copilot

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 14.29% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the worker-focused placement-aware image de-duplication change.
Description check	✅ Passed	The description follows the template and includes summary, verification, risk, governance, and notes.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch cursor/worker-image-placement-dedupe-1ae0

---

_{Comment @coderabbitai help to get the list of available commands.}