From f252b0df25c23f0b511cb5952e2d0a0f8051e65e Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Fri, 24 Jul 2026 17:33:08 +0000 Subject: [PATCH 1/6] test(pdf): red repro for page-edge table crop truncation (#076) Add a minimal authorised synthetic fixture and deliberately-red Python/Vitest contracts that name the failing pymupdf_find_tables stage and expected clip geometry before any retention/padding change from PR #1129. Co-authored-by: BigSimmo --- docs/outstanding-issues.md | 2 +- tests/pdf-extractor.test.ts | 54 ++++++++++ worker/python/fixtures/README.md | 22 ++++ .../malformed-table-crop-page-edge.pdf | Bin 0 -> 4449 bytes .../test_extract_pdf_assets_page_edge_crop.py | 102 ++++++++++++++++++ 5 files changed, 179 insertions(+), 1 deletion(-) create mode 100644 worker/python/fixtures/README.md create mode 100644 worker/python/fixtures/malformed-table-crop-page-edge.pdf create mode 100644 worker/python/test_extract_pdf_assets_page_edge_crop.py diff --git a/docs/outstanding-issues.md b/docs/outstanding-issues.md index 5c7002f7b..c0c308e75 100644 --- a/docs/outstanding-issues.md +++ b/docs/outstanding-issues.md @@ -142,7 +142,7 @@ removed after current-main verification; it is not missing recommended work. | #040 | P3 | rec | Add targeted visual-regression baselines | Keep a small approved baseline set for high-value desktop/mobile surfaces and accessibility modes instead of screenshotting every route. Start with account/settings, document viewer, mode homes and bottom-composer interactions; define an intentional-update workflow before enabling blocking comparisons. | design audit reconciliation; session 2026-07-22 | 2026-07-22 | | #041 | P3 | rec | Extend the existing Factsheets reading model | Do not add a second patient-facing Factsheets mode. Future patient-content work should extend the existing Easy Read/Standard presentation and its accessibility/content contracts. Revisit only with a concrete user need and source-governance plan. | design audit reconciliation; session 2026-07-22 | 2026-07-22 | | #075 | P2 | issue | Search-scope label enumeration can truncate after 1,000 rows | **Outcome:** every owner-visible source label can participate in scoped search without silently dropping rows at the Supabase response cap. **Next:** reproduce on current `main` with more than 1,000 distinct labels, then add bounded deterministic pagination and a focused multi-page regression. **Verify:** search-scope unit/API contracts, offline RAG contracts, `verify:cheap`, and production-readiness. **Stop:** do not replay mixed PR #1132, widen aliases, change ranking, or run a live canary unless the isolated fix demonstrably changes protected retrieval behaviour. | PR #1132 (`src/lib/search-scope.ts`, `tests/search-scope.test.ts`); primary reconciliation 2026-07-24 | 2026-07-24 | -| #076 | P3 | task | Reproduce malformed fallback PDF image/table crops | **Outcome:** a real current-main fixture proves whether fallback images or table crops are malformed before retention/storage behaviour changes. **Next:** capture one minimal authorised fixture, identify the exact extraction stage and expected geometry, and add a red Python/DOM regression. **Success:** the smallest fix preserves valid assets within existing count/byte budgets and signed-image privacy boundaries. **Stop:** do not merge broad PR #1129, arbitrary crop thresholds/padding, storage expansion, or timeout increases without the reproducer and budget evidence. | PR #1129; `worker/python/extract_pdf_assets.py`; `src/lib/image-filtering.ts`; primary reconciliation 2026-07-24 | 2026-07-24 | +| #076 | P3 | task | Reproduce malformed fallback PDF image/table crops | **Outcome:** a real current-main fixture proves whether fallback images or table crops are malformed before retention/storage behaviour changes. **Repro (landed):** synthetic fixture `worker/python/fixtures/malformed-table-crop-page-edge.pdf` + README; red contracts in `worker/python/test_extract_pdf_assets_page_edge_crop.py` (`expectedFailure`) and `tests/pdf-extractor.test.ts` (`it.fails`). **Failing stage:** `page.find_tables()` / `extraction_method=pymupdf_find_tables` — candidate bbox ends at last fully detected row (`y1=824`) so `save_page_crop` clip bottom is `828` and drops ~14px of still-on-page remnant for the straddling score-`5` row (`y0=824`..`page_h=841.89`) plus the structured row; no `crop_completeness` signal. **Next fix (minimal):** after `find_tables`, extend the candidate (or mark incompleteness) when drawings/text continue to the page edge so clip bottom reaches `page_height-1` or `crop_completeness`/`warning` is emitted; keep budgets and signed-image privacy unchanged. **Success:** smallest fix preserves valid assets within existing count/byte budgets. **Stop:** do not merge broad PR #1129, arbitrary crop thresholds/padding, storage expansion, or timeout increases without this geometry proof. | PR #1129; `worker/python/fixtures/`; `worker/python/test_extract_pdf_assets_page_edge_crop.py`; `tests/pdf-extractor.test.ts`; session 2026-07-24 | 2026-07-24 | | #077 | P2 | issue | Concurrent tasks can re-dirty the canonical primary checkout | **Outcome:** `C:\Dev\Apps\Database` remains a clean synchronization target while write work happens in task-owned worktrees. **Next:** add a cooperative owner/lease check to task-start and lifecycle transitions; before a primary write, branch switch, fast-forward, or cleanup, report owner, dirty state, and Git operation markers and fail closed on an active owner. Include stale-lease recovery. **Success:** a focused concurrency test refuses a second primary writer while read-only commands and separate worktrees continue normally. **Stop:** do not add an OS-wide lock, kill processes, discard existing dirty files, or serialize independent feature worktrees. | primary re-dirtied by another active task immediately after reconciliation proof; session 2026-07-24 | 2026-07-24 | | #078 | P3 | task | Generate a deterministic reconciliation evidence pack | **Outcome:** one report-only command produces the final local evidence now assembled manually. **Next:** extend the reconciliation lifecycle with an explicit output path and include the frozen base/HEAD, per-worktree dispositions, operation markers resolved through Git, archive refs, bundle path/size/SHA-256/verify result, retained worktree count, and local/base tree equality. Accept remote PR state only as explicit approved input. **Success:** fixture tests prove deterministic output and redaction; an interrupted run leaves no false completion record. **Stop:** never fetch, call GitHub/providers, inspect secret values, mutate refs, or delete work implicitly. | `scripts/reconciliation-preflight.mjs`; `docs/reconciliation-playbook.md`; session 2026-07-24 | 2026-07-24 | | #079 | P3 | task | Disposition retained worktrees in bounded cleanup batches | **Outcome:** the retained reconciliation tail is gradually classified without another disruptive all-worktree sweep. **Next:** process no more than ten worktrees per explicitly scheduled pass using current owner/process metadata, open-PR state, exact review-ledger coverage, ancestry, and cherry-pick-aware content proof. **Success:** remove only clean, inactive, bundled worktrees whose content is merged or explicitly rejected; record every disposition and retain recovery evidence. **Stop:** preserve dirty, active, secret-bearing, post-freeze, paused, or ambiguous work and never use reset, force deletion, broad clean, or process killing. | final reconciliation inventory retained 104 independent worktrees; session 2026-07-24 | 2026-07-24 | diff --git a/tests/pdf-extractor.test.ts b/tests/pdf-extractor.test.ts index e5c964a70..4a6ab47bd 100644 --- a/tests/pdf-extractor.test.ts +++ b/tests/pdf-extractor.test.ts @@ -154,4 +154,58 @@ describe.runIf(hasPyMuPDF)("Python PDF table extraction", () => { expect(tableCrop?.metadata?.table_role).toBe("admin"); expect(tableCrop?.metadata?.accessible_table_markdown).toContain("Published date"); }); + + // #076 deliberately-red contract: page-edge table crops currently omit the on-page + // remnant of a straddling final row at the pymupdf_find_tables stage. it.fails keeps + // CI green while the defect remains; flip to it() once geometry is fixed. + it.fails( + "includes on-page remnant geometry for a table row that straddles the page bottom (#076)", + async () => { + const root = await mkdtemp(path.join(tmpdir(), "clinical-kb-page-edge-crop-")); + const imageDir = path.join(root, "images"); + const jsonPath = path.join(root, "extract.json"); + await mkdir(imageDir, { recursive: true }); + + const fixturePath = path.join( + process.cwd(), + "worker", + "python", + "fixtures", + "malformed-table-crop-page-edge.pdf", + ); + const result = spawnSync( + pythonBin, + [path.join(process.cwd(), "worker", "python", "extract_pdf_assets.py"), fixturePath, imageDir, jsonPath], + { cwd: process.cwd(), encoding: "utf8" }, + ); + + expect(result.status).toBe(0); + const payload = JSON.parse(await readFile(jsonPath, "utf8")) as { + images: Array<{ + sourceKind?: string; + bbox?: number[]; + metadata?: Record; + }>; + warnings?: string[]; + }; + const tableCrop = payload.images.find((image) => image.sourceKind === "table_crop"); + expect(tableCrop).toBeTruthy(); + + const pageHeight = Number(tableCrop?.metadata?.page_height); + const clip = (tableCrop?.bbox ?? tableCrop?.metadata?.clip_bbox) as number[] | undefined; + expect(clip).toEqual(expect.any(Array)); + // Stage: pymupdf_find_tables → save_page_crop. Expected geometry: clip bottom + // reaches the page edge so the on-page remnant (y≈824..841.89) is retained. + expect(Number(clip?.[3])).toBeGreaterThanOrEqual(pageHeight - 1); + + const rows = (tableCrop?.metadata?.table_rows as string[][] | undefined) ?? []; + const hasScoreRow = rows.some((row) => String(row?.[0] ?? "").trim() === "5"); + const cropCompleteness = Number(tableCrop?.metadata?.crop_completeness); + const incomplete = + Number.isFinite(cropCompleteness) && cropCompleteness < 0.99 + ? true + : (payload.warnings ?? []).some((warning) => /crop/i.test(warning) && /incomplete/i.test(warning)); + expect(hasScoreRow || incomplete).toBe(true); + }, + ); }); diff --git a/worker/python/fixtures/README.md b/worker/python/fixtures/README.md new file mode 100644 index 000000000..c96653f1d --- /dev/null +++ b/worker/python/fixtures/README.md @@ -0,0 +1,22 @@ +# PDF extraction fixtures + +## `malformed-table-crop-page-edge.pdf` + +Minimal authorised **synthetic** clinical-table PDF for outstanding issue `#076`. + +- No patient identifiers, no third-party copyrighted content. +- A4 page (`595.28 × 841.89`), one titled 4-row table whose final row straddles the page bottom (`y0=824` … `y1=860`; page ends at `841.89`). +- Regenerated with PyMuPDF via the geometry constants pinned in `test_extract_pdf_assets_page_edge_crop.py`. + +### Failing stage (current `main`) + +`page.find_tables()` (`extraction_method=pymupdf_find_tables`) returns a candidate bbox that ends at the last **fully** detected row (`y1=824`). `expanded_rect(..., 4, 4)` then yields crop clip `[46, 712, 564, 828]`, so: + +- ~14 px of still-on-page content for the straddling score-`5` row is omitted from the `table_crop` pixmap +- structured `table_rows` silently drops that row +- no `crop_completeness` / incompleteness warning is emitted before `save_page_crop` + +### Expected geometry after a fix + +- `table_crop.bbox[3]` (clip bottom) ≥ `page_height - 1` **or** an explicit incompleteness signal (`crop_completeness < 1` / warning) when on-page remnant cannot be recovered +- structured rows either include the on-page remnant of score `5`, or the incomplete extraction is marked so retention/viewer code can treat it as cut-off diff --git a/worker/python/fixtures/malformed-table-crop-page-edge.pdf b/worker/python/fixtures/malformed-table-crop-page-edge.pdf new file mode 100644 index 0000000000000000000000000000000000000000..df36d10bece96c2298686eb1329cddbb1b0750b4 GIT binary patch literal 4449 zcmbVQ4^R}>8E1k-XCXqwfQh#ADgp|4yYIdIbE}kacW{vqqTxs)Axv^`3!DRYX75nc zN{l1vOvQ+?qm7F4C&`%Vq)iENLJS6?)`-b;#|@t?s|VMyUJaw1OOsWOW7h{jh{SUrTRUM)rz`!BEph9axsd} zLqQlied6&$CfGqf1_GW2vK9Dgq*4nk92kpVw8LfREYE$ zkhHs1yUW@gQs`0uX{g9QZ54nT5`Y>V05xv`)FGgb0d)|lqaYmx=_p7?K{^W3QIL*; zbQGkc5I)o_(owL6GBJE~&BUnC7_zR$svylKO(|sU?tr2VuONX{g5GL3Tj;Cyd%V6{ z_EE3T<_ma#{5~Ns)Qg+I4=BFcU>zaLu}*J&P*GWDJ&A0ALO>LfQB0MRw9+ANOg_x4(aU#&e$yyqkS{pIW!CGQcTsE|L2@bu_hr%a$$K#aD{auo1Sl%rOcjOfCo%!nSX+$3_u z+|(>4l%JbbA}m8xhano#>(E?O+ShfQytc@a zcYNTJlx?T?&B)&9+4bv3UVq{4?^Ye`p5LtA^Xh*OZ9M1v(zero$H9?xe^_3bRe5FR zz84prh)@i*SDsOQW^F_1Cu&=JlN!;5Ni{*V&GWjp1yilXQ|DTHR~rpQ5O;`2)Z*5f z03$x+&@{Oo&AO~ImrxlOV|TVOWWgdKeT>$0L2Ab+W3~b`Mc-h z;vNr9?(qbQ(_*nBkk-QoT!D@?I63KJ&Rqquh0j!tgP55*#C-D@q75g#IXnpl*y!r8aMgc+Mo46^^M$}G>G)0N-VxqQjb zy0#uz+|hCAhx)^9`^TG(f3>GN_54Zr{@7z@E!`PMMxN-)J^i2V&dukC`uT**Ys*&u z<;!h%zhu!ogaY1+d<0K9qnt|R~Kzt0Gc@m}iTNEwaLX&FO7g?}p)Q7d0->5Z`_4b?QtFC@yh!vx1S2Nq^-$!J*6 zzinRI>W|v`evwgkDke7mUw?ku{_{E4%lGVD@Ye5A&c43ybD!(ZQ~RzR^Bo!;|Lcc& z<9$0<%wcEcCS6(494qV<(vEN!+{1r-vWgSF8f*C&v>y+8`YkVI-ne=+Z}zd~J+8gm zzmFL`v@kY8fSOdLhP1~RGF<2aq99-a)PT?!PYyiv0>01~m5^ynGS5S_RSH|n%j6st zHYQljc#@T*FKnLGYPKQE@()DDb5&@{yfYEM%WZ9 z%yZ^3fv6|6mIpqbLWK@Y zK4H6rc=`$2@^Z;da*8%Pan*^*!Nh45F>ggS4vJ1eF5vU%K|2t1t3gfVq(ekZdV1kX GC-Z+culLIU literal 0 HcmV?d00001 diff --git a/worker/python/test_extract_pdf_assets_page_edge_crop.py b/worker/python/test_extract_pdf_assets_page_edge_crop.py new file mode 100644 index 000000000..c83e1caf4 --- /dev/null +++ b/worker/python/test_extract_pdf_assets_page_edge_crop.py @@ -0,0 +1,102 @@ +"""Red regression for #076: page-edge table crops omit on-page remnant geometry. + +Failing stage: pymupdf_find_tables candidate detection (before save_page_crop). +Expected geometry: table_crop clip bottom reaches the page bottom (or reports incompleteness) +so the on-page remnant of a straddling final row is not silently dropped. + +This module uses unittest.expectedFailure until a minimal geometry fix lands. +Do not "fix" by widening padding/timeouts/storage — see docs/outstanding-issues.md #076. +""" + +from __future__ import annotations + +import os +import sys +import tempfile +import unittest + +sys.path.insert(0, os.path.dirname(__file__)) +import extract_pdf_assets as extractor + +FIXTURE_DIR = os.path.join(os.path.dirname(__file__), "fixtures") +FIXTURE_PDF = os.path.join(FIXTURE_DIR, "malformed-table-crop-page-edge.pdf") + +# Geometry pinned by the checked-in synthetic fixture (A4). +PAGE_WIDTH = 595.28 +PAGE_HEIGHT = 841.89 +STRADDLING_ROW_Y0 = 824.0 +EXPECTED_CLIP_BOTTOM = PAGE_HEIGHT # include on-page remnant through the page edge +EXPECTED_SCORE_ROW = "5" + + +@unittest.skipUnless(os.path.isfile(FIXTURE_PDF), "page-edge crop fixture missing") +class PageEdgeTableCropGeometryTests(unittest.TestCase): + def test_fixture_find_tables_stage_drops_straddling_row_bbox(self): + """Diagnosis probe: names the exact failing stage without requiring a green crop yet.""" + document = extractor.fitz.open(FIXTURE_PDF) + try: + page = document[0] + self.assertAlmostEqual(float(page.rect.width), PAGE_WIDTH, places=2) + self.assertAlmostEqual(float(page.rect.height), PAGE_HEIGHT, places=2) + + tables = page.find_tables() + self.assertGreaterEqual(len(tables.tables), 1, "fixture must expose a find_tables candidate") + candidate = extractor.fitz.Rect(tables.tables[0].bbox) + # Current-main defect: candidate stops at the last fully detected row. + self.assertLess( + candidate.y1, + EXPECTED_CLIP_BOTTOM - 1, + "diagnosis changed: find_tables now includes the page-edge remnant", + ) + self.assertAlmostEqual(candidate.y1, STRADDLING_ROW_Y0, places=1) + finally: + document.close() + + @unittest.expectedFailure + def test_table_crop_includes_on_page_remnant_of_straddling_row(self): + """ + Red contract (#076). + + Stage: pymupdf_find_tables → expanded_rect → save_page_crop (sourceKind=table_crop) + Expected geometry: clip_bbox[3] >= page_height - 1 (on-page remnant retained) + Expected structure: table_rows includes the score-5 clinical row (or incompleteness is marked) + """ + with tempfile.TemporaryDirectory(prefix="page-edge-crop-") as output_dir: + result = extractor.extract(FIXTURE_PDF, output_dir) + crops = [image for image in result["images"] if image.get("sourceKind") == "table_crop"] + self.assertEqual(len(crops), 1, result) + crop = crops[0] + metadata = crop.get("metadata") or {} + clip = crop.get("bbox") or metadata.get("clip_bbox") + self.assertIsInstance(clip, list) + self.assertGreaterEqual(len(clip), 4) + + clip_bottom = float(clip[3]) + page_height = float(metadata.get("page_height") or PAGE_HEIGHT) + self.assertGreaterEqual( + clip_bottom, + page_height - 1.0, + ( + "stage=pymupdf_find_tables→save_page_crop; " + f"expected clip bottom ≥ {page_height - 1.0}, got {clip_bottom}; " + f"candidate method={metadata.get('extraction_method')}" + ), + ) + + rows = metadata.get("table_rows") or [] + has_score_row = any(str(row[0]).strip() == EXPECTED_SCORE_ROW for row in rows if row) + crop_completeness = metadata.get("crop_completeness") + incomplete = ( + isinstance(crop_completeness, (int, float)) and float(crop_completeness) < 0.99 + ) or any("crop" in str(warning).lower() and "incomplete" in str(warning).lower() for warning in result.get("warnings") or []) + self.assertTrue( + has_score_row or incomplete, + ( + "expected score-5 row in table_rows or an explicit incompleteness signal; " + f"rows={rows!r} crop_completeness={crop_completeness!r} warnings={result.get('warnings')!r}" + ), + ) + + +if __name__ == "__main__": + unittest.main() From 5391bf185cd5dffd00a31eb1d282ccfc93277a73 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Fri, 24 Jul 2026 17:44:03 +0000 Subject: [PATCH 2/6] fix(pdf): retain on-page remnant for page-edge table crops (#076) Extend find_tables candidates using contiguous cell drawings so straddling final rows are cropped to the page edge, recover structured remnant text when possible, and emit an incompleteness warning when content continues past the page. Flip the #076 red contracts green and archive the ledger item. Co-authored-by: BigSimmo --- docs/outstanding-issues.md | 2 +- tests/pdf-extractor.test.ts | 14 +- worker/python/extract_pdf_assets.py | 221 ++++++++++++++++-- worker/python/fixtures/README.md | 20 +- .../test_extract_pdf_assets_page_edge_crop.py | 41 ++-- 5 files changed, 242 insertions(+), 56 deletions(-) diff --git a/docs/outstanding-issues.md b/docs/outstanding-issues.md index c0c308e75..c1468efb2 100644 --- a/docs/outstanding-issues.md +++ b/docs/outstanding-issues.md @@ -142,7 +142,6 @@ removed after current-main verification; it is not missing recommended work. | #040 | P3 | rec | Add targeted visual-regression baselines | Keep a small approved baseline set for high-value desktop/mobile surfaces and accessibility modes instead of screenshotting every route. Start with account/settings, document viewer, mode homes and bottom-composer interactions; define an intentional-update workflow before enabling blocking comparisons. | design audit reconciliation; session 2026-07-22 | 2026-07-22 | | #041 | P3 | rec | Extend the existing Factsheets reading model | Do not add a second patient-facing Factsheets mode. Future patient-content work should extend the existing Easy Read/Standard presentation and its accessibility/content contracts. Revisit only with a concrete user need and source-governance plan. | design audit reconciliation; session 2026-07-22 | 2026-07-22 | | #075 | P2 | issue | Search-scope label enumeration can truncate after 1,000 rows | **Outcome:** every owner-visible source label can participate in scoped search without silently dropping rows at the Supabase response cap. **Next:** reproduce on current `main` with more than 1,000 distinct labels, then add bounded deterministic pagination and a focused multi-page regression. **Verify:** search-scope unit/API contracts, offline RAG contracts, `verify:cheap`, and production-readiness. **Stop:** do not replay mixed PR #1132, widen aliases, change ranking, or run a live canary unless the isolated fix demonstrably changes protected retrieval behaviour. | PR #1132 (`src/lib/search-scope.ts`, `tests/search-scope.test.ts`); primary reconciliation 2026-07-24 | 2026-07-24 | -| #076 | P3 | task | Reproduce malformed fallback PDF image/table crops | **Outcome:** a real current-main fixture proves whether fallback images or table crops are malformed before retention/storage behaviour changes. **Repro (landed):** synthetic fixture `worker/python/fixtures/malformed-table-crop-page-edge.pdf` + README; red contracts in `worker/python/test_extract_pdf_assets_page_edge_crop.py` (`expectedFailure`) and `tests/pdf-extractor.test.ts` (`it.fails`). **Failing stage:** `page.find_tables()` / `extraction_method=pymupdf_find_tables` — candidate bbox ends at last fully detected row (`y1=824`) so `save_page_crop` clip bottom is `828` and drops ~14px of still-on-page remnant for the straddling score-`5` row (`y0=824`..`page_h=841.89`) plus the structured row; no `crop_completeness` signal. **Next fix (minimal):** after `find_tables`, extend the candidate (or mark incompleteness) when drawings/text continue to the page edge so clip bottom reaches `page_height-1` or `crop_completeness`/`warning` is emitted; keep budgets and signed-image privacy unchanged. **Success:** smallest fix preserves valid assets within existing count/byte budgets. **Stop:** do not merge broad PR #1129, arbitrary crop thresholds/padding, storage expansion, or timeout increases without this geometry proof. | PR #1129; `worker/python/fixtures/`; `worker/python/test_extract_pdf_assets_page_edge_crop.py`; `tests/pdf-extractor.test.ts`; session 2026-07-24 | 2026-07-24 | | #077 | P2 | issue | Concurrent tasks can re-dirty the canonical primary checkout | **Outcome:** `C:\Dev\Apps\Database` remains a clean synchronization target while write work happens in task-owned worktrees. **Next:** add a cooperative owner/lease check to task-start and lifecycle transitions; before a primary write, branch switch, fast-forward, or cleanup, report owner, dirty state, and Git operation markers and fail closed on an active owner. Include stale-lease recovery. **Success:** a focused concurrency test refuses a second primary writer while read-only commands and separate worktrees continue normally. **Stop:** do not add an OS-wide lock, kill processes, discard existing dirty files, or serialize independent feature worktrees. | primary re-dirtied by another active task immediately after reconciliation proof; session 2026-07-24 | 2026-07-24 | | #078 | P3 | task | Generate a deterministic reconciliation evidence pack | **Outcome:** one report-only command produces the final local evidence now assembled manually. **Next:** extend the reconciliation lifecycle with an explicit output path and include the frozen base/HEAD, per-worktree dispositions, operation markers resolved through Git, archive refs, bundle path/size/SHA-256/verify result, retained worktree count, and local/base tree equality. Accept remote PR state only as explicit approved input. **Success:** fixture tests prove deterministic output and redaction; an interrupted run leaves no false completion record. **Stop:** never fetch, call GitHub/providers, inspect secret values, mutate refs, or delete work implicitly. | `scripts/reconciliation-preflight.mjs`; `docs/reconciliation-playbook.md`; session 2026-07-24 | 2026-07-24 | | #079 | P3 | task | Disposition retained worktrees in bounded cleanup batches | **Outcome:** the retained reconciliation tail is gradually classified without another disruptive all-worktree sweep. **Next:** process no more than ten worktrees per explicitly scheduled pass using current owner/process metadata, open-PR state, exact review-ledger coverage, ancestry, and cherry-pick-aware content proof. **Success:** remove only clean, inactive, bundled worktrees whose content is merged or explicitly rejected; record every disposition and retain recovery evidence. **Stop:** preserve dirty, active, secret-bearing, post-freeze, paused, or ambiguous work and never use reset, force deletion, broad clean, or process killing. | final reconciliation inventory retained 104 independent worktrees; session 2026-07-24 | 2026-07-24 | @@ -153,6 +152,7 @@ Move resolved rows here with the resolution date and a one-line outcome. Keep th | ID | Type | Summary | Outcome | Resolved | | ---- | ----- | ---------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | +| #076 | task | Reproduce malformed fallback PDF image/table crops | Reproduced truncated page-edge `table_crop`s on current-main with `worker/python/fixtures/malformed-table-crop-page-edge.pdf`. Root cause: `pymupdf_find_tables` stops at the last fully detected row; fix extends the candidate from contiguous cell drawings, recovers the on-page score-5 remnant, and emits `table_crop_edge_incomplete` / `crop_completeness=0.9` when content continues past the page. PR #1176. Broad PR #1129 retention/padding changes not merged. | 2026-07-24 | | #070 | issue | Presentation mobile tabs misroute Overview/Map/Related | Fixed in PR #1135: Overview/Map/Related deep-link to diagnosis `?tab=` sections; Compare stays on the presentation page. Regression in `tests/mobile-interaction-regressions.test.ts`. (Provisional PR-branch IDs `#068`–`#072` were renumbered after `main` claimed `#068`/`#069`.) | 2026-07-24 | | #071 | issue | Evidence/Clinical Notes Add fakes success without persistence | Fixed in PR #1135: sticky Add controls use the focusable coming-soon placeholder pattern instead of optimistic `setAdded(true)`. | 2026-07-24 | | #072 | issue | Tools hub exposes false Sort/More affordances | Fixed in PR #1135: Sort is a status label, More filter targets coordination/saved without a fake menu chevron, and the favourites shortcut is labelled Saved/Favourites. | 2026-07-24 | diff --git a/tests/pdf-extractor.test.ts b/tests/pdf-extractor.test.ts index 4a6ab47bd..4706efb74 100644 --- a/tests/pdf-extractor.test.ts +++ b/tests/pdf-extractor.test.ts @@ -155,10 +155,9 @@ describe.runIf(hasPyMuPDF)("Python PDF table extraction", () => { expect(tableCrop?.metadata?.accessible_table_markdown).toContain("Published date"); }); - // #076 deliberately-red contract: page-edge table crops currently omit the on-page - // remnant of a straddling final row at the pymupdf_find_tables stage. it.fails keeps - // CI green while the defect remains; flip to it() once geometry is fixed. - it.fails( + // #076: page-edge table crops must keep the on-page remnant of a straddling + // final row after find_tables truncates the candidate bbox. + it( "includes on-page remnant geometry for a table row that straddles the page bottom (#076)", async () => { const root = await mkdtemp(path.join(tmpdir(), "clinical-kb-page-edge-crop-")); @@ -194,8 +193,8 @@ describe.runIf(hasPyMuPDF)("Python PDF table extraction", () => { const pageHeight = Number(tableCrop?.metadata?.page_height); const clip = (tableCrop?.bbox ?? tableCrop?.metadata?.clip_bbox) as number[] | undefined; expect(clip).toEqual(expect.any(Array)); - // Stage: pymupdf_find_tables → save_page_crop. Expected geometry: clip bottom - // reaches the page edge so the on-page remnant (y≈824..841.89) is retained. + // Stage: pymupdf_find_tables → extend_table_rect_for_edge_content → save_page_crop. + // Expected geometry: clip bottom reaches the page edge so the on-page remnant is retained. expect(Number(clip?.[3])).toBeGreaterThanOrEqual(pageHeight - 1); const rows = (tableCrop?.metadata?.table_rows as string[][] | undefined) ?? []; @@ -204,8 +203,9 @@ describe.runIf(hasPyMuPDF)("Python PDF table extraction", () => { const incomplete = Number.isFinite(cropCompleteness) && cropCompleteness < 0.99 ? true - : (payload.warnings ?? []).some((warning) => /crop/i.test(warning) && /incomplete/i.test(warning)); + : (payload.warnings ?? []).some((warning) => warning.includes("table_crop_edge_incomplete")); expect(hasScoreRow || incomplete).toBe(true); + expect(tableCrop?.metadata?.edge_content_extended).toBe(true); }, ); }); diff --git a/worker/python/extract_pdf_assets.py b/worker/python/extract_pdf_assets.py index f504de76e..cf6198a75 100644 --- a/worker/python/extract_pdf_assets.py +++ b/worker/python/extract_pdf_assets.py @@ -293,6 +293,150 @@ def expanded_rect(rect, page_rect, x_padding=4, y_padding=4): ) +# Max gap between a find_tables bbox edge and continuing cell drawings that +# still belong to the same table. Keeps footers/unrelated blocks from merging in. +TABLE_EDGE_CONTENT_GAP = 24.0 + + +def _rects_overlap_horizontally(a, b, tolerance=2.0): + return not (a.x1 < b.x0 - tolerance or a.x0 > b.x1 + tolerance) + + +def _rects_overlap_vertically(a, b, tolerance=2.0): + return not (a.y1 < b.y0 - tolerance or a.y0 > b.y1 + tolerance) + + +def page_drawing_rects(page): + rects = [] + for drawing in page.get_drawings() or []: + rect = drawing.get("rect") + if not rect: + continue + rect = fitz.Rect(rect) + # Ignore hairline page rules / tiny marks. + if rect.width < 8 or rect.height < 8: + continue + rects.append(rect) + return rects + + +def extend_table_rect_for_edge_content(page, rect, max_gap=TABLE_EDGE_CONTENT_GAP): + """Grow a find_tables bbox to keep on-page remnants of straddling table rows/cols. + + `page.find_tables()` often stops at the last fully detected row when the final + row straddles the page edge. Cell drawings for that remnant still exist on the + page; fold them into the crop rectangle (clamped to the page). Text blocks are + intentionally ignored here so titles/footers do not inflate the crop. + """ + page_rect = page.rect + content_rects = page_drawing_rects(page) + x0, y0, x1, y1 = float(rect.x0), float(rect.y0), float(rect.x1), float(rect.y1) + edges = [] + clipped_by_page = False + + def absorb(axis): + nonlocal x0, y0, x1, y1, clipped_by_page + changed = False + current = fitz.Rect(x0, y0, x1, y1) + for content in content_rects: + if axis in ("bottom", "top") and not _rects_overlap_horizontally(current, content): + continue + if axis in ("left", "right") and not _rects_overlap_vertically(current, content): + continue + if axis == "bottom": + if content.y1 <= y1 + 0.5 or content.y0 > y1 + max_gap: + continue + if content.y1 > page_rect.y1 + 0.5: + clipped_by_page = True + next_y1 = min(max(y1, float(content.y1)), float(page_rect.y1)) + if next_y1 > y1 + 0.5: + y1 = next_y1 + changed = True + elif axis == "top": + if content.y0 >= y0 - 0.5 or content.y1 < y0 - max_gap: + continue + if content.y0 < page_rect.y0 - 0.5: + clipped_by_page = True + next_y0 = max(min(y0, float(content.y0)), float(page_rect.y0)) + if next_y0 < y0 - 0.5: + y0 = next_y0 + changed = True + elif axis == "right": + if content.x1 <= x1 + 0.5 or content.x0 > x1 + max_gap: + continue + if content.x1 > page_rect.x1 + 0.5: + clipped_by_page = True + next_x1 = min(max(x1, float(content.x1)), float(page_rect.x1)) + if next_x1 > x1 + 0.5: + x1 = next_x1 + changed = True + elif axis == "left": + if content.x0 >= x0 - 0.5 or content.x1 < x0 - max_gap: + continue + if content.x0 < page_rect.x0 - 0.5: + clipped_by_page = True + next_x0 = max(min(x0, float(content.x0)), float(page_rect.x0)) + if next_x0 < x0 - 0.5: + x0 = next_x0 + changed = True + return changed + + for axis in ("bottom", "top", "right", "left"): + guard = 0 + while absorb(axis) and guard < 8: + if axis not in edges: + edges.append(axis) + guard += 1 + + extended = fitz.Rect(x0, y0, x1, y1) & page_rect + return extended, { + "extended": bool(edges), + "clipped_by_page": clipped_by_page, + "edges": edges, + } + + +def recover_table_rows_from_edge_strip(page, original_rect, extended_rect, existing_rows, column_count=0): + """Recover structured rows from text that continues past a truncated find_tables bbox.""" + existing_keys = { + tuple(clean_cell(cell) for cell in row) + for row in existing_rows or [] + if any(clean_cell(cell) for cell in row) + } + recovered = [] + for block in page.get_text("blocks") or []: + if len(block) < 5: + continue + block_rect = fitz.Rect(block[:4]) + intersection = extended_rect & block_rect + if intersection.is_empty: + continue + # Only consider text that sits in the strip beyond the original detection. + outside_bottom = block_rect.y1 > original_rect.y1 + 0.5 and block_rect.y0 >= original_rect.y1 - 2.0 + outside_top = block_rect.y0 < original_rect.y0 - 0.5 and block_rect.y1 <= original_rect.y0 + 2.0 + outside_right = block_rect.x1 > original_rect.x1 + 0.5 and block_rect.x0 >= original_rect.x1 - 2.0 + outside_left = block_rect.x0 < original_rect.x0 - 0.5 and block_rect.x1 <= original_rect.x0 + 2.0 + if not (outside_bottom or outside_top or outside_right or outside_left): + continue + lines = [clean_cell(line) for line in str(block[4] or "").splitlines() if clean_cell(line)] + if not lines: + continue + # Prefer blocks that match the known column count (one cell per line). + if column_count and len(lines) == column_count: + row = lines + elif column_count and len(lines) > 1 and len(lines) < column_count: + # Partial remnant still useful for search/OCR text. + row = lines + [""] * (column_count - len(lines)) + else: + continue + key = tuple(row) + if key in existing_keys: + continue + existing_keys.add(key) + recovered.append(row) + return recovered + + def rect_intersection_ratio(a, b): intersection = a & b if intersection.is_empty: @@ -937,33 +1081,68 @@ def extract(pdf_path, output_dir, budget=None): table_candidates.extend(fallback_table_candidates(page, [candidate["rect"] for candidate in table_candidates])) table_candidates = merge_related_table_candidates(table_candidates, page.rect) for table_candidate in table_candidates: - table_rect = expanded_rect(table_candidate["rect"], page.rect, 4, 4) + detected_rect = table_candidate["rect"] + edge_rect, edge_info = extend_table_rect_for_edge_content(page, detected_rect) + table_rows = list(table_candidate.get("table_rows") or []) + column_count = int(table_candidate.get("column_count") or (len(table_rows[0]) if table_rows else 0) or 0) + recovered_rows = [] + if edge_info["extended"]: + recovered_rows = recover_table_rows_from_edge_strip( + page, + detected_rect, + edge_rect, + table_rows, + column_count, + ) + if recovered_rows: + table_rows = (table_rows + recovered_rows)[:MAX_TABLE_ROWS] + rows_truncated = bool(table_candidate.get("rows_truncated")) or bool(edge_info["clipped_by_page"]) + if edge_info["clipped_by_page"]: + warnings.append( + f"table_crop_edge_incomplete: page {page_number} table continues past the page edge; " + "retained the on-page remnant in the crop" + ) + table_rect = expanded_rect(edge_rect, page.rect, 4, 4) + table_text = (table_candidate.get("table_text") or "")[:MAX_TABLE_TEXT_CHARS] + accessible = ( + table_candidate.get("accessible_table_markdown") or table_candidate.get("table_text") or "" + )[:MAX_TABLE_TEXT_CHARS] + if recovered_rows: + accessible = (table_rows_to_markdown(table_rows) or accessible)[:MAX_TABLE_TEXT_CHARS] + table_text = accessible + crop_metadata = { + "pageNumber": page_number, + "source_kind": "table_crop", + "candidate_type": "table", + "table_label": table_candidate.get("table_label"), + "table_title": table_candidate.get("table_title"), + "table_text": table_text, + "table_role": table_candidate.get("table_role"), + "table_confidence": table_candidate.get("table_confidence"), + "table_rows": table_rows, + "table_columns": table_candidate.get("table_columns") or [], + "accessible_table_markdown": accessible, + "row_count": len(table_rows) if table_rows else table_candidate.get("row_count"), + "rows_truncated": rows_truncated, + "column_count": column_count or table_candidate.get("column_count"), + "heading_text": table_candidate.get("heading_text"), + "bbox": rect_payload(table_rect), + "extraction_method": table_candidate.get("extraction_method"), + "table_index": table_candidate.get("table_index"), + } + if edge_info["extended"]: + crop_metadata["edge_content_extended"] = True + crop_metadata["edge_content_edges"] = edge_info["edges"] + if edge_info["clipped_by_page"]: + # Partial last row still visible on-page, but structured extraction cannot be complete. + crop_metadata["crop_completeness"] = 0.9 crop = save_page_crop( page, table_rect, output_dir, f"page-{page_number}-table-crop-{crop_index}.png", "table_crop", - { - "pageNumber": page_number, - "source_kind": "table_crop", - "candidate_type": "table", - "table_label": table_candidate.get("table_label"), - "table_title": table_candidate.get("table_title"), - "table_text": (table_candidate.get("table_text") or "")[:MAX_TABLE_TEXT_CHARS], - "table_role": table_candidate.get("table_role"), - "table_confidence": table_candidate.get("table_confidence"), - "table_rows": table_candidate.get("table_rows") or [], - "table_columns": table_candidate.get("table_columns") or [], - "accessible_table_markdown": (table_candidate.get("accessible_table_markdown") or table_candidate.get("table_text") or "")[:MAX_TABLE_TEXT_CHARS], - "row_count": table_candidate.get("row_count"), - "rows_truncated": bool(table_candidate.get("rows_truncated")), - "column_count": table_candidate.get("column_count"), - "heading_text": table_candidate.get("heading_text"), - "bbox": rect_payload(table_rect), - "extraction_method": table_candidate.get("extraction_method"), - "table_index": table_candidate.get("table_index"), - }, + crop_metadata, budget, warnings, ) diff --git a/worker/python/fixtures/README.md b/worker/python/fixtures/README.md index c96653f1d..35b6c67a9 100644 --- a/worker/python/fixtures/README.md +++ b/worker/python/fixtures/README.md @@ -8,15 +8,19 @@ Minimal authorised **synthetic** clinical-table PDF for outstanding issue `#076` - A4 page (`595.28 × 841.89`), one titled 4-row table whose final row straddles the page bottom (`y0=824` … `y1=860`; page ends at `841.89`). - Regenerated with PyMuPDF via the geometry constants pinned in `test_extract_pdf_assets_page_edge_crop.py`. -### Failing stage (current `main`) +### Failing stage (pre-fix) -`page.find_tables()` (`extraction_method=pymupdf_find_tables`) returns a candidate bbox that ends at the last **fully** detected row (`y1=824`). `expanded_rect(..., 4, 4)` then yields crop clip `[46, 712, 564, 828]`, so: +`page.find_tables()` (`extraction_method=pymupdf_find_tables`) returns a candidate bbox that ends at the last **fully** detected row (`y1=824`). Without extension, `expanded_rect(..., 4, 4)` yielded crop clip `[46, 712, 564, 828]`, so: -- ~14 px of still-on-page content for the straddling score-`5` row is omitted from the `table_crop` pixmap -- structured `table_rows` silently drops that row -- no `crop_completeness` / incompleteness warning is emitted before `save_page_crop` +- ~14 px of still-on-page content for the straddling score-`5` row was omitted from the `table_crop` pixmap +- structured `table_rows` silently dropped that row +- no incompleteness warning was emitted -### Expected geometry after a fix +### Fix stage -- `table_crop.bbox[3]` (clip bottom) ≥ `page_height - 1` **or** an explicit incompleteness signal (`crop_completeness < 1` / warning) when on-page remnant cannot be recovered -- structured rows either include the on-page remnant of score `5`, or the incomplete extraction is marked so retention/viewer code can treat it as cut-off +`extend_table_rect_for_edge_content` grows the candidate using contiguous **cell drawings** toward the page edge, then `save_page_crop` renders the clamped remnant. Structured text in the extended strip is recovered when possible; page-clipped continuation sets `rows_truncated`, `crop_completeness=0.9`, and a `table_crop_edge_incomplete` warning. + +### Expected geometry after the fix + +- `table_crop.bbox[3]` (clip bottom) ≥ `page_height - 1` +- structured rows include the on-page remnant of score `5`, or incompleteness is marked diff --git a/worker/python/test_extract_pdf_assets_page_edge_crop.py b/worker/python/test_extract_pdf_assets_page_edge_crop.py index c83e1caf4..7163cc866 100644 --- a/worker/python/test_extract_pdf_assets_page_edge_crop.py +++ b/worker/python/test_extract_pdf_assets_page_edge_crop.py @@ -1,11 +1,9 @@ -"""Red regression for #076: page-edge table crops omit on-page remnant geometry. +"""Regression for #076: page-edge table crops keep on-page remnant geometry. -Failing stage: pymupdf_find_tables candidate detection (before save_page_crop). -Expected geometry: table_crop clip bottom reaches the page bottom (or reports incompleteness) -so the on-page remnant of a straddling final row is not silently dropped. - -This module uses unittest.expectedFailure until a minimal geometry fix lands. -Do not "fix" by widening padding/timeouts/storage — see docs/outstanding-issues.md #076. +Failing stage (pre-fix): pymupdf_find_tables candidate detection stopped at the last +fully detected row. Fix stage: extend_table_rect_for_edge_content + save_page_crop. +Expected geometry: table_crop clip bottom reaches the page bottom and the straddling +score-5 remnant is retained (structured row and/or incompleteness signal). """ from __future__ import annotations @@ -25,14 +23,13 @@ PAGE_WIDTH = 595.28 PAGE_HEIGHT = 841.89 STRADDLING_ROW_Y0 = 824.0 -EXPECTED_CLIP_BOTTOM = PAGE_HEIGHT # include on-page remnant through the page edge EXPECTED_SCORE_ROW = "5" @unittest.skipUnless(os.path.isfile(FIXTURE_PDF), "page-edge crop fixture missing") class PageEdgeTableCropGeometryTests(unittest.TestCase): - def test_fixture_find_tables_stage_drops_straddling_row_bbox(self): - """Diagnosis probe: names the exact failing stage without requiring a green crop yet.""" + def test_fixture_find_tables_stage_still_drops_straddling_row_bbox(self): + """Upstream diagnosis: find_tables alone still truncates; the fix extends afterward.""" document = extractor.fitz.open(FIXTURE_PDF) try: page = document[0] @@ -42,22 +39,24 @@ def test_fixture_find_tables_stage_drops_straddling_row_bbox(self): tables = page.find_tables() self.assertGreaterEqual(len(tables.tables), 1, "fixture must expose a find_tables candidate") candidate = extractor.fitz.Rect(tables.tables[0].bbox) - # Current-main defect: candidate stops at the last fully detected row. self.assertLess( candidate.y1, - EXPECTED_CLIP_BOTTOM - 1, - "diagnosis changed: find_tables now includes the page-edge remnant", + PAGE_HEIGHT - 1, + "fixture no longer stresses find_tables truncation at the page edge", ) self.assertAlmostEqual(candidate.y1, STRADDLING_ROW_Y0, places=1) + + extended, info = extractor.extend_table_rect_for_edge_content(page, candidate) + self.assertTrue(info["extended"]) + self.assertIn("bottom", info["edges"]) + self.assertTrue(info["clipped_by_page"]) + self.assertGreaterEqual(float(extended.y1), PAGE_HEIGHT - 1) finally: document.close() - @unittest.expectedFailure def test_table_crop_includes_on_page_remnant_of_straddling_row(self): """ - Red contract (#076). - - Stage: pymupdf_find_tables → expanded_rect → save_page_crop (sourceKind=table_crop) + Stage: pymupdf_find_tables → extend_table_rect_for_edge_content → save_page_crop Expected geometry: clip_bbox[3] >= page_height - 1 (on-page remnant retained) Expected structure: table_rows includes the score-5 clinical row (or incompleteness is marked) """ @@ -77,7 +76,7 @@ def test_table_crop_includes_on_page_remnant_of_straddling_row(self): clip_bottom, page_height - 1.0, ( - "stage=pymupdf_find_tables→save_page_crop; " + "stage=pymupdf_find_tables→extend_table_rect_for_edge_content→save_page_crop; " f"expected clip bottom ≥ {page_height - 1.0}, got {clip_bottom}; " f"candidate method={metadata.get('extraction_method')}" ), @@ -88,7 +87,9 @@ def test_table_crop_includes_on_page_remnant_of_straddling_row(self): crop_completeness = metadata.get("crop_completeness") incomplete = ( isinstance(crop_completeness, (int, float)) and float(crop_completeness) < 0.99 - ) or any("crop" in str(warning).lower() and "incomplete" in str(warning).lower() for warning in result.get("warnings") or []) + ) or any( + "table_crop_edge_incomplete" in str(warning) for warning in result.get("warnings") or [] + ) self.assertTrue( has_score_row or incomplete, ( @@ -96,6 +97,8 @@ def test_table_crop_includes_on_page_remnant_of_straddling_row(self): f"rows={rows!r} crop_completeness={crop_completeness!r} warnings={result.get('warnings')!r}" ), ) + self.assertTrue(metadata.get("edge_content_extended")) + self.assertIn("bottom", metadata.get("edge_content_edges") or []) if __name__ == "__main__": From 164fd0d50079d35bc48755052e84c609488f9613 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Fri, 24 Jul 2026 17:44:03 +0000 Subject: [PATCH 3/6] docs: record PR #1176 page-edge crop review in branch ledger Co-authored-by: BigSimmo --- docs/branch-review-ledger.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/branch-review-ledger.md b/docs/branch-review-ledger.md index a694ec191..5131dbd65 100644 --- a/docs/branch-review-ledger.md +++ b/docs/branch-review-ledger.md @@ -760,3 +760,4 @@ This file is append-only. Never rewrite or delete an existing review record; app | 2026-07-25 | cursor/pr-babysit-bugbot-agents-6c52 (PR #1167) | ee44812aae9dad1973d8302eba5bfca5000dffb6 | Open-PR maintenance: review-thread fixes | Before: 8 unresolved Codex/CodeRabbit threads; branch current with main. After: target-head pinning, fresh-main verification, exact `cursor[bot]` identity checks, explicit mutation/provider authorization, direct reply-then-resolve semantics, and no-op ledger bookkeeping are documented. | Prettier check on both agent files pass; `git diff --check` pass; GitHub author probe confirmed `cursor[bot]` account type `Bot`; no provider-backed checks run. | | 2026-07-25 | codex/fix-merge-conflicts-and-ci-on-open-prs (PR #1170) | e979fc892f11a17b1a8f2ef1ab058ef40d629182 | Open-PR maintenance: CI fix + threads + drift | Before: Static PR checks failed because route-group moves made nine valid legacy `src/app/*` documentation references appear missing; 0 unresolved threads; branch already contained current main. After: docs-link resolution checks known App Router route groups and the focused failure is fixed. | `node scripts/check-docs-links.mjs` pass (1162 references); Prettier check pass; `git diff --check` pass; no provider-backed checks run. | | 2026-07-25 | cursor/search-performance-review-4ee9 (PR #1134) | 692834a86e612cc8b311dc6895e007f182f5c5b8 | Open-PR maintenance: superseded docs-link thread and clean main sync | Before: branch was behind current main with one outdated docs-link thread; its product tree already matched main. After: merged current main cleanly and verified the route-group-aware docs-link fix now covers legacy route references. RAG impact: no retrieval behaviour change — history sync and docs tooling verification only. | `node scripts/check-docs-links.mjs` pass (1154 references); clean merge-tree; no live RAG canary or provider-backed check run. | +| 2026-07-24 | PR #1176 / `cursor/pdf-crop-malformed-repro-9b3e` | `5391bf185cd5dffd00a31eb1d282ccfc93277a73` | #076 page-edge table crop geometry fix + fixture regression | APPROVE. No P0-P1. Fix is narrowly scoped to post-find_tables candidate extension from contiguous cell drawings; title/footer inflation avoided by ignoring text during geometry growth; incompleteness warning retained when content continues past the page. Highest residual risk: text-grid tables without cell drawings still will not edge-extend; left/right/top paths are symmetric but fixture-proven only for bottom. Broad PR #1129 retention/padding/storage changes remain out of scope. | Python page-edge + budget 6/6; Vitest pdf-extractor 3 passed / 1 skipped; offline only. | From 7b82b41ae2de510fb5f64d0d3895ddadef6e6528 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Fri, 24 Jul 2026 17:44:30 +0000 Subject: [PATCH 4/6] docs: supersede #1176 review ledger row for merged tip Co-authored-by: BigSimmo --- docs/branch-review-ledger.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/branch-review-ledger.md b/docs/branch-review-ledger.md index 5131dbd65..0fc674d68 100644 --- a/docs/branch-review-ledger.md +++ b/docs/branch-review-ledger.md @@ -761,3 +761,4 @@ This file is append-only. Never rewrite or delete an existing review record; app | 2026-07-25 | codex/fix-merge-conflicts-and-ci-on-open-prs (PR #1170) | e979fc892f11a17b1a8f2ef1ab058ef40d629182 | Open-PR maintenance: CI fix + threads + drift | Before: Static PR checks failed because route-group moves made nine valid legacy `src/app/*` documentation references appear missing; 0 unresolved threads; branch already contained current main. After: docs-link resolution checks known App Router route groups and the focused failure is fixed. | `node scripts/check-docs-links.mjs` pass (1162 references); Prettier check pass; `git diff --check` pass; no provider-backed checks run. | | 2026-07-25 | cursor/search-performance-review-4ee9 (PR #1134) | 692834a86e612cc8b311dc6895e007f182f5c5b8 | Open-PR maintenance: superseded docs-link thread and clean main sync | Before: branch was behind current main with one outdated docs-link thread; its product tree already matched main. After: merged current main cleanly and verified the route-group-aware docs-link fix now covers legacy route references. RAG impact: no retrieval behaviour change — history sync and docs tooling verification only. | `node scripts/check-docs-links.mjs` pass (1154 references); clean merge-tree; no live RAG canary or provider-backed check run. | | 2026-07-24 | PR #1176 / `cursor/pdf-crop-malformed-repro-9b3e` | `5391bf185cd5dffd00a31eb1d282ccfc93277a73` | #076 page-edge table crop geometry fix + fixture regression | APPROVE. No P0-P1. Fix is narrowly scoped to post-find_tables candidate extension from contiguous cell drawings; title/footer inflation avoided by ignoring text during geometry growth; incompleteness warning retained when content continues past the page. Highest residual risk: text-grid tables without cell drawings still will not edge-extend; left/right/top paths are symmetric but fixture-proven only for bottom. Broad PR #1129 retention/padding/storage changes remain out of scope. | Python page-edge + budget 6/6; Vitest pdf-extractor 3 passed / 1 skipped; offline only. | +| 2026-07-24 | PR #1176 / `cursor/pdf-crop-malformed-repro-9b3e` | `28c1e2352a7cabb9c73f52d5084ee69f47895c5e` | #076 page-edge table crop geometry fix (post-merge tip) | APPROVE (supersedes prior #1176 row for 5391bf18 after remote merge). No P0-P1. Same scoped drawing-based edge extension; residual: text-grid tables without drawings, and left/right/top only unit-symmetric. PR #1129 retention/padding still out of scope. | Re-ran Python page-edge + budget 6/6 and Vitest pdf-extractor 3/1 skipped after merge; offline only. | From c31543f5b5862dbe6911079029f28500a6af2a59 Mon Sep 17 00:00:00 2001 From: BigSimmo <87357024+BigSimmo@users.noreply.github.com> Date: Sat, 25 Jul 2026 04:24:45 +0800 Subject: [PATCH 5/6] fix(ci): format PDF crop follow-up --- docs/outstanding-issues.md | 66 +++++++++++++------------- tests/pdf-extractor.test.ts | 93 +++++++++++++++++-------------------- 2 files changed, 75 insertions(+), 84 deletions(-) diff --git a/docs/outstanding-issues.md b/docs/outstanding-issues.md index c1468efb2..40b940058 100644 --- a/docs/outstanding-issues.md +++ b/docs/outstanding-issues.md @@ -150,36 +150,36 @@ removed after current-main verification; it is not missing recommended work. Move resolved rows here with the resolution date and a one-line outcome. Keep them — do not delete. -| ID | Type | Summary | Outcome | Resolved | -| ---- | ----- | ---------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | -| #076 | task | Reproduce malformed fallback PDF image/table crops | Reproduced truncated page-edge `table_crop`s on current-main with `worker/python/fixtures/malformed-table-crop-page-edge.pdf`. Root cause: `pymupdf_find_tables` stops at the last fully detected row; fix extends the candidate from contiguous cell drawings, recovers the on-page score-5 remnant, and emits `table_crop_edge_incomplete` / `crop_completeness=0.9` when content continues past the page. PR #1176. Broad PR #1129 retention/padding changes not merged. | 2026-07-24 | -| #070 | issue | Presentation mobile tabs misroute Overview/Map/Related | Fixed in PR #1135: Overview/Map/Related deep-link to diagnosis `?tab=` sections; Compare stays on the presentation page. Regression in `tests/mobile-interaction-regressions.test.ts`. (Provisional PR-branch IDs `#068`–`#072` were renumbered after `main` claimed `#068`/`#069`.) | 2026-07-24 | -| #071 | issue | Evidence/Clinical Notes Add fakes success without persistence | Fixed in PR #1135: sticky Add controls use the focusable coming-soon placeholder pattern instead of optimistic `setAdded(true)`. | 2026-07-24 | -| #072 | issue | Tools hub exposes false Sort/More affordances | Fixed in PR #1135: Sort is a status label, More filter targets coordination/saved without a fake menu chevron, and the favourites shortcut is labelled Saved/Favourites. | 2026-07-24 | -| #073 | issue | Presentation compare dock CTA is a self-link no-op | Fixed in PR #1135: dock shows non-link "Comparing (N)" status while already comparing. | 2026-07-24 | -| #074 | issue | Mode-action popup hard-reloads internal clinical routes | Fixed in PR #1135: `master-search-header` uses `router.push` for DSM/Specifiers/Formulation actions and mode href fallback. | 2026-07-24 | -| #068 | task | Regenerate full drift-manifest snapshot after schema hygiene | Full Docker `npm run drift:manifest` replay succeeded on a Docker-capable host; `supabase/drift-manifest.json` now carries live `def_hash` values for the plpgsql table-facts body (offline generator_note removed). | 2026-07-24 | -| #052 | issue | Reindex can overlap a fresh agent-enrichment pass | PR #1143 retained the friendly full/retry preflight and closed its check-then-enqueue race with an owner-scoped transactional RPC. Reindex enqueue and the agent claim path serialize on the document row; disposable PostgreSQL proved both interleavings, and exact-head migration replay/unit/build/Chromium/policy/security checks passed. | 2026-07-24 | -| #062 | issue | Upload crash can strand a queued document without a job | Aged owner-scoped `queued`-without-open-job rows are detected by `reindex:health`; the six-hour autopilot raises a durable alert, and guarded recovery uses PR #1143's transactional RPC so enqueue is owner-scoped, idempotent and atomic. `recover:ingestion --include-stranded-queued` remains dry-run/confirmation-first; scheduled production mutation is not enabled. | 2026-07-24 | -| #060 | issue | Safety Plan Generator contradicted the privacy contract | PR #1119 removed patient identifier entry, leaves the post-export name line blank, and aligned tool, privacy and PIA copy. DOM/privacy tests and Chromium copy/print/network coverage prove working content remains in React memory with no fetch/XHR; hosted Production UI, build, unit, policy, safety, static-analysis and secret checks passed. Support-contact details remain classified as sensitive local-only working content. | 2026-07-24 | -| #061 | issue | Missing answer relevance metadata was treated as source-backed | PR #1125 now requires explicit source-backed relevance for trusted/grounded presentation and prevents visual tables, clinical-note sections and quotes, and comparison metadata from bypassing the render model. Three actionable P2 review paths were fixed; focused policy/DOM tests, offline RAG, production-readiness, build, unit, static, security, and Production UI gates passed. No retrieval, ranking, generation, provider, or data behavior changed. | 2026-07-24 | -| #034 | issue | Answer cache can serve stale governance metadata | Current-source verification found direct route coverage already asserts RAG-cache invalidation on document PATCH, source review, label, bulk, and reindex mutation paths. The residual test recommendation is already met; changing the protected cache key is unnecessary. | 2026-07-24 | -| #014 | rec | Realize the `next/image` win on signed previews | Superseded: `SignedImage` uses `next/image` for layout and sizing but deliberately sets `unoptimized`, preventing bearer signed URLs from entering the unauthenticated optimizer cache where cached content could outlive the token. No optimization task remains unless private-image delivery changes. | 2026-07-24 | -| #026 | task | Wire the Supabase document-change trigger | PR #1100 merged after disposable PostgreSQL replay and hosted migration replay. Production migration history and read-only catalog proof confirm the enabled metadata trigger, security-definer function, pinned search path and denied anonymous/authenticated execution; `npm run check:drift` reports no unexpected live drift. Delivery remains intentionally inert until the operator inputs tracked in #025 are configured. | 2026-07-24 | -| #031 | issue | Populate canary Source Governance table | The answer-quality step now consumes the preceding `golden-retrieval.json` only for source-governance reporting. Offline replay of run `30018289898` populated 338 top results, including 202 review-required entries, while retaining zero retrieval cases and no additional threshold failures. Retrieval and ranking behavior are unchanged. | 2026-07-24 | -| #020 | task | Validate eval:quality cost readout post-fix | Confirmed on merged-main canary run `30018289898`: Answer Metrics reported 9 nonzero-cost cases and an estimated answer cost of `$0.234736`; the structured report retained the same value. The PR #1050 estimator fix is operationally proven. | 2026-07-23 | -| #003 | task | Staging tenancy release evidence outstanding | Ran GitHub Action and validated isolation | 2026-07-21 | -| #002 | task | Process-ownership fix not yet isolated on `main` | Fixed process isolation using child.pid termination | 2026-07-21 | -| #008 | rec | Dead href builders in `document-flow-routes.ts` | Not dead code (false positive): `documentReaderHref`/`documentEvidenceHref` are live via the mock wrappers in `src/components/document-search-mockups.tsx` + `src/components/master-document-flow-mockups.tsx` (rendered under `src/app/mockups/document-search/`) and covered by `tests/document-flow-routes.test.ts`; removing breaks the build. Only the production non-mock hrefs are unlinked from prod UI — a wiring gap, not dead code. | 2026-07-22 | -| #015 | task | Content-first fallback regression tests | Added `tests/registry-record-loader.dom.test.tsx` (8) + `tests/medication-record-page.dom.test.tsx` (6) covering content-first fallback paint, live swap-in, spinner/skeleton, error + not-found/unauthorized states, and the invariant that no authoritative verification badge shows before live governance reconciles (registry fixture-flag neutralization + medication governance-drop-on-error). | 2026-07-22 | -| #004 | rec | Rescope provider-gated RAG safety ideas | Closed obsolete — rescue source (754-line RAG-safety worktree) unrecoverable/pruned across all refs; answer-quality thresholds + deep-health already shipped on `main` (#585/#587); only cost-cap preflight was genuinely missing and, per session decision, dropped rather than re-filed. | 2026-07-22 | -| #006 | issue | Globe "Language & region" button had no handler | Resolved on main with the repository's disabled "Coming soon" placeholder convention and button-wiring coverage. Future language/region work remains a feature request, not an inert-control defect. | 2026-07-22 | -| #042 | issue | Invalid optional credentials fell into anonymous access | PRs #1078/#1079 introduced `absent \| valid \| invalid`, return 401 for presented invalid credentials, preserve authoritative header precedence and prefer the current-project session cookie. The archived anonymous-upload metadata patch was rejected as stale because uploads are already administrator-only before duplicate lookup. | 2026-07-22 | -| #043 | issue | Readiness could report healthy or throw on Supabase errors | PR #1080 now fails readiness closed for returned and thrown dependency failures, preserves recognized actionable messages, and prevents raw dependency-error disclosure. | 2026-07-22 | -| #044 | issue | Publication approval was not bound to immutable reviewed state | PR #1081 added a canonical reviewed-state digest, row locks, active-job rejection and a new forward migration with replay/schema/type/drift evidence. | 2026-07-22 | -| #045 | issue | Bulk reindex discarded partial-success results | PR #1084 reserves preflight conflicts for non-2xx responses; completed mixed batches return per-item success/failure/missing results, and the UI refreshes successful work. | 2026-07-22 | -| #046 | issue | DOCX extraction lacked explicit resource budgets | PR #1085 added pre-inflate declared-size checks and post-read fail-safes for artifact count, per-artifact bytes, aggregate media, Word XML and extracted UTF-8 text. | 2026-07-22 | -| #047 | issue | XLSX extraction could construct unbounded results | PR #1086 bounds worksheets, non-empty rows, rendered cells and UTF-8 output while preserving sparse-column rendering. | 2026-07-22 | -| #048 | issue | Account copy overstated sync/privacy and enabled unavailable SSO | PR #1087 now maps copy to actual favourites/preferences persistence, identifies browser-session recents, removes the contradictory "never shared" claim and clearly disables unavailable providers using the accessible placeholder contract. | 2026-07-22 | -| #049 | issue | Process diagnostic exposed a Cursor worker API key | The exact worker was stopped, the key was revoked server-side, both local encrypted worker-secret records were removed, and authorized repository/backup scans found no plaintext copy. Follow-up guardrails now prevent repository process inventory from serializing command lines and redact heavyweight-lock command text before persistence or errors. | 2026-07-23 | -| #050 | issue | Next.js 16.2.10 remained in a high-severity security range | Upgraded `next` and `@next/env` to 16.2.11, regenerated the npm lockfile, confirmed the production dependency audit is clean, and passed focused framework checks, `verify:cheap`, and the full Chromium UI gate. | 2026-07-23 | +| ID | Type | Summary | Outcome | Resolved | +| ---- | ----- | ---------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | +| #076 | task | Reproduce malformed fallback PDF image/table crops | Reproduced truncated page-edge `table_crop`s on current-main with `worker/python/fixtures/malformed-table-crop-page-edge.pdf`. Root cause: `pymupdf_find_tables` stops at the last fully detected row; fix extends the candidate from contiguous cell drawings, recovers the on-page score-5 remnant, and emits `table_crop_edge_incomplete` / `crop_completeness=0.9` when content continues past the page. PR #1176. Broad PR #1129 retention/padding changes not merged. | 2026-07-24 | +| #070 | issue | Presentation mobile tabs misroute Overview/Map/Related | Fixed in PR #1135: Overview/Map/Related deep-link to diagnosis `?tab=` sections; Compare stays on the presentation page. Regression in `tests/mobile-interaction-regressions.test.ts`. (Provisional PR-branch IDs `#068`–`#072` were renumbered after `main` claimed `#068`/`#069`.) | 2026-07-24 | +| #071 | issue | Evidence/Clinical Notes Add fakes success without persistence | Fixed in PR #1135: sticky Add controls use the focusable coming-soon placeholder pattern instead of optimistic `setAdded(true)`. | 2026-07-24 | +| #072 | issue | Tools hub exposes false Sort/More affordances | Fixed in PR #1135: Sort is a status label, More filter targets coordination/saved without a fake menu chevron, and the favourites shortcut is labelled Saved/Favourites. | 2026-07-24 | +| #073 | issue | Presentation compare dock CTA is a self-link no-op | Fixed in PR #1135: dock shows non-link "Comparing (N)" status while already comparing. | 2026-07-24 | +| #074 | issue | Mode-action popup hard-reloads internal clinical routes | Fixed in PR #1135: `master-search-header` uses `router.push` for DSM/Specifiers/Formulation actions and mode href fallback. | 2026-07-24 | +| #068 | task | Regenerate full drift-manifest snapshot after schema hygiene | Full Docker `npm run drift:manifest` replay succeeded on a Docker-capable host; `supabase/drift-manifest.json` now carries live `def_hash` values for the plpgsql table-facts body (offline generator_note removed). | 2026-07-24 | +| #052 | issue | Reindex can overlap a fresh agent-enrichment pass | PR #1143 retained the friendly full/retry preflight and closed its check-then-enqueue race with an owner-scoped transactional RPC. Reindex enqueue and the agent claim path serialize on the document row; disposable PostgreSQL proved both interleavings, and exact-head migration replay/unit/build/Chromium/policy/security checks passed. | 2026-07-24 | +| #062 | issue | Upload crash can strand a queued document without a job | Aged owner-scoped `queued`-without-open-job rows are detected by `reindex:health`; the six-hour autopilot raises a durable alert, and guarded recovery uses PR #1143's transactional RPC so enqueue is owner-scoped, idempotent and atomic. `recover:ingestion --include-stranded-queued` remains dry-run/confirmation-first; scheduled production mutation is not enabled. | 2026-07-24 | +| #060 | issue | Safety Plan Generator contradicted the privacy contract | PR #1119 removed patient identifier entry, leaves the post-export name line blank, and aligned tool, privacy and PIA copy. DOM/privacy tests and Chromium copy/print/network coverage prove working content remains in React memory with no fetch/XHR; hosted Production UI, build, unit, policy, safety, static-analysis and secret checks passed. Support-contact details remain classified as sensitive local-only working content. | 2026-07-24 | +| #061 | issue | Missing answer relevance metadata was treated as source-backed | PR #1125 now requires explicit source-backed relevance for trusted/grounded presentation and prevents visual tables, clinical-note sections and quotes, and comparison metadata from bypassing the render model. Three actionable P2 review paths were fixed; focused policy/DOM tests, offline RAG, production-readiness, build, unit, static, security, and Production UI gates passed. No retrieval, ranking, generation, provider, or data behavior changed. | 2026-07-24 | +| #034 | issue | Answer cache can serve stale governance metadata | Current-source verification found direct route coverage already asserts RAG-cache invalidation on document PATCH, source review, label, bulk, and reindex mutation paths. The residual test recommendation is already met; changing the protected cache key is unnecessary. | 2026-07-24 | +| #014 | rec | Realize the `next/image` win on signed previews | Superseded: `SignedImage` uses `next/image` for layout and sizing but deliberately sets `unoptimized`, preventing bearer signed URLs from entering the unauthenticated optimizer cache where cached content could outlive the token. No optimization task remains unless private-image delivery changes. | 2026-07-24 | +| #026 | task | Wire the Supabase document-change trigger | PR #1100 merged after disposable PostgreSQL replay and hosted migration replay. Production migration history and read-only catalog proof confirm the enabled metadata trigger, security-definer function, pinned search path and denied anonymous/authenticated execution; `npm run check:drift` reports no unexpected live drift. Delivery remains intentionally inert until the operator inputs tracked in #025 are configured. | 2026-07-24 | +| #031 | issue | Populate canary Source Governance table | The answer-quality step now consumes the preceding `golden-retrieval.json` only for source-governance reporting. Offline replay of run `30018289898` populated 338 top results, including 202 review-required entries, while retaining zero retrieval cases and no additional threshold failures. Retrieval and ranking behavior are unchanged. | 2026-07-24 | +| #020 | task | Validate eval:quality cost readout post-fix | Confirmed on merged-main canary run `30018289898`: Answer Metrics reported 9 nonzero-cost cases and an estimated answer cost of `$0.234736`; the structured report retained the same value. The PR #1050 estimator fix is operationally proven. | 2026-07-23 | +| #003 | task | Staging tenancy release evidence outstanding | Ran GitHub Action and validated isolation | 2026-07-21 | +| #002 | task | Process-ownership fix not yet isolated on `main` | Fixed process isolation using child.pid termination | 2026-07-21 | +| #008 | rec | Dead href builders in `document-flow-routes.ts` | Not dead code (false positive): `documentReaderHref`/`documentEvidenceHref` are live via the mock wrappers in `src/components/document-search-mockups.tsx` + `src/components/master-document-flow-mockups.tsx` (rendered under `src/app/mockups/document-search/`) and covered by `tests/document-flow-routes.test.ts`; removing breaks the build. Only the production non-mock hrefs are unlinked from prod UI — a wiring gap, not dead code. | 2026-07-22 | +| #015 | task | Content-first fallback regression tests | Added `tests/registry-record-loader.dom.test.tsx` (8) + `tests/medication-record-page.dom.test.tsx` (6) covering content-first fallback paint, live swap-in, spinner/skeleton, error + not-found/unauthorized states, and the invariant that no authoritative verification badge shows before live governance reconciles (registry fixture-flag neutralization + medication governance-drop-on-error). | 2026-07-22 | +| #004 | rec | Rescope provider-gated RAG safety ideas | Closed obsolete — rescue source (754-line RAG-safety worktree) unrecoverable/pruned across all refs; answer-quality thresholds + deep-health already shipped on `main` (#585/#587); only cost-cap preflight was genuinely missing and, per session decision, dropped rather than re-filed. | 2026-07-22 | +| #006 | issue | Globe "Language & region" button had no handler | Resolved on main with the repository's disabled "Coming soon" placeholder convention and button-wiring coverage. Future language/region work remains a feature request, not an inert-control defect. | 2026-07-22 | +| #042 | issue | Invalid optional credentials fell into anonymous access | PRs #1078/#1079 introduced `absent \| valid \| invalid`, return 401 for presented invalid credentials, preserve authoritative header precedence and prefer the current-project session cookie. The archived anonymous-upload metadata patch was rejected as stale because uploads are already administrator-only before duplicate lookup. | 2026-07-22 | +| #043 | issue | Readiness could report healthy or throw on Supabase errors | PR #1080 now fails readiness closed for returned and thrown dependency failures, preserves recognized actionable messages, and prevents raw dependency-error disclosure. | 2026-07-22 | +| #044 | issue | Publication approval was not bound to immutable reviewed state | PR #1081 added a canonical reviewed-state digest, row locks, active-job rejection and a new forward migration with replay/schema/type/drift evidence. | 2026-07-22 | +| #045 | issue | Bulk reindex discarded partial-success results | PR #1084 reserves preflight conflicts for non-2xx responses; completed mixed batches return per-item success/failure/missing results, and the UI refreshes successful work. | 2026-07-22 | +| #046 | issue | DOCX extraction lacked explicit resource budgets | PR #1085 added pre-inflate declared-size checks and post-read fail-safes for artifact count, per-artifact bytes, aggregate media, Word XML and extracted UTF-8 text. | 2026-07-22 | +| #047 | issue | XLSX extraction could construct unbounded results | PR #1086 bounds worksheets, non-empty rows, rendered cells and UTF-8 output while preserving sparse-column rendering. | 2026-07-22 | +| #048 | issue | Account copy overstated sync/privacy and enabled unavailable SSO | PR #1087 now maps copy to actual favourites/preferences persistence, identifies browser-session recents, removes the contradictory "never shared" claim and clearly disables unavailable providers using the accessible placeholder contract. | 2026-07-22 | +| #049 | issue | Process diagnostic exposed a Cursor worker API key | The exact worker was stopped, the key was revoked server-side, both local encrypted worker-secret records were removed, and authorized repository/backup scans found no plaintext copy. Follow-up guardrails now prevent repository process inventory from serializing command lines and redact heavyweight-lock command text before persistence or errors. | 2026-07-23 | +| #050 | issue | Next.js 16.2.10 remained in a high-severity security range | Upgraded `next` and `@next/env` to 16.2.11, regenerated the npm lockfile, confirmed the production dependency audit is clean, and passed focused framework checks, `verify:cheap`, and the full Chromium UI gate. | 2026-07-23 | diff --git a/tests/pdf-extractor.test.ts b/tests/pdf-extractor.test.ts index 4706efb74..4b39d7d99 100644 --- a/tests/pdf-extractor.test.ts +++ b/tests/pdf-extractor.test.ts @@ -157,55 +157,46 @@ describe.runIf(hasPyMuPDF)("Python PDF table extraction", () => { // #076: page-edge table crops must keep the on-page remnant of a straddling // final row after find_tables truncates the candidate bbox. - it( - "includes on-page remnant geometry for a table row that straddles the page bottom (#076)", - async () => { - const root = await mkdtemp(path.join(tmpdir(), "clinical-kb-page-edge-crop-")); - const imageDir = path.join(root, "images"); - const jsonPath = path.join(root, "extract.json"); - await mkdir(imageDir, { recursive: true }); - - const fixturePath = path.join( - process.cwd(), - "worker", - "python", - "fixtures", - "malformed-table-crop-page-edge.pdf", - ); - const result = spawnSync( - pythonBin, - [path.join(process.cwd(), "worker", "python", "extract_pdf_assets.py"), fixturePath, imageDir, jsonPath], - { cwd: process.cwd(), encoding: "utf8" }, - ); - - expect(result.status).toBe(0); - const payload = JSON.parse(await readFile(jsonPath, "utf8")) as { - images: Array<{ - sourceKind?: string; - bbox?: number[]; - metadata?: Record; - }>; - warnings?: string[]; - }; - const tableCrop = payload.images.find((image) => image.sourceKind === "table_crop"); - expect(tableCrop).toBeTruthy(); - - const pageHeight = Number(tableCrop?.metadata?.page_height); - const clip = (tableCrop?.bbox ?? tableCrop?.metadata?.clip_bbox) as number[] | undefined; - expect(clip).toEqual(expect.any(Array)); - // Stage: pymupdf_find_tables → extend_table_rect_for_edge_content → save_page_crop. - // Expected geometry: clip bottom reaches the page edge so the on-page remnant is retained. - expect(Number(clip?.[3])).toBeGreaterThanOrEqual(pageHeight - 1); - - const rows = (tableCrop?.metadata?.table_rows as string[][] | undefined) ?? []; - const hasScoreRow = rows.some((row) => String(row?.[0] ?? "").trim() === "5"); - const cropCompleteness = Number(tableCrop?.metadata?.crop_completeness); - const incomplete = - Number.isFinite(cropCompleteness) && cropCompleteness < 0.99 - ? true - : (payload.warnings ?? []).some((warning) => warning.includes("table_crop_edge_incomplete")); - expect(hasScoreRow || incomplete).toBe(true); - expect(tableCrop?.metadata?.edge_content_extended).toBe(true); - }, - ); + it("includes on-page remnant geometry for a table row that straddles the page bottom (#076)", async () => { + const root = await mkdtemp(path.join(tmpdir(), "clinical-kb-page-edge-crop-")); + const imageDir = path.join(root, "images"); + const jsonPath = path.join(root, "extract.json"); + await mkdir(imageDir, { recursive: true }); + + const fixturePath = path.join(process.cwd(), "worker", "python", "fixtures", "malformed-table-crop-page-edge.pdf"); + const result = spawnSync( + pythonBin, + [path.join(process.cwd(), "worker", "python", "extract_pdf_assets.py"), fixturePath, imageDir, jsonPath], + { cwd: process.cwd(), encoding: "utf8" }, + ); + + expect(result.status).toBe(0); + const payload = JSON.parse(await readFile(jsonPath, "utf8")) as { + images: Array<{ + sourceKind?: string; + bbox?: number[]; + metadata?: Record; + }>; + warnings?: string[]; + }; + const tableCrop = payload.images.find((image) => image.sourceKind === "table_crop"); + expect(tableCrop).toBeTruthy(); + + const pageHeight = Number(tableCrop?.metadata?.page_height); + const clip = (tableCrop?.bbox ?? tableCrop?.metadata?.clip_bbox) as number[] | undefined; + expect(clip).toEqual(expect.any(Array)); + // Stage: pymupdf_find_tables → extend_table_rect_for_edge_content → save_page_crop. + // Expected geometry: clip bottom reaches the page edge so the on-page remnant is retained. + expect(Number(clip?.[3])).toBeGreaterThanOrEqual(pageHeight - 1); + + const rows = (tableCrop?.metadata?.table_rows as string[][] | undefined) ?? []; + const hasScoreRow = rows.some((row) => String(row?.[0] ?? "").trim() === "5"); + const cropCompleteness = Number(tableCrop?.metadata?.crop_completeness); + const incomplete = + Number.isFinite(cropCompleteness) && cropCompleteness < 0.99 + ? true + : (payload.warnings ?? []).some((warning) => warning.includes("table_crop_edge_incomplete")); + expect(hasScoreRow || incomplete).toBe(true); + expect(tableCrop?.metadata?.edge_content_extended).toBe(true); + }); }); From e005d7f2c9d2ef05af6473e237a859a1c89b2bae Mon Sep 17 00:00:00 2001 From: BigSimmo <87357024+BigSimmo@users.noreply.github.com> Date: Sat, 25 Jul 2026 04:25:03 +0800 Subject: [PATCH 6/6] docs: record PR 1176 maintenance --- docs/branch-review-ledger.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/branch-review-ledger.md b/docs/branch-review-ledger.md index a11d79b78..53fb766b0 100644 --- a/docs/branch-review-ledger.md +++ b/docs/branch-review-ledger.md @@ -793,3 +793,4 @@ This file is append-only. Never rewrite or delete an existing review record; app | 2026-07-24 | open-PR conflict sync (22 PRs) | multi-head | Conflict resolution pass | Before: all 22 open PRs behind/dirty vs main (several CONFLICTING/DIRTY). After: merged origin/main into every open head; all pushes OK; merge-tree classified 22/22 clean. | merge origin/main per branch; check:branch-review-ledger on #1172; no provider-backed checks run | | 2026-07-24 | cursor/pr-queue-hygiene-72ec | pending-push | PR queue hygiene | Add pr-branch-sync workflow + sync:pr-branches helper; bump postcss to clear npm audit high; document anti-churn guidance in AGENTS/process-hardening/pr-babysit/run-pr. | check:github-actions PASS; docs:check-scripts/index PASS; vitest sync-open-pr-branches 3/3; npm audit high clean; no provider-backed checks run | | 2026-07-24 | codex/apply-phone-layout-to-all-home-pages (PR #1124) | pending | Babysit: ledger dedupe + merge readiness | Before: Static PR failed on exact duplicate ledger rows after main sync. After: removed duplicate rows; squash auto-merge armed. | check:branch-review-ledger PASS; no provider-backed checks run | +| 2026-07-25 | cursor/pdf-crop-malformed-repro-9b3e (PR #1176) | c31543f5b5862dbe6911079029f28500a6af2a59 | Run PR sweep: CI fix + threads + drift | Before: GitHub reported DIRTY and Static PR checks failed formatting `docs/outstanding-issues.md` and `tests/pdf-extractor.test.ts`; 0 unresolved threads. After: merged current `origin/main` cleanly and formatted both failing files. | Prettier check pass; `git diff --check` pass; no provider-backed checks run. |