Skip to content

Split toPdf() for worker-parallel page rendering - #28

Merged
philips merged 2 commits into
mainfrom
worktree-rtr-searchable-pdf-plan
Jul 25, 2026
Merged

Split toPdf() for worker-parallel page rendering#28
philips merged 2 commits into
mainfrom
worktree-rtr-searchable-pdf-plan

Conversation

@philips

@philips philips commented Jul 25, 2026

Copy link
Copy Markdown
Owner

Summary

  • Implements plans/rtr-searchable-pdf-workers.md: splits toPdf() into createPdfContext (font/document setup) + addPdfPage (per-page image embed + invisible RTR text layer), so applications can render pages in parallel across Web Workers / worker_threads while PDF assembly (which needs non-clonable pdf-lib objects) stays on the main thread. toPdf() itself is now a thin wrapper and its behavior/signature is unchanged.
  • Adds extractPageRenderData(note, pageNumber), exported alongside a widened toImage parameter type (IRenderableNote), so a single page's render-only data can be posted to a Worker without cloning the whole note or reconstructing a fake SupernoteX.
  • addPdfPage accepts either an image-js Image or pre-encoded PNG bytes, so a Worker that already ran toImage + encodePng can hand back just bytes.
  • Profiling (tests/pdf.bench.ts; one-shot breakdown recorded in the plan doc) found encodePng, not RLE decode, dominates the worker-eligible side of the work, and that pdfDoc.save() alone is ~34% of total time and inherently unparallelizable (one whole-document serialization, regardless of how pages were rendered) — so this implements render-only parallelism (Option A) and skips per-page PDF merging (Option B), per the plan's non-goals.
  • Fixes relative imports in src/ to include explicit .js extensions. tsc's "bundler" module resolution allowed omitting them, but that left the built lib/ output unloadable by Node's own ESM resolver — needed here because the new worker_threads round-trip test's worker fixture (tests/fixtures/render-worker.mjs) loads lib/ directly (workers can't run .ts).
  • Adds a pretest script (npm run build) so npm test reliably has lib/ built locally; CI already builds before testing.
  • Documents the application-side Worker orchestration pattern in the README.

Test plan

  • npm run build
  • npm run lint
  • npm test (21 tests across main.test.ts, pdf.test.ts, pdf-worker-roundtrip.test.ts, all passing, including the 4 pre-existing pdf.test.ts cases unmodified)
  • New: toPdf() output is text-equivalent to manual createPdfContext+addPdfPage composition (backward-compat check)
  • New: addPdfPage given an Image vs. pre-encoded PNG bytes produces text-equivalent output
  • New: real worker_threads round trip — extractPageRenderDatastructuredClone sanity check → Worker renders via toImage+encodePng → main thread decodePngs the result and byte-compares against a main-thread toImage render (byte-identical)
  • npx vitest bench tests/pdf.bench.ts for the render-vs-assemble timing breakdown

@philips

philips commented Jul 25, 2026

Copy link
Copy Markdown
Owner Author

Added `tests/pdf-parallel.bench.ts`, which spawns a real 4-worker `worker_threads` pool and compares wall-clock time directly against `toPdf()`'s single-thread path — the earlier `pdf.bench.ts` only measured the render/assemble time split, not actual parallel execution.

Result on `1to10.note` (10 pages): 1.80x faster (2242ms mean serial vs. 1247ms mean parallel with a 4-worker pool), confirming the split is a real win, not just theoretically parallelizable.

@philips
philips marked this pull request as ready for review July 25, 2026 17:40
philips added 2 commits July 25, 2026 10:41
…ndering

Implements plans/rtr-searchable-pdf-workers.md: toPdf() becomes a thin
wrapper around createPdfContext (font setup) + addPdfPage (per-page image
embed + invisible RTR text layer), so applications can render pages in
parallel across Workers (via extractPageRenderData + toImage + encodePng,
all worker-safe) while assembly stays on the main thread (pdf-lib objects
aren't structured-clone-safe). addPdfPage accepts either an image-js Image
or pre-encoded PNG bytes so a Worker's output can be handed straight in.

Profiling (tests/pdf.bench.ts, findings recorded in the plan doc) found
encodePng, not RLE decode, dominates the worker-eligible side, and that
pdfDoc.save() alone is ~34% of total time and inherently unparallelizable
(one whole-document serialization) — so this implements render-only
parallelism (Option A) and skips per-page PDF merging.

Also fixes relative imports in src/ to include explicit .js extensions,
which tsc's "bundler" resolution allows but was previously omitting,
leaving the built lib/ output unloadable by Node's own ESM resolver
(needed for tests/pdf-worker-roundtrip.test.ts's worker_threads fixture,
which loads lib/ directly since worker_threads can't run .ts).
pdf.bench.ts only measured whether render vs. assemble time split favored
parallelizing render (Option A) — it never actually spawned Workers.
tests/pdf-parallel.bench.ts closes that gap: a persistent worker_threads
pool (render-worker-pool.mjs) renders pages concurrently via
extractPageRenderData + toImage + encodePng, then addPdfPage assembles on
the main thread, compared directly against toPdf()'s single-thread path.

On 1to10.note with a 4-worker pool: 1.80x faster wall-clock (2242ms mean
serial vs. 1247ms mean parallel) — confirms the split actually pays off,
not just that it's theoretically parallelizable.
@philips
philips force-pushed the worktree-rtr-searchable-pdf-plan branch from 49f88dd to ceeec9b Compare July 25, 2026 17:41
@philips
philips merged commit cb5b115 into main Jul 25, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant