Split toPdf() for worker-parallel page rendering - #28
Merged
Conversation
Owner
Author
|
Added `tests/pdf-parallel.bench.ts`, which spawns a real 4-worker `worker_threads` pool and compares wall-clock time directly against `toPdf()`'s single-thread path — the earlier `pdf.bench.ts` only measured the render/assemble time split, not actual parallel execution. Result on `1to10.note` (10 pages): 1.80x faster (2242ms mean serial vs. 1247ms mean parallel with a 4-worker pool), confirming the split is a real win, not just theoretically parallelizable. |
philips
marked this pull request as ready for review
July 25, 2026 17:40
…ndering Implements plans/rtr-searchable-pdf-workers.md: toPdf() becomes a thin wrapper around createPdfContext (font setup) + addPdfPage (per-page image embed + invisible RTR text layer), so applications can render pages in parallel across Workers (via extractPageRenderData + toImage + encodePng, all worker-safe) while assembly stays on the main thread (pdf-lib objects aren't structured-clone-safe). addPdfPage accepts either an image-js Image or pre-encoded PNG bytes so a Worker's output can be handed straight in. Profiling (tests/pdf.bench.ts, findings recorded in the plan doc) found encodePng, not RLE decode, dominates the worker-eligible side, and that pdfDoc.save() alone is ~34% of total time and inherently unparallelizable (one whole-document serialization) — so this implements render-only parallelism (Option A) and skips per-page PDF merging. Also fixes relative imports in src/ to include explicit .js extensions, which tsc's "bundler" resolution allows but was previously omitting, leaving the built lib/ output unloadable by Node's own ESM resolver (needed for tests/pdf-worker-roundtrip.test.ts's worker_threads fixture, which loads lib/ directly since worker_threads can't run .ts).
pdf.bench.ts only measured whether render vs. assemble time split favored parallelizing render (Option A) — it never actually spawned Workers. tests/pdf-parallel.bench.ts closes that gap: a persistent worker_threads pool (render-worker-pool.mjs) renders pages concurrently via extractPageRenderData + toImage + encodePng, then addPdfPage assembles on the main thread, compared directly against toPdf()'s single-thread path. On 1to10.note with a 4-worker pool: 1.80x faster wall-clock (2242ms mean serial vs. 1247ms mean parallel) — confirms the split actually pays off, not just that it's theoretically parallelizable.
philips
force-pushed
the
worktree-rtr-searchable-pdf-plan
branch
from
July 25, 2026 17:41
49f88dd to
ceeec9b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
plans/rtr-searchable-pdf-workers.md: splitstoPdf()intocreatePdfContext(font/document setup) +addPdfPage(per-page image embed + invisible RTR text layer), so applications can render pages in parallel across Web Workers /worker_threadswhile PDF assembly (which needs non-clonablepdf-libobjects) stays on the main thread.toPdf()itself is now a thin wrapper and its behavior/signature is unchanged.extractPageRenderData(note, pageNumber), exported alongside a widenedtoImageparameter type (IRenderableNote), so a single page's render-only data can be posted to a Worker without cloning the whole note or reconstructing a fakeSupernoteX.addPdfPageaccepts either animage-jsImageor pre-encoded PNG bytes, so a Worker that already rantoImage+encodePngcan hand back just bytes.tests/pdf.bench.ts; one-shot breakdown recorded in the plan doc) foundencodePng, not RLE decode, dominates the worker-eligible side of the work, and thatpdfDoc.save()alone is ~34% of total time and inherently unparallelizable (one whole-document serialization, regardless of how pages were rendered) — so this implements render-only parallelism (Option A) and skips per-page PDF merging (Option B), per the plan's non-goals.src/to include explicit.jsextensions.tsc's"bundler"module resolution allowed omitting them, but that left the builtlib/output unloadable by Node's own ESM resolver — needed here because the newworker_threadsround-trip test's worker fixture (tests/fixtures/render-worker.mjs) loadslib/directly (workers can't run.ts).pretestscript (npm run build) sonpm testreliably haslib/built locally; CI already builds before testing.Test plan
npm run buildnpm run lintnpm test(21 tests acrossmain.test.ts,pdf.test.ts,pdf-worker-roundtrip.test.ts, all passing, including the 4 pre-existingpdf.test.tscases unmodified)toPdf()output is text-equivalent to manualcreatePdfContext+addPdfPagecomposition (backward-compat check)addPdfPagegiven anImagevs. pre-encoded PNG bytes produces text-equivalent outputworker_threadsround trip —extractPageRenderData→structuredClonesanity check → Worker renders viatoImage+encodePng→ main threaddecodePngs the result and byte-compares against a main-threadtoImagerender (byte-identical)npx vitest bench tests/pdf.bench.tsfor the render-vs-assemble timing breakdown