Skip to content

Fix #154: memory blowup and freezes when viewing/exporting long notes - #170

Merged
philips merged 2 commits into
mainfrom
fix-note-memory-and-pdf-export-issue-154
Aug 1, 2026
Merged

Fix #154: memory blowup and freezes when viewing/exporting long notes#170
philips merged 2 commits into
mainfrom
fix-note-memory-and-pdf-export-issue-154

Conversation

@philips-clanker

Copy link
Copy Markdown
Collaborator

Summary

Squashed version of #166 (kept open for reference/history) with the full investigation trail summarized below. Fixes issue #154.

Depends on philips/supernote-typescript#42 (submodule) — needs to merge first, or at least stay pushed, for this branch's submodule pointer to resolve cleanly long-term.

The final fix

  • Lazy-loads main-view page images and thumbnails via IntersectionObserver, evicting whichever have scrolled out of view — memory is now bounded by "how many pages are near the viewport right now," not by how many have been scrolled through in total.
  • The Worker pool sends only the page(s) actually requested to a Worker, never the whole parsed note — previously every worker ended up holding a full copy of the entire document, the single largest contributor to the reported memory growth.
  • Fixed a race condition where a page's image could finalize and get stuck "loaded" forever if the page scrolled back out of view mid-decode.
  • Reduced the Worker pool from navigator.hardwareConcurrency (8-9 workers) to a fixed 2 — the peak during a fast scroll scales with how many workers can be simultaneously mid-growth at once, so this was the actual lever for the peak.
  • Tears down the whole Worker pool after 2s of inactivity, reclaiming its resting memory (~200MB); recreated lazily, and never mid-flight.
  • Debounced both the main-view and thumbnail-sidebar load triggers, so a long-distance jump (e.g. a search match 100 pages away) doesn't rasterize every page it flies past on the way there.
  • Thumbnails are no longer evicted at all (cheap enough now that they're downscaled) and get a generous prefetch margin — also fixes a "broken image" icon bug the old evict-on-scroll-away behavior caused.
  • PDF export: caps how many pages get embedded per pdf-lib batch, correctly alpha-composites each page toward white before embedding, and runs the entire rasterize-embed-save sequence inside a dedicated Worker, since pdf-lib's own embedPng()/save() are genuinely expensive (confirmed ~1.7GB / 10+s for a 100-page note) no matter how the input is prepared.

Things attempted while locating the final fix

  • Capping devicePixelRatio and using a zoom-aware rasterization scale to shrink canvas memory — disproven by direct testing (memory unaffected).
  • Destroying the previous pdf.js document on file switch — a real but minor fix, not the primary driver.
  • Chrome heap-snapshot/heap-timeline analysis (custom parsing scripts) to rule out a JS-heap leak, since canvas/ImageBitmap memory is invisible to the JS heap profiler.
  • Temporary DevTools-console toggles to bisect the PDF text layer vs. the image layer, isolating the image/Worker rasterization pipeline as the actual cause.
  • Periodically recycling (terminating/recreating) Worker instances after N calls, to bound per-worker internal growth — confirmed working, but later measured to make no real difference once the pool-size reduction and idle teardown were in place, so removed rather than kept as unjustified complexity.
  • Dropping the PNG alpha channel outright before PDF embedding instead of properly alpha-compositing it — shrank memory/time but produced a black-background PDF (background pixels are packed as black with alpha 0, not white), fixed by compositing toward white instead.

Test plan

  • tsc --noEmit, eslint, vitest run (95 tests) all pass
  • supernote-typescript's own test suite (48 tests, including new coverage for the alpha-compositing fix) passes
  • Confirmed on real devices across the investigation: peak memory during a full scroll through a 100+ page note dropped from 1.5GB+ to well under 400MB; PDF export no longer freezes the UI or produces a corrupted (black-background) file

… notes

Squashed history of the full investigation - see below for the trail of
things tried while narrowing this down. The final, shipped fix:

- Lazy-load main-view page images and thumbnails via IntersectionObserver,
  evicting whichever have scrolled out of view, so memory is bounded by
  "how many pages are near the viewport right now" instead of growing with
  total pages scrolled through.
- Worker pool sends only the page(s) actually requested to a Worker
  (extractPagesRenderData()), never the whole parsed note - previously every
  worker ended up holding a full copy of the entire document, the single
  largest contributor to the reported memory growth.
- Fixed a race condition where a page's image finalized and got "stuck"
  loaded forever if the page scrolled back out of view mid-decode, since
  eviction only re-triggers on the next intersection transition.
- Reduced the Worker pool from navigator.hardwareConcurrency (8-9 workers)
  to a fixed 2 - the peak during a fast scroll scales with how many workers
  can be simultaneously mid-growth at once, so this was the actual lever for
  the peak, not anything about individual worker lifetime.
- Tears down the whole Worker pool after 2s of inactivity, reclaiming its
  resting memory (~200MB) once a note is just sitting open; recreated
  lazily, and never mid-flight (tracks in-flight call count so idle teardown
  can only happen between calls, not during one).
- Debounced both the main-view and thumbnail-sidebar load triggers, so a
  long-distance jump (e.g. a search match 100 pages away, which animates
  through every intermediate page in a fixed, short duration) doesn't
  rasterize every page it flies past.
- Thumbnails are no longer evicted at all (cheap enough now that they're
  downscaled - a few MB for a whole document) and get a generous prefetch
  margin - also fixes a "broken image" icon bug the old evict-on-scroll-away
  behavior caused.
- PDF export: caps how many pages get embedded per pdf-lib batch, correctly
  alpha-composites each page toward white before embedding (pdf-lib's own
  PNG embedder is considerably cheaper without an alpha channel to also
  decode/retain/compress), and - since pdf-lib's own embedPng()/save() are
  still genuinely expensive no matter what (confirmed ~1.7GB / 10+s for a
  100-page note, entirely inherent to how pdf-lib decodes and retains every
  embedded image) - runs the entire rasterize-embed-save sequence inside a
  dedicated Worker so it can never freeze Obsidian's own UI thread.

Things attempted while locating the final fix:
- Capping devicePixelRatio and using a zoom-aware rasterization scale to
  shrink canvas memory - disproven by direct testing (memory unaffected).
- pdf.js document destroy on file switch - a real but minor fix, not the
  primary driver of the reported growth.
- Chrome heap-snapshot/heap-timeline analysis (custom parsing scripts) to
  rule out a JS-heap leak, since canvas/ImageBitmap memory is invisible to
  the JS heap profiler.
- Temporary DevTools-console toggles to bisect the PDF text layer vs. the
  image layer, isolating the image/Worker rasterization pipeline as the
  actual cause (removed once that bisection concluded).
- Periodically recycling (terminating/recreating) Worker instances after N
  calls, to bound per-worker internal growth - confirmed working, but later
  measured to make no real difference once the pool-size reduction and idle
  teardown were in place, so removed rather than kept as unjustified
  complexity.
- Dropping the PNG alpha channel outright before PDF embedding, instead of
  properly alpha-compositing it - shrank memory/time but produced a
  black-background PDF (background pixels are packed as black with alpha 0,
  not white), fixed by compositing toward white instead.

Bumps the supernote-typescript submodule for extractPdfPageData()/
IPdfPage (lets a whole note's pages, including recognition text, be sent to
a Worker in one structured-clone message) and flattenToWhite() (correct
alpha-to-white compositing, with tests).
supernote-typescript#42 merged as 18d5438 - repointing here from the
now-superseded pdf-page-data-for-worker branch tip (397c8fe, still an
ancestor, so this wasn't actually broken) to main directly, so this
doesn't depend on that branch sticking around.
@philips
philips merged commit 0ce47f7 into main Aug 1, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants