Skip to content

perf(mac): incremental streaming Markdown — parser path now costs the same as the fast path at real scale - #453

Merged
SergeSerb2 merged 3 commits into
mainfrom
surgecode/incremental-markdown-stream
Aug 2, 2026
Merged

perf(mac): incremental streaming Markdown — parser path now costs the same as the fast path at real scale#453
SergeSerb2 merged 3 commits into
mainfrom
surgecode/incremental-markdown-stream

Conversation

@SergeSerb2

Copy link
Copy Markdown
Owner

Summary

Closes the streaming-perf P0 that #450 measured: any answer containing a bullet, link, or code span fell onto an O(n²) main-actor reparse (pinned p95 1,166 ms, only 224/1,819 deltas consumed in the 60 s soak). Four compounding fixes, each measured:

  1. List-safe split boundaries: the scanner now promotes boundaries before zero-indent list markers — long streaming lists (the most common assistant shape) previously never settled any prefix. The cache renumbers a lazily numbered ("1./1./1.") ordered run continuing across a seam so streamed display equals the full parse; nested/indented markers still refuse the split. Every-split equivalence with the exact parser is property-tested (fixtures cut at every character offset).
  2. Frozen-prefix projection: ParsedMarkdownDocument.settledBlockCount lets both the fragment projection and the store's render-row replacement reuse the settled prefix verbatim — per-soak render-row projections fell 3.65 M → 10 k.
  3. Validated append hint reaches the cache: no more O(accumulated) exact prefix compare per delta (was 24.6 s of the soak's main-actor time; the exactness fix from fix(mac): AppKit markdown emphasis/appearance and sidebar selection fixes #441 stays as the fallback for anything without a valid hint).
  4. Content-gap recovery: a pure content-lane revision gap applies the slice's absolute item values instead of a full presentation rebuild. The rebuild used to outlast two stream ticks and cascade — nine rebuild-gap-rebuild chains per soak, now zero. Structural/metadata/environment gaps still rebuild; append hints are ignored on the recovery path.

Render-row diffs now trust the engine's revision contract instead of comparing AttributedStrings per fragment; MarkdownMessageChrome drops the source string (copy actions read the semantic row) so it stays O(1)-comparable.

Measured (new structured-prose payload + scenes — the shape providers actually send; commands + artifacts in the parity matrix):

  • 150 KB bounded: pinned p95 23.6 ms vs the plain fast path's 23.3 ms in the same harness (pre-fix: 87.7 / 219.9 / 286.1 ms) — parser path == fast path at realistic scale.
  • Adversarial 60 s / 3.7 MB / ~12,000-block single message: all 1,817 deltas consumed (was 224), zero fallbacks, pinned p95 ~170–192 ms. Residual cost is the update protocol's O(rows) ID arrays per mutation — recorded in the matrix as the follow-up; it only bites at that scale.

Area

  • apps/mac — native macOS app
  • apps/windows
  • apps/mobile
  • apps/server
  • Shared packages or relay
  • Build, CI, or release tooling
  • Docs

Release size

  • size:XS
  • size:S
  • size:M
  • size:L
  • size:XL

Verification

🤖 Generated with Claude Code

SergeSerb2 and others added 3 commits August 2, 2026 15:57
… end

The parser path reparsed and re-projected the whole accumulated message
per provider delta. Four compounding fixes:

- MarkdownSafeSplitScanner promotes boundaries before zero-indent list
  markers (indented markers still refuse — a fresh parse would flatten a
  nested item). Long streaming lists — the most common assistant output
  shape — previously never settled any prefix. StreamingMarkdownCache
  renumbers a lazily numbered ordered run continuing across a settle
  seam; every-split equivalence with the full parse is test-enforced,
  and the completed message's exact parse still canonicalizes.
- ParsedMarkdownDocument carries settledBlockCount; the fragment
  projection and the store's render-row replacement reuse the frozen
  prefix instead of re-segmenting/re-comparing/re-allocating the whole
  document per delta (3.65M -> 10k render-row projections per soak).
- The engine's validated append hint reaches the streaming cache, which
  appends just the suffix instead of an O(accumulated) exact prefix
  compare per delta (the compare was the single largest remaining
  main-actor cost at 24.6 s per soak).
- A pure content-lane revision gap applies the slice's absolute item
  values and adopts the revision instead of a full presentation rebuild.
  The rebuild outlasted two stream ticks, dropping the next mutation and
  gapping again — a measured cascade of nine rebuilds per soak. Append
  hints are ignored on the recovery path; structural/metadata/environment
  gaps still rebuild.

Render-row diffs compare revisions (and O(1)-in-the-common-case chrome)
instead of AttributedStrings, per the engine's revision contract.
MarkdownMessageChrome drops the markdown source (copy actions read the
semantic row) so streaming chrome stays comparably empty.

New structured-prose stress payload + scenes measure the parser path in
the shape providers actually send: 150 KB bounded now equals the plain
fast path (23.6 ms vs 23.3 ms pinned, from 87.7/219.9/286.1 ms); the
adversarial 3.7 MB 60 s soak consumes all 1,817 deltas (was 224/1,819 at
p95 1,166 ms) and its residual cost — O(rows) update-protocol ID arrays —
is recorded in the parity matrix as the follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@SergeSerb2 SergeSerb2 added the size:M Normal feature or meaningful behavior change label Aug 2, 2026
@SergeSerb2
SergeSerb2 merged commit 98908d1 into main Aug 2, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M Normal feature or meaningful behavior change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant