perf(mac): incremental streaming Markdown — parser path now costs the same as the fast path at real scale - #453
Merged
Conversation
… end The parser path reparsed and re-projected the whole accumulated message per provider delta. Four compounding fixes: - MarkdownSafeSplitScanner promotes boundaries before zero-indent list markers (indented markers still refuse — a fresh parse would flatten a nested item). Long streaming lists — the most common assistant output shape — previously never settled any prefix. StreamingMarkdownCache renumbers a lazily numbered ordered run continuing across a settle seam; every-split equivalence with the full parse is test-enforced, and the completed message's exact parse still canonicalizes. - ParsedMarkdownDocument carries settledBlockCount; the fragment projection and the store's render-row replacement reuse the frozen prefix instead of re-segmenting/re-comparing/re-allocating the whole document per delta (3.65M -> 10k render-row projections per soak). - The engine's validated append hint reaches the streaming cache, which appends just the suffix instead of an O(accumulated) exact prefix compare per delta (the compare was the single largest remaining main-actor cost at 24.6 s per soak). - A pure content-lane revision gap applies the slice's absolute item values and adopts the revision instead of a full presentation rebuild. The rebuild outlasted two stream ticks, dropping the next mutation and gapping again — a measured cascade of nine rebuilds per soak. Append hints are ignored on the recovery path; structural/metadata/environment gaps still rebuild. Render-row diffs compare revisions (and O(1)-in-the-common-case chrome) instead of AttributedStrings, per the engine's revision contract. MarkdownMessageChrome drops the markdown source (copy actions read the semantic row) so streaming chrome stays comparably empty. New structured-prose stress payload + scenes measure the parser path in the shape providers actually send: 150 KB bounded now equals the plain fast path (23.6 ms vs 23.3 ms pinned, from 87.7/219.9/286.1 ms); the adversarial 3.7 MB 60 s soak consumes all 1,817 deltas (was 224/1,819 at p95 1,166 ms) and its residual cost — O(rows) update-protocol ID arrays — is recorded in the parity matrix as the follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the streaming-perf P0 that #450 measured: any answer containing a bullet, link, or code span fell onto an O(n²) main-actor reparse (pinned p95 1,166 ms, only 224/1,819 deltas consumed in the 60 s soak). Four compounding fixes, each measured:
ParsedMarkdownDocument.settledBlockCountlets both the fragment projection and the store's render-row replacement reuse the settled prefix verbatim — per-soak render-row projections fell 3.65 M → 10 k.Render-row diffs now trust the engine's revision contract instead of comparing AttributedStrings per fragment;
MarkdownMessageChromedrops the source string (copy actions read the semantic row) so it stays O(1)-comparable.Measured (new
structured-prosepayload + scenes — the shape providers actually send; commands + artifacts in the parity matrix):Area
apps/mac— native macOS appapps/windowsapps/mobileapps/serverRelease size
size:XSsize:Ssize:Msize:Lsize:XLVerification
pnpm run verify --allafter merging current main (incl. fix(mac): optimistic send — the message appears when you press Enter #451/feat: narrate worktree setup — echo first, typed phase progress, setup script on the eager path #452): 5/5 steps, exit 0.artifacts/incremental-md-*, honest numbers in the parity matrix🤖 Generated with Claude Code