fix(mac): let real prose reach the streaming fast path — and measure what the parser path actually costs - #450
Merged
Conversation
The plain-streaming projection only engaged when every scalar in the delta was alphanumeric or whitespace. One period, comma or apostrophe disqualified the message, so essentially no natural-language answer ever qualified: in production every assistant message took the O(n^2) full-reparse path, and the recorded fast-path perf evidence measured a payload no provider sends. Replace the predicate with PlainProseStreamGate, a CommonMark-significance rule. It is an allowlist over ASCII — an unlisted ASCII scalar disqualifies rather than being assumed inert — and it covers more than core CommonMark because the renderer parses with cmark-gfm and then linkifies bare file paths, so `main.swift` and `www.example.com` are structural too. Ordinary prose punctuation (periods, commas, colons, quotes, slashes) and every non-ASCII scalar stay plain; delimiters, four-space indents, and a line's leading digit run before `.` or `)` do not. The gate is a state machine rather than a per-delta character test because the engine only ever sees one delta and three rules straddle the seam: `main` + `.` + `swift`, `link` + `](`, and a line start split from the digit that makes it a list marker. Carrying the line state and a pending-period flag makes consuming `a` then `b` identical to consuming `a + b` by construction, which is asserted directly. Two existing fixtures used syntax-free prose to exercise per-CommonMark-block fragment identity. That is the parser path's contract, so they now carry a heading or a code span; the plain path's own segmentation is by character budget and settles exactly, which the new tests state explicitly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every stream fixture emitted `\n<ordinal> ` followed by a run of one repeated letter. That payload is always eligible for the bounded plain projection, so the recorded stream evidence — the bounded 150 KB attribution scene and the 60-second soak — only ever measured the fast path, and nothing in the matrix covered what production actually did. Parameterize the stream payload by style and add the missing scenes: `sustained-stream-prose` joins the default matrix and `stream-soak-60s-prose` joins the opt-in soaks. The prose payload is deterministic assistant-shaped sentences whose every sentence carries at least one construct that leaves the plain projection, at the same byte volume, cadence, and single-paragraph shape as the existing payload, so the only variable between the paired scenes is which Markdown path engages. The fixture test asserts that separation against the production gate rather than by eyeballing the payload, and each scene's evidence note now records which payload it ran, so an artifact cannot silently claim the wrong path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Re-ran the stream evidence now that both payloads exist. The plain 60-second soak still qualifies (pinned p95/p99 8.474/14.811 ms, unpinned 4.551/5.751 ms, offscreen 4.888/6.512 ms, zero stalls over 100 ms, 13,706 frames). The prose soak does not: pinned 1,166/1,572 ms, unpinned 2,674/2,846 ms, offscreen 2,228/2,887 ms, 53 stalls over 100 ms, and 255 frames delivered in 61 seconds. It emitted only 224 of the 1,819 deltas its cadence asked for because 32.4 s of markdown.parse and 15.7 s of fragment projection ran on the main actor against 448 KB of accumulated text. Both soaks recorded zero full reloads, grouping passes, activity scans, fallbacks, and anchor failures, so the cost is parsing and projection alone. The prose scenes are therefore opt-in rather than part of the default matrix. They are a known-failing diagnostic: leaving them in `defaultCases` would make every evidence run red and destroy the matrix's ability to report a regression. The parity record now carries the numbers, the artifacts, and the exact command, and states plainly that any answer containing a delimiter still misses the frame budget by two orders of magnitude. That is the next finding, deliberately not tuned here. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
15 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The bounded plain-stream fast path could never engage for real assistant output: a single
.or,disqualified a delta, so production streams always took the full-reparse path — and every recorded perf number measured the fast path via punctuation-free fixtures.Part 1 — CommonMark-significance gate (
PlainProseStreamGate, replacing the alphanumeric-only check): ordinary prose (contractions, quotes, em dashes, CJK, emoji, sentence periods) stays plain; every Markdown-significant character class (backticks, emphasis, brackets, headings, list markers, tables, four-space indents, digit-dot line starts) still disqualifies; periods are context-sensitive somain.swift/ domains / autolinks leave the plain path butDone. Next.doesn't. Implemented as a boundary-safe state machine — a split-invariance test cuts every fixture at every character offset and asserts the verdict never changes. Disqualification is sticky per message.Part 2 — honest fixtures + measurements: the stress payloads gained a
realisticProsestyle; newsustained-stream-proseandstream-soak-60s-prosescenes exercise the parser path at identical byte volume/cadence. Results recorded in the parity matrix with the exact probe commands:markdown.parseagainst 448 KB accumulated text (51.6 MB cumulatively reparsed). FAIL, stated plainly in the matrix.So: this PR keeps syntax-free prose on the ~8.5 ms path, and produces the hard evidence that any answer containing a delimiter still cannot stream at production cadence — the O(total-per-delta) reparse needs incremental/off-main-actor parsing. That is filed as the next finding, not tuned here.
Behavior note: multi-paragraph plain prose now streams as one growing tail row (16 KB-budget segmentation) instead of freezing at each blank line, and settles into exact paragraph fragments on completion (test-covered). Three fixtures that relied on syntax-free prose hitting the parser path were retargeted with a comment each.
Carries the #447 compile fix as a
cherry-pick -xof the same commit (drops out cleanly whichever lands first).Area
apps/mac— native macOS appapps/windowsapps/mobileapps/serverRelease size
size:XSsize:Ssize:Msize:Lsize:XLVerification
pnpm run verifyafter the last edit: pass (full Swift suite)🤖 Generated with Claude Code