perf(start-client-core): zero-copy frame payload extraction#1
Closed
anonrig wants to merge 1 commit into
Closed
Conversation
extractFlattened() always allocated a new Uint8Array and copied `count` bytes, even when the requested bytes were fully contained in the first buffered chunk (the common case, since most frames arrive within a single network read). Add a fast path that returns a subarray view of the first chunk when the bytes are contiguous, avoiding the allocation and the byte copy. The multi-chunk path is unchanged. The returned view shares the chunk's backing ArrayBuffer, which is safe because buffered chunks are never mutated in place after being read. This is on the hot path for decoding every streamed server-function response and RawStream binary payload on the client. A micro-benchmark shows ~3x faster extraction for 1KB frames and ~27x for 64KB frames (the win scales with payload size).
Owner
Author
|
Superseded by upstream PR TanStack#7662 (opened against TanStack/router). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Add a zero-copy fast path to
extractFlattened()in the client-side frame decoder (packages/start-client-core/src/client-rpc/frame-decoder.ts).Previously, every frame payload (and header) was extracted by allocating a fresh
Uint8Arrayand copyingcountbytes — even when the requested bytes were fully contained in the first buffered chunk, which is the common case since most frames arrive within a single network read.The new fast path returns a
subarrayview of the first chunk when the bytes are contiguous, avoiding both the allocation and the byte copy. The multi-chunk (spanning) path is unchanged.Why
This is on the hot path for decoding every streamed server-function response and every
RawStreambinary payload on the client. Removing the per-frame copy matters most for large binary streams.Standalone micro-benchmark (Node, median of 12 runs), copy vs. zero-copy extraction of contiguous payloads:
The win scales with payload size because it eliminates the byte copy.
Safety / tradeoff
The returned view shares the chunk's backing
ArrayBuffer. This is safe because buffered chunks are never mutated in place after being read from the network.There is one nuance worth flagging for review: for
CHUNKframes the payload is enqueued onto aReadableStreamand may be retained by a downstream consumer, so a retained view pins its (slightly larger) source network chunk rather than a tightly-sized copy. For the JSON/header/error paths the bytes are consumed immediately, so the view is strictly a win. In normal streaming usage (process-and-release), the view is better on both CPU and peak memory. If reviewers prefer, we can restrict the view to JSON/header reads or gateCHUNKon a size threshold — happy to adjust.Tests
frame-decodersuite passes (typed-array assertions use value equality, so views are transparent).CHUNKpayload delivered in 7-byte reads, exercising the multi-chunk slow path the fast path branches around.Notes
bufferList.shift()with an index pointer. Both touchextractFlattened, so whichever merges second will need a trivial rebase.