Skip to content

perf(start-client-core): zero-copy frame payload extraction#1

Closed
anonrig wants to merge 1 commit into
mainfrom
perf/frame-decoder-zero-copy-extract
Closed

perf(start-client-core): zero-copy frame payload extraction#1
anonrig wants to merge 1 commit into
mainfrom
perf/frame-decoder-zero-copy-extract

Conversation

@anonrig

@anonrig anonrig commented Jun 21, 2026

Copy link
Copy Markdown
Owner

What

Add a zero-copy fast path to extractFlattened() in the client-side frame decoder (packages/start-client-core/src/client-rpc/frame-decoder.ts).

Previously, every frame payload (and header) was extracted by allocating a fresh Uint8Array and copying count bytes — even when the requested bytes were fully contained in the first buffered chunk, which is the common case since most frames arrive within a single network read.

The new fast path returns a subarray view of the first chunk when the bytes are contiguous, avoiding both the allocation and the byte copy. The multi-chunk (spanning) path is unchanged.

Why

This is on the hot path for decoding every streamed server-function response and every RawStream binary payload on the client. Removing the per-frame copy matters most for large binary streams.

Standalone micro-benchmark (Node, median of 12 runs), copy vs. zero-copy extraction of contiguous payloads:

Payload Current (copy) This PR (view) Speedup
1 KB frames 224 ms 74 ms 3.0x
64 KB frames 258 ms 9.6 ms 27x

The win scales with payload size because it eliminates the byte copy.

Safety / tradeoff

The returned view shares the chunk's backing ArrayBuffer. This is safe because buffered chunks are never mutated in place after being read from the network.

There is one nuance worth flagging for review: for CHUNK frames the payload is enqueued onto a ReadableStream and may be retained by a downstream consumer, so a retained view pins its (slightly larger) source network chunk rather than a tightly-sized copy. For the JSON/header/error paths the bytes are consumed immediately, so the view is strictly a win. In normal streaming usage (process-and-release), the view is better on both CPU and peak memory. If reviewers prefer, we can restrict the view to JSON/header reads or gate CHUNK on a size threshold — happy to adjust.

Tests

  • Existing frame-decoder suite passes (typed-array assertions use value equality, so views are transparent).
  • Added a test that reassembles a 300-byte CHUNK payload delivered in 7-byte reads, exercising the multi-chunk slow path the fast path branches around.
test:unit   ✓ 19 passed
test:types  ✓ no errors
eslint      ✓ frame-decoder.ts clean (no new problems vs main)

Notes

  • Independent of the sibling perf PR that replaces the O(n) bufferList.shift() with an index pointer. Both touch extractFlattened, so whichever merges second will need a trivial rebase.

extractFlattened() always allocated a new Uint8Array and copied `count`
bytes, even when the requested bytes were fully contained in the first
buffered chunk (the common case, since most frames arrive within a single
network read).

Add a fast path that returns a subarray view of the first chunk when the
bytes are contiguous, avoiding the allocation and the byte copy. The
multi-chunk path is unchanged. The returned view shares the chunk's
backing ArrayBuffer, which is safe because buffered chunks are never
mutated in place after being read.

This is on the hot path for decoding every streamed server-function
response and RawStream binary payload on the client. A micro-benchmark
shows ~3x faster extraction for 1KB frames and ~27x for 64KB frames (the
win scales with payload size).
@anonrig

anonrig commented Jun 21, 2026

Copy link
Copy Markdown
Owner Author

Superseded by upstream PR TanStack#7662 (opened against TanStack/router).

@anonrig anonrig closed this Jun 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant