Streaming appends ship O(K) delta frames (protocol v8)

[Alek99]

Stacked on #187 (base: feat/shader-channel-domains). Final PR of the streaming-append stack (#178 → #186 → #187 → this): takes the append hot path from O(N) to O(rows appended).

What

A direct-tier append still re-shipped the whole affected trace — appending one point to a 100k scatter cost ~783 KiB. Now the kernel emits an append_rows delta (only the K new rows) whenever the shipped representation is a pure tail extension of what the client holds.

Kernel — strict, recorded eligibility (every exclusion rides the full message as append.delta_fallback, per §28): direct-tier scatter/line; no tier flip; no keyed/configured animation; no per-point style channels; no step/smooth vertex expansion; no log axes; fully-finite rows (a drop would fork shipped row order); raw-encoded channels only. Plus an offset-drift budget: tails encode with the offsets the client already holds, capped at 1024× the encoded half-span recorded at the last full ship (~0.25 px worst-case at 2048 px) — past it, one full re-ship re-centers exactly like deep zoom (§4/§16). Screen-bounded tiers (decimated/density) always take the full path: their whole payload is already O(pixels).

Client — _applyAppendRows is an in-place tail write, not a payload swap: capacity-doubling GPU buffers (bufferSubData, one amortized re-upload per doubling), retained CPU spans grown the same way (GL context restore stays whole, §27), spec metas advanced, grown channel domains applied as uniform updates — this is what #187's raw channel wire bought: domain growth costs zero re-encoded bytes. Enabled attribute ranges keep covering n (selection masks zero-fill new rows), and the home/live-edge/history follow policy runs off the fresh axis ranges. prev_marks and the encode params must match exactly; any disagreement (missed message, drifted baseline) sends one refresh instead of writing a torn tail.

Fixed in passing: appending to a trace whose x and y alias one deduplicated canonical column (scatter(v, v)) silently interleaved both axes into that single column — pre-existing corruption, surfaced by a new test fixture. Now rejected like cross-trace sharing, with a test.

Protocol bumps to v8 — a pre-v8 bundle would silently drop delta frames mid-stream.

Measured

Median per tick, same bench as the stack (base = #187):

Case	Wire/tick	Kernel ms/tick
direct scatter 100k, append 1 pt	783 KiB → 383 bytes (~2000×)	0.20 → 0.14
10×50k dashboard, 1 trace streaming	397 KiB → ~0.4 KiB (~1000×)	0.27 → 0.19
decimated line 2M	15 KiB → 15 KiB (by design)	0.52

Cumulative across the stack for the dashboard case: 7.8 MB → 0.4 KiB per tick (#178 halved it, #186 removed the unchanged traces, this PR removes the O(N) re-ship of the affected trace).

Verified

• Full suite 2206 passed (new: delta shape + consecutive prev_marks tracking, pick-after-delta exactness, offset-drift fallback + recovery, per-reason fallbacks (non-finite tail, keyed animation, tier flip), post-delta full-build coherence, post-delta neighbor-append re-ships the streamed trace, the x≡y alias rejection).
• Browser probe (built ESM bundle, real kernel payloads): delta applies in place (GPU count 101→103→104, CPU spans + spec metas grow, tail decodes back to exact data units through the retained metas, raw color tail + moved domain updates the uniform map, home refit follows), consecutive deltas apply, and a replayed out-of-order delta is refused with exactly one refresh and no tear. All green.
• ruff / format / pre-commit / ty (25 pre-existing, none new) / node js/build.mjs.

Spec

wire-protocol.md §4 (append_rows contract + eligibility + drift budget), §7 (v8); rust-engine.md §5 (Python-side delta landed; stream.rs remains the native follow-up). CHANGELOG under Unreleased → Changed.

🤖 Generated with Claude Code

[codspeed-hq]

Merging this PR will not alter performance

✅ 97 untouched benchmarks
⏩ 1 skipped benchmark¹

---

_{Comparing feat/append-delta-frames (5199672) with feat/shader-channel-domains (df3687a)}

Footnotes

1. 1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports. ↩

[greptile-apps]

Greptile Summary

This PR adds compact delta frames for streaming appends. The main changes are:

• Protocol v8 support for append_rows messages.
• Strict kernel eligibility checks with full-message fallbacks.
• In-place CPU and GPU buffer growth on the client.
• Selection-mask preservation and atomic delta validation.
• Updated streaming tests, protocol docs, and generated bundles.

Confidence Score: 5/5

This looks safe to merge.

Delta roles are validated before retained columns are changed. Selection growth preserves the existing mask and tracks the real GPU allocation. No blocking issue remains in the updated paths.

T-Rex Logs

What T-Rex did

• Established the initial contract state: the ESM protocol 8 was built with 8 marks, retained lengths [8,8], and zero refreshes.
• Observed the post-update state: the kernel emitted two tail-only append_rows frames carrying 8-byte and 4-byte buffers per channel, and the client reached 12 marks with the expected decoded tail.
• Ran replay checks by applying the older prev_marks=9 delta at count 12, which caused exactly one refresh and produced no torn or duplicated tail.
• Reviewed and organized artifacts, including logs, Python and JavaScript artifacts, an HTML resource, and a rendered image, to support inspection.
• Overall, the gathered evidence confirms the contract-validation path behaved as described across initial, post-update, and replay scenarios.

_{Ran code and verified through T-Rex}

Important Files Changed

Filename	Overview
js/src/54_kernel.ts	Applies append deltas in place and validates all referenced columns before changing retained state.
js/src/53_interaction.ts	Retains selection masks and synchronizes GPU capacity after mask uploads.
python/xy/interaction.py	Builds eligible append deltas, records fallback reasons, and rejects aliased x/y columns.
python/xy/_payload.py	Records full-ship geometry spans for the delta drift limit.
python/xy/config.py	Updates the Python wire protocol version to v8.

_{Reviews (3): Last reviewed commit: "Merge feat/shader-channel-domains: paylo..." | Re-trigger Greptile}