perf(server): skip shell refresh for streaming deltas#4009
Conversation
Streaming assistant messages still update their message projection and thread timestamp, but no longer rebuild the full thread shell summary for every delta. Add a regression test that fails if the hot path reads activity history.
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
ApprovabilityVerdict: Approved A small, well-tested performance optimization that conditionally skips shell summary refresh for streaming assistant message deltas. The change is self-contained with clear intent and comprehensive test coverage. You can customize Macroscope's approvability policy. Learn more. |
What Changed
Skip
refreshThreadShellSummaryfor streaming assistant messages while preserving message projection and the threadupdatedAt. Add a regression test that fails if this hot path reads thread activity history.Why
Large threads can retain hundreds of MB of tool activity. Rebuilding the shell summary for every text delta made projection time proportional to the full thread history and throttled visible streaming to about one delta per second.
Fixes #4008.
This is the minimal shell-refresh subset of #2631; it does not change message repository APIs or persistence behavior.
UI Changes
No UI code changed; this fixes streaming timing.
T3.Code.Slow.TPS.FIXED.2026-07-15.101452.mp4
Testing
vp test apps/server/src/orchestration/Layers/ProjectionPipeline.test.ts(22 passed)vp check(passes; 9 pre-existing warnings)vp run typecheckChecklist
Note
Medium Risk
Changes projection hot-path behavior so thread shell summary fields may lag during streaming until a non-streaming event refreshes them; scoped change with test coverage.
Overview
Streaming assistant deltas no longer trigger a full thread shell summary rebuild on each
thread.message-sentprojection. The threads projector still bumpsupdated_at, but it skipsrefreshThreadShellSummarywhen the event is an assistant message withstreaming: true, avoiding repeated scans of messages, activities, and pending approvals on every text chunk.A regression test seeds invalid activity JSON for the thread and projects a streaming assistant delta; if the hot path reads activity history, projection fails. It also asserts the streamed message and thread timestamp still update correctly.
Reviewed by Cursor Bugbot for commit 69d5c69. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Skip shell summary refresh for streaming assistant messages in projection pipeline
When processing a
thread.message-sentevent where the role isassistantand the message is marked as streaming, the projector in ProjectionPipeline.ts now skips the shell summary refresh. The threadupdated_atis still updated. A new test verifies no historical activity scan occurs during streaming delta handling.Macroscope summarized 69d5c69.