feat(realtime): land LiveKit dead code + LK-based subscribe (PR 2 of 3) by VerioN1 · Pull Request #138 · DecartAI/sdk

VerioN1 · 2026-05-15T11:28:42Z

Summary

Brings in the LiveKit transport layer as importable-but-mostly-unwired code, and rewires the subscribe path onto it. Publish (createRealTimeClient) is untouched and still uses the existing aiortc WebRTCManager — the final swap happens in PR 3 (the current alon/livekit-prod HEAD).

This PR is structured so that every line either belongs to the LK transport (new files) or to a strictly additive change on the existing surface. Nothing aiortc was modified for the sake of giving this PR meaning.

What lands

New modules: stream-session.ts, signaling-channel.ts, media-channel.ts, remote-stream-exposure.ts, observability/livekit-stats-provider.ts, realtime/config-realtime.ts
New utils: utils/media.ts (shared imageToBase64), utils/platform.ts
RealtimeObservability gains setLiveKitRoom() that wires a LK-room stats provider into the same stats pipeline introduced in PR 1
types.ts becomes a superset:
- keeps every aiortc message type (OfferMessage/AnswerMessage/IceCandidateMessage/ReadyMessage/SessionIdMessage/etc.) — still consumed by webrtc-connection/webrtc-manager
- adds LK message + domain types (LiveKitJoinMessage, LiveKitRoomInfoMessage, QueuePosition/QueuePositionMessage, ConnectionStatus, SessionStarted, InitialState, ServerError, ImageSetOptions, …)
subscribe-client.ts is rewritten onto livekit-client + RealtimeObservability with the new room_name token format (was sid/ip/port). The aiortc-form encoder/decoder previously living here and used by the publish path's subscribeToken is inlined into realtime/client.ts so it dies cleanly when client.ts is rewritten in PR 3
createDecartClient now exposes realtime.{ connect, subscribe } where subscribe comes from createRealTimeSubscribeClient
Adds livekit-client@^2 to packages/sdk deps

Intentionally deferred to PR 3

Logger default stays noopLogger (PR 3 flips to createConsoleLogger(\"info\"))
diagnostics.ts keeps the LK-incompatible event types (IceCandidateEvent, IceStateEvent, PeerConnectionStateEvent, SignalingStateEvent, SelectedCandidatePairEvent) because aiortc publish still emits them; PR 3 drops them along with aiortc
Publish path swap (client.ts → StreamSession, deletion of webrtc-manager.ts/webrtc-connection.ts, methods.ts rewrite, aiortc tests removal)

Tests

Adds tests/realtime-observability-unit.test.ts and tests/realtime.unit.test.ts (the set() block from the LK branch is excluded — it depends on PR 3's methods.ts)
A few tests/unit.test.ts assertions adjusted to the new shape:
- removed the aiortc-form Subscribe Token round-trip block (now covered by the new room_name tests in realtime.unit.test.ts)
- inlined the legacy decoder in the session_id populates subscribeToken test
- removed the TelemetryReporter "warns on non-2xx" test (telemetry is now silent on non-2xx by design — .catch(() => {}))

Test plan

pnpm typecheck clean
pnpm test — 217/217 passing
pnpm exec biome check . clean
Manual smoke: producer (createDecartClient().realtime.connect) still works against existing aiortc backend
Manual smoke: consumer (createDecartClient().realtime.subscribe) connects to a LK-backed /watch-stream token

🤖 Generated with Claude Code

Note

Medium Risk
Introduces a new LiveKit-based realtime transport layer (WebSocket signaling + LiveKit media) plus new retry/handshake logic and telemetry diagnostics; although mostly additive, it touches connection and observability paths that can affect reliability and reporting.

Overview
Adds a new LiveKit-based realtime transport scaffold: SignalingChannel manages WebSocket join/ack flows (including queue position and generation events), MediaChannel manages LiveKit room connection, remote stream aggregation, and local track publishing, and StreamSession ties them together with retry/reconnect behavior and initial-state gating.

Enhances observability/telemetry by centralizing timeouts/thresholds in REALTIME_CONFIG, adding a client-session-connection-breakdown diagnostic (phase timings + initial image size), and wiring LiveKit rooms into the existing stats pipeline via setLiveKitRoom() + a LiveKit RTC stats provider.

Adds utility helpers (utils/media, utils/platform), unit tests for the new observability breakdown/stats behavior and signaling close handling, and introduces the livekit-client dependency (lockfile updates included).

^{Reviewed by Cursor Bugbot for commit 7269fa0. Bugbot is set up for automated code reviews on this repo. Configure here.}

pkg-pr-new · 2026-05-15T11:29:52Z

Open in StackBlitz

npm i https://pkg.pr.new/@decartai/sdk@138

commit: 7269fa0

cursor · 2026-05-15T11:30:45Z

     * totalInterFrameDelay + totalSquaredInterFrameDelay.
     */
-    interFrameDelayStdDevMs: number | null;
+    interFrameDelayVarianceMs: number | null;


Field named "variance" actually computes standard deviation

Medium Severity

The field interFrameDelayStdDevMs was renamed to interFrameDelayVarianceMs, but the computation still calculates the standard deviation (square root of variance), not variance. The comment even still states "Report std-dev in ms." Consumers relying on this field as a variance value will get incorrect results — variance and standard deviation have different units and magnitudes (variance is in ms², std-dev is in ms).

Additional Locations (1)

packages/sdk/src/realtime/observability/webrtc-stats.ts#L294-L303

^{Reviewed by Cursor Bugbot for commit a7a41d2. Configure here.}

Brings in the LiveKit transport layer as importable-but-unwired code. No live caller on the publish or subscribe path reaches any of the new modules — they sit alongside the existing aiortc transport until PR 3 performs the final swap. What lands: - New modules: stream-session, signaling-channel, media-channel, remote-stream-exposure, observability/livekit-stats-provider, realtime/config-realtime - New utils: utils/media (imageToBase64), utils/platform - RealtimeObservability gains setLiveKitRoom() that wires a LiveKit-room stats provider into the same stats pipeline introduced in PR 1. setLiveKitRoom is only invoked from within MediaChannel (also dead), so the orchestrator's behavior on the live aiortc path is unchanged. - types.ts becomes a superset: keeps every aiortc message type (still consumed by webrtc-connection/manager) and adds LK message + domain types (LiveKitJoin/RoomInfo, QueuePosition, ConnectionStatus, SessionStarted, InitialState, ServerError, ImageSetOptions, ...). No new types are re-exported from index.ts — public surface unchanged. - Adds livekit-client@^2 to packages/sdk deps (required for the new modules to typecheck even with no live caller). Notes for PR 3: - subscribe-client.ts and createDecartClient still expose the aiortc subscribe path unchanged. PR 3 swaps the implementation behind it. - diagnostics keeps the LK-incompatible event types (IceCandidateEvent, IceStateEvent, PeerConnectionStateEvent, SignalingStateEvent, SelectedCandidatePairEvent) because aiortc publish still emits them; PR 3 drops them along with aiortc. - New tests: realtime-observability-unit.test.ts (covers the orchestrator including setLiveKitRoom indirectly via setStatsProvider). Dead-code invariant: every reference to stream-session, signaling-channel, media-channel, remote-stream-exposure, livekit-stats-provider, and livekit-client lives inside one of the new LK files. No aiortc-path file imports any of them. Validation: typecheck clean, 193/193 tests pass, biome diagnostics unchanged vs origin/main. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Refresh the LK transport modules landed in PR 2 to match their current state on alon/livekit-prod so PR 3 reduces to wiring + aiortc removal. - stream-session, media-channel, signaling-channel, observability/realtime-observability, utils/platform, tests/realtime-observability-unit.test.ts: taken from prod verbatim. - initial-state-gate.ts: pulled from prod (now imported by stream-session). - remote-stream-exposure.ts: deleted (no longer used in prod). - types.ts and observability/diagnostics.ts: prod content + aiortc-only types/events retained so webrtc-connection/webrtc-manager still compile. PR 3 will drop those superset entries with the aiortc files. Validation: typecheck clean, 196/196 tests pass, biome clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…kit-deadcode

AdirAmsalem · 2026-05-19T12:03:57Z

+  private async request<TAck extends IncomingRealtimeMessage>(
+    message: OutgoingRealtimeMessage,
+    matchAck: (msg: IncomingRealtimeMessage) => boolean,
+    timeoutMs: number,
+    label: string,
+  ): Promise<TAck> {


lets make it accept a configuration obj? having 4 different params (and those one specifically) makes it too vague on the caller side

Co-authored-by: Cursor <cursoragent@cursor.com>

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 7269fa0. Configure here.}

cursor · 2026-05-19T12:33:33Z

    }

    this.stopStats();
+    this.resetStallDetection();


Redundant resetStallDetection call after stopStats

Low Severity

In setStatsProvider, this.stopStats() is called on line 159 and then this.resetStallDetection() is called immediately after on line 160. However, stopStats() already calls this.resetStallDetection() internally (line 191), making the second call redundant.

Additional Locations (1)

packages/sdk/src/realtime/observability/realtime-observability.ts#L185-L192

^{Reviewed by Cursor Bugbot for commit 7269fa0. Configure here.}

cursor Bot reviewed May 15, 2026

View reviewed changes

VerioN1 force-pushed the alon/livekit-pr2-livekit-deadcode branch from a7a41d2 to 0b24f12 Compare May 17, 2026 15:16

cursor Bot reviewed May 17, 2026

View reviewed changes

Comment thread packages/sdk/src/realtime/signaling-channel.ts

Comment thread packages/sdk/src/realtime/remote-stream-exposure.ts Outdated

VerioN1 and others added 2 commits May 18, 2026 18:27

Merge remote-tracking branch 'origin/main' into alon/livekit-pr2-live…

0772b86

…kit-deadcode

cursor Bot reviewed May 19, 2026

View reviewed changes

Comment thread packages/sdk/src/realtime/stream-session.ts

Comment thread packages/sdk/src/realtime/observability/realtime-observability.ts

AdirAmsalem reviewed May 19, 2026

View reviewed changes

VerioN1 and others added 2 commits May 19, 2026 15:20

test(realtime): cover LiveKit room stats dedupe

56d5e60

Co-authored-by: Cursor <cursoragent@cursor.com>

fix(realtime): preserve LiveKit room stats dedupe

3771e64

Co-authored-by: Cursor <cursoragent@cursor.com>

cursor Bot reviewed May 19, 2026

View reviewed changes

Comment thread packages/sdk/src/realtime/media-channel.ts

added fix to signaling state and params

7269fa0

cursor Bot reviewed May 19, 2026

View reviewed changes

AdirAmsalem approved these changes May 19, 2026

View reviewed changes

VerioN1 merged commit 4a7c4b5 into main May 19, 2026
5 checks passed

VerioN1 deleted the alon/livekit-pr2-livekit-deadcode branch May 19, 2026 12:44

tomershlasky mentioned this pull request May 26, 2026

feat(realtime): lucy-2.5 + optional sampleFrameData on set/setImage #135

Draft

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(realtime): land LiveKit dead code + LK-based subscribe (PR 2 of 3)#138

feat(realtime): land LiveKit dead code + LK-based subscribe (PR 2 of 3)#138
VerioN1 merged 6 commits into
mainfrom
alon/livekit-pr2-livekit-deadcode

VerioN1 commented May 15, 2026 •

edited by cursor Bot

Loading

Uh oh!

pkg-pr-new Bot commented May 15, 2026 •

edited

Loading

Uh oh!

cursor Bot May 15, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AdirAmsalem May 19, 2026

Uh oh!

Uh oh!

cursor Bot left a comment

Uh oh!

cursor Bot May 19, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

VerioN1 commented May 15, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What lands

Intentionally deferred to PR 3

Tests

Test plan

Uh oh!

pkg-pr-new Bot commented May 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cursor Bot May 15, 2026

Choose a reason for hiding this comment

Field named "variance" actually computes standard deviation

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AdirAmsalem May 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot May 19, 2026

Choose a reason for hiding this comment

Redundant resetStallDetection call after stopStats

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

VerioN1 commented May 15, 2026 •

edited by cursor Bot

Loading

pkg-pr-new Bot commented May 15, 2026 •

edited

Loading