Skip to content

fix(realtime): raise WebRTC data-channel max-message-size + keep sendLoop alive - #10407

Merged
mudler merged 2 commits into
masterfrom
fix/realtime-dc-max-message-size
Jun 19, 2026
Merged

fix(realtime): raise WebRTC data-channel max-message-size + keep sendLoop alive#10407
mudler merged 2 commits into
masterfrom
fix/realtime-dc-max-message-size

Conversation

@localai-bot

Copy link
Copy Markdown
Collaborator

What

Raise the SCTP max-message-size LocalAI honors for the realtime oai-events
data channel, so larger server-generated events can be sent instead of being
silently dropped.

Why

In the WebRTC realtime transport, audio goes over an RTP track and events go
over the oai-events data channel
(core/http/endpoints/openai/realtime_transport_webrtc.go,
t.dc.SendText(...)). Browsers advertise a conservative a=max-message-size
in their SDP offer — Chrome uses 262144 (256 KiB). pion enforces the remote's
advertised value on send, so a single realtime event larger than 256 KiB
cannot be sent
: SendText fails, the event is dropped, the sendLoop exits,
and the turn yields no response to the client.

Some turns legitimately produce a single JSON event above 256 KiB — notably
tool-call turns (sizeable tool schemas/results). The failure is silent from
the client's side: transcription completes, then nothing comes back.

Browsers advertise the value conservatively, but their SCTP stacks reassemble
much larger messages in practice. So we raise the max-message-size honored for
our own server-generated events by rewriting the attribute in the offer
before SetRemoteDescription.

How

Two commits:

  1. Raise the negotiated max-message-size. New
    raiseDataChannelMaxMessageSize(sdp) rewrites a=max-message-size:<n> in the
    offer to realtimeDataChannelMaxMessageSize (16 MiB) before
    pc.SetRemoteDescription. Offers without the attribute are unchanged.
    Self-contained (regexp/fmt only); applied at one call site in
    RealtimeCalls.

  2. Don't tear down the session on a single failed send. sendLoop exited
    the sender goroutine on any SendText error, so one dropped event silently
    killed every subsequent event for the session. It now logs and skips the
    offending event (and keeps draining), while a genuinely dead transport is
    still handled by the <-t.closed / connection-state path. This is
    defense-in-depth: even with (1), an over-limit event degrades to a single
    dropped turn instead of a dead session.

Testing

  • Unit specs for the helper (realtime_webrtc_sctp_test.go): raises a present
    value, leaves an attribute-less offer unchanged, rewrites every occurrence,
    and asserts the ceiling is above the 256 KiB browsers advertise.
  • Reproduced end-to-end downstream: a realtime voice tool call failed with
    data channel send failed: outbound packet larger than maximum message size: 262144, producing no reply; raising the advertised size let the event through
    and the turn completed normally.

Notes

  • 16 MiB is a generous-but-bounded ceiling (pion still fragments; these are
    trusted, server-generated events). Could be made configurable if preferred.
  • Discovered while building a realtime voice app against LocalAI; the same
    failure mode affects any client whose browser advertises the 256 KiB default.

mudler added 2 commits June 19, 2026 16:40
…vents

Browsers advertise a conservative SCTP max-message-size in their SDP offer
(Chrome uses 256 KiB). pion enforces the remote's advertised value on send, so
a single realtime event larger than it cannot be sent over the "oai-events"
data channel: SendText fails, the event is dropped, and the turn silently
yields no response. Some turns legitimately produce a >256 KiB JSON event —
notably tool calls with sizeable schemas or results.

Browsers advertise the value conservatively but their SCTP stacks reassemble
much larger messages, so raise the max-message-size honored for our own
server-generated events by rewriting the attribute in the offer before
SetRemoteDescription.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A failed SendText on the oai-events data channel exited the sender goroutine,
so a single dropped event (e.g. one over the negotiated SCTP max-message-size)
tore down the session and silently dropped every subsequent event. Log and skip
the offending event instead and keep draining; a genuinely dead transport is
still handled by the closed / connection-state path.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
mudler merged commit 079ac0e into master Jun 19, 2026
60 checks passed
@mudler
mudler deleted the fix/realtime-dc-max-message-size branch June 19, 2026 19:36
@localai-bot localai-bot added the bug Something isn't working label Jun 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants