fix(realtime): raise WebRTC data-channel max-message-size + keep sendLoop alive - #10407
Merged
Conversation
…vents Browsers advertise a conservative SCTP max-message-size in their SDP offer (Chrome uses 256 KiB). pion enforces the remote's advertised value on send, so a single realtime event larger than it cannot be sent over the "oai-events" data channel: SendText fails, the event is dropped, and the turn silently yields no response. Some turns legitimately produce a >256 KiB JSON event — notably tool calls with sizeable schemas or results. Browsers advertise the value conservatively but their SCTP stacks reassemble much larger messages, so raise the max-message-size honored for our own server-generated events by rewriting the attribute in the offer before SetRemoteDescription. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A failed SendText on the oai-events data channel exited the sender goroutine, so a single dropped event (e.g. one over the negotiated SCTP max-message-size) tore down the session and silently dropped every subsequent event. Log and skip the offending event instead and keep draining; a genuinely dead transport is still handled by the closed / connection-state path. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Raise the SCTP
max-message-sizeLocalAI honors for the realtimeoai-eventsdata channel, so larger server-generated events can be sent instead of being
silently dropped.
Why
In the WebRTC realtime transport, audio goes over an RTP track and events go
over the
oai-eventsdata channel (core/http/endpoints/openai/realtime_transport_webrtc.go,t.dc.SendText(...)). Browsers advertise a conservativea=max-message-sizein their SDP offer — Chrome uses
262144(256 KiB). pion enforces the remote'sadvertised value on send, so a single realtime event larger than 256 KiB
cannot be sent:
SendTextfails, the event is dropped, thesendLoopexits,and the turn yields no response to the client.
Some turns legitimately produce a single JSON event above 256 KiB — notably
tool-call turns (sizeable tool schemas/results). The failure is silent from
the client's side: transcription completes, then nothing comes back.
Browsers advertise the value conservatively, but their SCTP stacks reassemble
much larger messages in practice. So we raise the
max-message-sizehonored forour own server-generated events by rewriting the attribute in the offer
before
SetRemoteDescription.How
Two commits:
Raise the negotiated max-message-size. New
raiseDataChannelMaxMessageSize(sdp)rewritesa=max-message-size:<n>in theoffer to
realtimeDataChannelMaxMessageSize(16 MiB) beforepc.SetRemoteDescription. Offers without the attribute are unchanged.Self-contained (
regexp/fmtonly); applied at one call site inRealtimeCalls.Don't tear down the session on a single failed send.
sendLoopexitedthe sender goroutine on any
SendTexterror, so one dropped event silentlykilled every subsequent event for the session. It now logs and skips the
offending event (and keeps draining), while a genuinely dead transport is
still handled by the
<-t.closed/ connection-state path. This isdefense-in-depth: even with (1), an over-limit event degrades to a single
dropped turn instead of a dead session.
Testing
realtime_webrtc_sctp_test.go): raises a presentvalue, leaves an attribute-less offer unchanged, rewrites every occurrence,
and asserts the ceiling is above the 256 KiB browsers advertise.
data channel send failed: outbound packet larger than maximum message size: 262144, producing no reply; raising the advertised size let the event throughand the turn completed normally.
Notes
trusted, server-generated events). Could be made configurable if preferred.
failure mode affects any client whose browser advertises the 256 KiB default.