fix(openai): stop realtime reconnect loop on fatal errors - #2031
Merged
Conversation
🦋 Changeset detectedLatest commit: 839dada The changes in this PR will be included in the next version bump. This PR includes changesets to release 36 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
Preserve empty-code fallback and resolve discarded generations while verifying terminal websocket errors stop reconnecting. Co-authored-by: Cursor <cursoragent@cursor.com>
toubatbrian
approved these changes
Jul 14, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
errorand failedresponse.doneevents out of the websocket loop so the session stops reconnectinggenerateReplyfutures when reconnecting or when the realtime session exits@livekit/agents-plugin-openaiSource diff coverage
Files from livekit/agents#6352
livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/realtime/realtime_model.py: adapted toplugins/openai/src/realtime/realtime_model.ts. The fatal-code set and helper were translated to TypeScript; fatal server errors now create non-retryableAPIErrors, websocket message handling propagates those errors to the main reconnect loop, and reconnect/session-exit cleanup closes active generations plus rejects pending response-created futures. This is adapted rather than literal because the JS implementation useswscallbacks,Task,Future, and event-driven websocket close futures instead of Python async receive tasks.tests/test_realtime/test_openai_realtime_model.py: ported toplugins/openai/src/realtime/realtime_model.test.ts. The source fatal classification tests and handler behavior tests were added to the existing Vitest realtime suite using the target’sRealtimeModel/RealtimeSessiontest setup.Validation
pnpm prettier --write "plugins/openai/src/realtime/realtime_model.ts" "plugins/openai/src/realtime/realtime_model.test.ts" ".changeset/openai-realtime-fatal-errors.md"pnpm buildpassed after the follow-up fixespnpm lintpassed with existing warningspnpm test plugins/openai/src/realtime/realtime_model.test.tspassed: 41 testspnpm test plugins/openaipassed 6 files / 109 tests (3 skipped); the package-wide selector remains blocked by an existing unrelated collection failure inplugins/openai/src/tts.test.tsbecausegpt-realtime-whisperis constructed without the now-required VADPorted from livekit/agents#6352
Original PR description
Addresses the production failure in #6205 . Rescoped per review (https://github.com/livekit/agents/pull/6352#issuecomment-4920965533)— see "Scoped out" below.
Motivation
On an OpenAI account whose quota ran out mid-service, the realtime session reconnect-loops forever while the caller hears dead air. From the production capture, every ~300ms cycle is:
error=APIError('OpenAI Realtime API returned an error',
body=RealtimeError(message='You exceeded your current quota, ...',
type='insufficient_quota', code='insufficient_quota'), retryable=True) recoverable=True
error=APIConnectionError('OpenAI S2S connection closed unexpectedly', body=None, retryable=True) recoverable=True
Two things combine to cause this, and neither is handled today:
The quota error is emitted with recoverable=True (the handlers optimistically mark everything recoverable), so nothing — including RealtimeModelFallbackAdapter, whose swap is triggered by recoverable=False — treats it as terminal.
The socket itself always reconnects fine (the key is valid; only generation fails, then the server closes). _main_task resets num_retries on every successful connect, so the give-up branch is unreachable and the loop never ends.
Meanwhile a pending generate_reply() sits out its full 10s timeout and fails with a generic "generate_reply timed out" that says nothing about quota.
Changes (OpenAI plugin only)
_FATAL_ERROR_CODES (insufficient_quota, invalid_api_key, account_deactivated, billing_hard_limit_reached) — codes that can never succeed on retry — classified where errors are parsed: _handle_error (server error events) and _handle_response_done_but_not_complete (failed response.done). These now emit recoverable=False instead of the hardcoded True; everything else keeps the optimistic default.
When a fatal code is seen, the session records it and _main_task stops reconnecting (raises the fatal error instead of retrying), ending the infinite loop. No retry-budget heuristics needed — the fatal error event arrives on every cycle, so classification alone is sufficient.
Pending generate_reply futures fail immediately with the actual code ("openai returned a fatal error: insufficient_quota") instead of waiting out the timeout.
With recoverable=False now emitted correctly, the existing recovery path works as designed: RealtimeModelFallbackAdapter swaps to the next model (and re-issues the reply via regenerate_on_swap), or a bare session fails fast and loud instead of spinning silently.
Scoped out (per review)
Earlier revisions of this PR added a framework-level reconnect state machine and an automatic generate_reply retry in the turn loop; both were dropped — speech retry belongs to RealtimeModelFallbackAdapter / the application. Two small follow-up PRs planned:
Make the pending-reply drop in _reconnect() observable via SpeechHandle.exception() (building on #6304.
Graceful handling of the update_chat_ctx timeout inside generate_reply (today an unhandled RealtimeError).
Testing
8 new unit tests in tests/test_realtime/test_openai_realtime_model.py: fatal-code matcher, classification at both parse sites (fatal + transient), fail-fast of pending reply futures, transient errors left untouched, Cancellation failed early-return.
ruff format --check, ruff check, full mypy (agents + all plugins), and pytest tests/test_realtime --unit all green.
Cue voice E2E
sid_3cffb55e639eat head839dada2.insufficient_quota; the persisted QCLI event recordedretryable=false,recoverable=false,sessionClosed=true, andconnectionCount=1after a 3-second no-reconnect observation window.~/.cue-cli/sessions/sid_3cffb55e639e/.