feat(realtime): add OpenAI-compatible WebRTC voice endpoints#500
Merged
Conversation
Expose the GA realtime WebRTC surface behind REALTIME_ENABLED: - POST /v1/realtime/calls: model-routed SDP exchange accepting both GA shapes (application/sdp offer with ?model=, or multipart sdp + session JSON). Provider credentials are injected, the requested model is rewritten to the resolved provider model (aliases/virtual models work), and the SDP answer is relayed with a gateway-relative Location header. - POST /v1/realtime/client_secrets: ephemeral browser credentials routed by session.model (with the transcription-model fallback), gated by the same model-access, budget, and rate-limit rules as other endpoints. - GET /v1/realtime?call_id=...: sideband websocket attach to an existing call, routed via an in-memory call registry (explicit model+provider params as the stateless fallback). WebRTC media and events flow directly between client and provider, so the gateway attaches a best-effort sideband observer websocket to each call it creates and records usage per response.done (endpoint /v1/realtime/calls); gateway-relayed attaches for registry-known calls skip the usage tap to avoid double counting. Providers opt in via the new core.RealtimeCallProvider interface (OpenAI only for now). Swagger regen also picks up previously ungenerated /admin/rate-limits paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
Contributor
📝 WalkthroughWalkthroughAdds realtime call attach via ChangesRealtime signaling feature
Realtime and API documentation
Estimated code review effort: 4 (Complex) | ~75 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
…responses Pin the protocol-mandated Content-Type (application/sdp for call answers, application/json for client secrets) instead of echoing the upstream header, and set X-Content-Type-Options: nosniff on relayed bodies, so a misbehaving upstream cannot turn the relay into browser-rendered markup (CodeQL go/reflected-xss). Add tests for capability gating, malformed multipart bodies, unreachable upstreams, missing Location headers, and invalid client-secret JSON. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
cmd/gomodel/docs/docs.go (1)
4396-4522: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winMissing 429 response for rate-limited realtime endpoints.
Both
POST /v1/realtime/callsandPOST /v1/realtime/client_secretscallenforceRateLimitbefore forwarding to the provider (perinternal/server/realtime_webrtc_service.go), yet neither documents a429response, unlike sibling endpoints such as/v1/chat/completionsand/v1/embeddingswhich document 429 for the same gate.📝 Suggested fix
"responses": { "201": { "description": "SDP answer", ... }, + "429": { + "description": "Too Many Requests", + "schema": { + "$ref": "`#/definitions/core.OpenAIErrorEnvelope`" + } + }, "501": {(and similarly for
/v1/realtime/client_secrets)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@cmd/gomodel/docs/docs.go` around lines 4396 - 4522, Add a documented 429 response to both realtime OpenAPI operations because `enforceRateLimit` can reject them before provider forwarding. Update the Swagger entries for the `POST /v1/realtime/calls` and `POST /v1/realtime/client_secrets` operations in `docs.go` so they match sibling rate-limited endpoints like `chatCompletions` and `embeddings`, using the same `core.OpenAIErrorEnvelope` schema for the `429` response.docs/openapi.json (1)
6482-6570: 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy liftAdd request bodies for both realtime POST endpoints
/v1/realtime/callsand/v1/realtime/client_secretsstill have norequestBody, so client generators and the docs UI can’t model the SDP/multipart payload or the JSON session body. Add explicit body annotations to the handlers and regenerate the spec.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/openapi.json` around lines 6482 - 6570, The `/v1/realtime/calls` POST operation is missing an explicit request body, so update the handler annotations for the realtime create-call endpoint to declare the accepted `application/sdp` and multipart payloads (including the JSON session field), then regenerate `docs/openapi.json`. Also check the companion `/v1/realtime/client_secrets` POST handler and add its request body annotation too, using the existing realtime route symbols so the generated spec and client generators can model both payloads correctly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@cmd/gomodel/docs/docs.go`:
- Around line 4396-4461: The OpenAPI spec for the realtime call endpoint is out
of sync with the realtime handler’s actual media types. Update the
/v1/realtime/calls operation in docs.go so its consumes includes both
application/sdp and multipart/form-data, and its produces matches the SDP answer
returned by the handler instead of text/plain. Use the existing realtime call
definition to keep the request/response contract aligned for client generation
and validation.
In `@internal/realtime/registry_test.go`:
- Around line 60-79: Current coverage in TestCallRegistryEvictsAtCapacity only
checks inserting a brand-new call at capacity; add a focused case around
Registry.Register that re-registers an existing call ID when len(r.entries) is
already maxCalls. Verify the existing entry is updated in place and that no
unrelated call is evicted, so Lookup on a different preexisting ID still
succeeds after the second Register call.
In `@internal/realtime/registry.go`:
- Around line 51-64: The Register method in CallRegistry should avoid evicting
when re-registering an existing call ID at capacity. Check whether callID
already exists in r.entries after pruneLocked and before calling
evictSoonestLocked, and only evict when this is a new entry that would actually
increase the map size. Keep the overwrite path in Register unchanged so existing
entries are refreshed without removing an unrelated route.
In `@internal/server/handlers.go`:
- Around line 339-357: The Swagger annotations for RealtimeCalls only declare
multipart form data, but the endpoint also supports raw application/sdp offers
with the model query parameter. Update the doc comments on Handler.RealtimeCalls
to reflect both accepted request content types by adding the raw SDP media type
alongside mpfd, while keeping the existing description of the dual input paths
accurate.
In `@internal/server/realtime_service.go`:
- Around line 79-98: When realtime_service.go handles a successful
s.calls.Lookup(callID) in the attach path, make the registered call’s Model and
Provider fully override any client-supplied model/providerHint instead of only
backfilling empty values. Update the logic in the call_id lookup branch so later
checks in prepare and enforceRateLimit always use the registry values, and add a
test covering call_id with a conflicting model to verify it resolves to the
registered call’s model/provider.
In `@internal/server/realtime_webrtc_service.go`:
- Around line 193-201: The fallback in the realtime signaling request path is
using an unbounded client when s.httpClient is nil. Update the request logic in
the realtime WebRTC service method that calls client.Do so it never falls back
to http.DefaultClient; instead use a bounded default client consistent with
httpclient.NewDefaultHTTPClient(), or make the httpClient dependency required so
the nil path cannot happen.
---
Outside diff comments:
In `@cmd/gomodel/docs/docs.go`:
- Around line 4396-4522: Add a documented 429 response to both realtime OpenAPI
operations because `enforceRateLimit` can reject them before provider
forwarding. Update the Swagger entries for the `POST /v1/realtime/calls` and
`POST /v1/realtime/client_secrets` operations in `docs.go` so they match sibling
rate-limited endpoints like `chatCompletions` and `embeddings`, using the same
`core.OpenAIErrorEnvelope` schema for the `429` response.
In `@docs/openapi.json`:
- Around line 6482-6570: The `/v1/realtime/calls` POST operation is missing an
explicit request body, so update the handler annotations for the realtime
create-call endpoint to declare the accepted `application/sdp` and multipart
payloads (including the JSON session field), then regenerate
`docs/openapi.json`. Also check the companion `/v1/realtime/client_secrets` POST
handler and add its request body annotation too, using the existing realtime
route symbols so the generated spec and client generators can model both
payloads correctly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: ca216231-99d2-4ecb-9fed-dc7614800d70
📒 Files selected for processing (22)
.env.templateCLAUDE.mdcmd/gomodel/docs/docs.godocs/advanced/api-endpoints.mdxdocs/openapi.jsoninternal/core/endpoints.gointernal/core/realtime.gointernal/providers/openai/realtime.gointernal/providers/openai/realtime_test.gointernal/providers/realtime_url.gointernal/providers/realtime_url_test.gointernal/providers/router.gointernal/providers/router_realtime_test.gointernal/realtime/observer.gointernal/realtime/observer_test.gointernal/realtime/registry.gointernal/realtime/registry_test.gointernal/server/handlers.gointernal/server/http.gointernal/server/realtime_service.gointernal/server/realtime_webrtc_service.gointernal/server/realtime_webrtc_service_test.go
…nd spec - Attach by call_id: the registry route now overrides a client-supplied model/provider instead of only backfilling, so model-access and rate-limit checks cannot be steered toward a model the registered call is not running. - Call registry: re-registering an existing call id at capacity overwrites in place instead of evicting an unrelated entry. - Signaling requests never fall back to the unbounded http.DefaultClient; the nil-client path now uses the configured-timeout default client. - OpenAPI: declare both accepted call content types (application/sdp + multipart), the application/sdp answer, request bodies for both realtime POST endpoints, and 429 responses on the rate-limited realtime routes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
Addressed the two outside-diff findings in fd0395e as well: both realtime POST operations now document 429 responses (core.OpenAIErrorEnvelope, matching the sibling rate-limited endpoints, plus the GET /v1/realtime route which shares the same gate), and both declare request bodies — the SDP/multipart payload for /v1/realtime/calls and the JSON session body for /v1/realtime/client_secrets — so client generators can model the payloads. |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docs/openapi.json (1)
6533-6548: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winDrop the
application/json201 response fromPOST /v1/realtime/calls(docs/openapi.json:6533-6548).RealtimeCallsalways relays the SDP answer asapplication/sdp; keeping a JSON success arm misleads API consumers and codegen.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/openapi.json` around lines 6533 - 6548, The POST /v1/realtime/calls 201 response in the RealtimeCalls OpenAPI spec currently advertises both application/sdp and application/json, but it should only describe the SDP answer. Update the response definition in docs/openapi.json so the 201 success arm for RealtimeCalls removes the application/json content and leaves application/sdp as the only response format.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/openapi.json`:
- Around line 6654-6664: The client-secret request body schema is too generic as
an untyped object, so update the OpenAPI definition for the requestBody in the
client-secret endpoint to expose the known fields described in the text. Keep
session.model (and the nested transcription model if applicable) plus optional
expires_after as explicit, loosely typed properties, and allow
additionalProperties so the remaining payload can still pass through verbatim.
- Around line 6517-6532: The multipart request body for the handler is still
being documented as an opaque string, so the generated OpenAPI schema does not
expose the actual multipart fields. Update the request documentation in
internal/server/handlers.go for the mpfd handling so it describes the multipart
form explicitly, either by using separate formData parameters or a struct-backed
multipart schema, and make sure the request shape includes both the sdp and
session parts.
---
Outside diff comments:
In `@docs/openapi.json`:
- Around line 6533-6548: The POST /v1/realtime/calls 201 response in the
RealtimeCalls OpenAPI spec currently advertises both application/sdp and
application/json, but it should only describe the SDP answer. Update the
response definition in docs/openapi.json so the 201 success arm for
RealtimeCalls removes the application/json content and leaves application/sdp as
the only response format.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: d41aa3c6-4410-4d74-8b2a-de77c9c6f337
📒 Files selected for processing (8)
cmd/gomodel/docs/docs.godocs/openapi.jsoninternal/realtime/registry.gointernal/realtime/registry_test.gointernal/server/handlers.gointernal/server/realtime_service.gointernal/server/realtime_webrtc_service.gointernal/server/realtime_webrtc_service_test.go
xAI mirrors OpenAI's shape exactly, so calls, client secrets, and
call_id sideband attach derive from the base URL via the shared
helpers. Verified live: /v1/realtime/client_secrets mints ephemeral
secrets through the gateway; /v1/realtime/calls exists upstream but is
team-gated by xAI, so unauthorized accounts get the 403 relayed.
Azure OpenAI uses its GA v1 surface at
<resource>/openai/v1/realtime/{calls,client_secrets} (api-key header,
no api-version), with sideband attach on the same GA path; the model
query parameter selects the deployment.
Bailian is deliberately not wired: its WebRTC endpoint is
allowlist-only and provided per customer by sales (probes 404 on both
public hosts), and the documented flow returns no call id for sideband
observation. Z.ai has no WebRTC realtime; both keep websocket-only
support.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Main's realtime hardening (#497) added a ping/pong heartbeat to the websocket relay so silently dead peers tear sessions down. The WebRTC sideband observer has the same exposure — a dead provider connection left it blocked in Read until the 6h call TTL. The heartbeat loop is now shared: relay pings both peers, the observer pings its upstream, and a ping timeout ends the observation immediately. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@coderabbitai continue |
Contributor
|
✅ Action performedReview finished.
|
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
internal/providers/azure/realtime.go (1)
90-148: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winExtract the shared base-URL normalization to avoid drift between
realtimeURLandrealtimeGARoot.
realtimeGARoot(lines 90-110) and the unchangedrealtimeURL(lines 123-148) both parseresourceRootBaseURL, validateu.Host, and strip/openai/v1//openaisuffixes — differing only in the final scheme mapping, path suffix, and query params. Also, inrealtimeAttachURL(line 81) theTrimSuffix(u.Path, "/realtime") + "/realtime"is a no-op:realtimeGARootalready guarantees the path ends in/realtime, so this can be dropped.As per coding guidelines, "avoid adding abstractions until a repeated pattern clearly justifies them" — this duplication is now a repeated pattern worth consolidating so a future change to the path-stripping logic doesn't silently diverge between the GA and legacy realtime URL builders.
♻️ Proposed refactor
+// azureRealtimeBase parses and normalizes the configured base URL down to its +// resource root, stripping any existing /openai[/v1] or deployment sub-path. +func (p *Provider) azureRealtimeBase() (*url.URL, error) { + root := resourceRootBaseURL(p.GetBaseURL()) + u, err := url.Parse(root) + if err != nil || u.Host == "" { + return nil, core.NewInvalidRequestError("invalid azure realtime base url: "+root, err) + } + path := strings.TrimRight(u.Path, "/") + path = strings.TrimSuffix(path, "/openai/v1") + path = strings.TrimSuffix(path, "/openai") + u.Path = path + u.RawQuery = "" + return u, nil +} + func (p *Provider) realtimeGARoot() (*url.URL, error) { - root := resourceRootBaseURL(p.GetBaseURL()) - u, err := url.Parse(root) - if err != nil || u.Host == "" { - return nil, core.NewInvalidRequestError("invalid azure realtime base url: "+root, err) - } + u, err := p.azureRealtimeBase() + if err != nil { + return nil, err + } switch strings.ToLower(u.Scheme) { case "https", "wss", "": u.Scheme = "https" case "http", "ws": u.Scheme = "http" default: return nil, core.NewInvalidRequestError("unsupported azure realtime base url scheme: "+u.Scheme, nil) } - path := strings.TrimRight(u.Path, "/") - path = strings.TrimSuffix(path, "/openai/v1") - path = strings.TrimSuffix(path, "/openai") - u.Path = path + "/openai/v1/realtime" - u.RawQuery = "" + u.Path += "/openai/v1/realtime" return u, nil }func (p *Provider) realtimeAttachURL(callID string) (string, error) { u, err := p.realtimeGARoot() if err != nil { return "", err } switch strings.ToLower(u.Scheme) { case "https": u.Scheme = "wss" case "http": u.Scheme = "ws" } - u.Path = strings.TrimSuffix(u.Path, "/realtime") + "/realtime" q := url.Values{} q.Set("call_id", callID) u.RawQuery = q.Encode() return u.String(), nil }
realtimeURL(line 123) can similarly delegate toazureRealtimeBase()for the parse/strip step.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/providers/azure/realtime.go` around lines 90 - 148, The Azure realtime URL builders duplicate the same base-URL parsing and path normalization logic, which can drift over time. Extract the shared `resourceRootBaseURL`/`url.Parse`/host validation and `/openai[/v1]` stripping into a common helper used by both `realtimeGARoot` and `realtimeURL`, keeping only the scheme mapping, final path suffix, and query assembly different. Also simplify `realtimeAttachURL` by removing the redundant `/realtime` trim-then-append since `realtimeGARoot` already returns a `/realtime` path.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@internal/providers/azure/realtime.go`:
- Around line 90-148: The Azure realtime URL builders duplicate the same
base-URL parsing and path normalization logic, which can drift over time.
Extract the shared `resourceRootBaseURL`/`url.Parse`/host validation and
`/openai[/v1]` stripping into a common helper used by both `realtimeGARoot` and
`realtimeURL`, keeping only the scheme mapping, final path suffix, and query
assembly different. Also simplify `realtimeAttachURL` by removing the redundant
`/realtime` trim-then-append since `realtimeGARoot` already returns a
`/realtime` path.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 632b82a4-b6e2-4bd2-8b45-e30443cb9871
📒 Files selected for processing (12)
.env.templateCLAUDE.mdcmd/gomodel/docs/docs.godocs/openapi.jsoninternal/providers/azure/realtime.gointernal/providers/azure/realtime_test.gointernal/providers/xai/realtime.gointernal/providers/xai/realtime_test.gointernal/realtime/observer.gointernal/realtime/observer_test.gointernal/realtime/proxy.gointernal/server/handlers.go
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
internal/providers/azure/realtime.go (1)
56-96: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winConsider consolidating the three near-identical URL builders.
realtimeHTTPTarget,realtimeAttachURL, andrealtimeURLall follow the same pattern: get root viarealtimeRoot, append a path, optionally set query params, return. Extracting a small private helper (e.g.,buildRealtimeURL(insecure, secure, path string, query url.Values)) would reduce repetition across these three call sites.♻️ Example consolidation
-func (p *Provider) realtimeHTTPTarget(req *core.RealtimeRequest, endpoint string) (*core.RealtimeHTTPTarget, error) { - if req == nil || strings.TrimSpace(req.Model) == "" { - return nil, core.NewInvalidRequestError("model is required for realtime calls", nil) - } - u, err := p.realtimeRoot("http", "https") - if err != nil { - return nil, err - } - u.Path += "/openai/v1/realtime/" + endpoint - return &core.RealtimeHTTPTarget{URL: u.String(), Headers: p.realtimeAuthHeaders()}, nil -} - -func (p *Provider) realtimeAttachURL(callID string) (string, error) { - u, err := p.realtimeRoot("ws", "wss") - if err != nil { - return "", err - } - u.Path += "/openai/v1/realtime" - q := url.Values{} - q.Set("call_id", callID) - u.RawQuery = q.Encode() - return u.String(), nil -} +func (p *Provider) buildRealtimeURL(insecure, secure, path string, query url.Values) (*url.URL, error) { + u, err := p.realtimeRoot(insecure, secure) + if err != nil { + return nil, err + } + u.Path += path + if query != nil { + u.RawQuery = query.Encode() + } + return u, nil +} + +func (p *Provider) realtimeHTTPTarget(req *core.RealtimeRequest, endpoint string) (*core.RealtimeHTTPTarget, error) { + if req == nil || strings.TrimSpace(req.Model) == "" { + return nil, core.NewInvalidRequestError("model is required for realtime calls", nil) + } + u, err := p.buildRealtimeURL("http", "https", "/openai/v1/realtime/"+endpoint, nil) + if err != nil { + return nil, err + } + return &core.RealtimeHTTPTarget{URL: u.String(), Headers: p.realtimeAuthHeaders()}, nil +} + +func (p *Provider) realtimeAttachURL(callID string) (string, error) { + q := url.Values{} + q.Set("call_id", callID) + u, err := p.buildRealtimeURL("ws", "wss", "/openai/v1/realtime", q) + if err != nil { + return "", err + } + return u.String(), nil +}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/providers/azure/realtime.go` around lines 56 - 96, The three realtime URL builders in Provider are duplicating the same root-path-query assembly logic. Extract a shared private helper around realtimeRoot that accepts the path and optional query values, then update realtimeHTTPTarget, realtimeAttachURL, and realtimeURL to call it so the URL construction is centralized and easier to maintain.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@internal/providers/azure/realtime.go`:
- Around line 56-96: The three realtime URL builders in Provider are duplicating
the same root-path-query assembly logic. Extract a shared private helper around
realtimeRoot that accepts the path and optional query values, then update
realtimeHTTPTarget, realtimeAttachURL, and realtimeURL to call it so the URL
construction is centralized and easier to maintain.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 6260647c-f74d-43bf-a368-beb5870c13c8
📒 Files selected for processing (1)
internal/providers/azure/realtime.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the OpenAI GA realtime WebRTC surface to the gateway, completing the voice story started with the websocket proxy. All routes are gated by the existing
REALTIME_ENABLED(default true) — no new configuration.POST /v1/realtime/callsapplication/sdpoffer with?model=, or multipartsdp+session(JSON) fields. Relays the SDP answer and a gateway-relativeLocation: /v1/realtime/calls/{call_id}.POST /v1/realtime/client_secretssession.model(transcription-model fallback), relayed verbatim.GET /v1/realtime?call_id=…model+providerparams work statelessly otherwise.User-visible impact
response.done(endpoint/v1/realtime/calls, audio/text token details and cost priced as usual). Gateway-relayed sideband attaches for registry-known calls skip the usage tap to avoid double counting.client_secretsauthenticate directly against the provider and bypass the gateway (inherent to the ephemeral-key flow; documented).Provider-specific behavior
core.RealtimeCallProviderinterface (mirrorsRealtimeProvider), implemented by OpenAI, xAI, and Azure OpenAI:client_secretsmints ephemeral secrets through the gateway; thecallsendpoint exists upstream but xAI gates WebRTC per team, so unauthorized accounts get the upstream 403 relayed.<resource>/openai/v1/realtime/{calls,client_secrets},api-keyheader, no api-version); unit-tested against the documented shape (no Azure key available for a live check)./p/openai/v1/realtime/calls,…/client_secrets) already relays these endpoints opaquely — verified, no changes needed.Verification
call_id(101 + frame relay), observer-recorded usage rows in SQLite with exact audio-rate cost math, 401/404 negative cases, passthrough parity.Note: the swagger regen also picks up previously ungenerated
/admin/rate-limits*paths (pre-existing drift).Possible follow-ups (out of scope): typed call-control proxies (
hangupetc. — available via passthrough today), WebRTC for additional providers.🤖 Generated with Claude Code
Summary by CodeRabbit
call_id.POST /v1/realtime/callsandPOST /v1/realtime/client_secrets.modelrequirements, supported query parameters, and additional response/error codes.