feat(audio): add OpenAI-compatible text-to-speech and transcription endpoints#362
Merged
Conversation
…ndpoints Add first-class /v1/audio/speech and /v1/audio/transcriptions endpoints following the existing optional-interface + thin-service pattern used by files and batches. - core: AudioSpeechRequest/AudioTranscriptionRequest/AudioResponse types, a JSON decode helper, and response content-type derivation; new optional AudioProvider interface - openai: CreateSpeech (JSON -> binary via DoRaw) and CreateTranscription (streamed multipart -> bytes) on CompatibleProvider, inherited by the OpenAI-compatible providers - router: model-routed CreateSpeech/CreateTranscription dispatch via a shared routeAudioCall helper that type-asserts AudioProvider, with an unsupported-model guard - server: thin audioService (validate -> authorize -> budget -> route -> blob); authorization runs on the registry-resolved selector so model override and user-path rules match the inference orchestrator - llmclient: report the model for audio speech requests in metrics hooks Audio bypasses the inference orchestrator (no fallback, guardrails, response cache, or usage metering), matching the files/passthrough path. Tests: core content-type/decode units, OpenAI contract replay tests for both endpoints, and service-layer tests covering validation, resolved- selector authorization, and authorizer denial. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
📝 WalkthroughWalkthroughThis PR adds OpenAI-compatible text-to-speech and speech-to-text audio endpoints, including core types and helpers, an optional AudioProvider interface and OpenAI provider implementation, router integration, HTTP handlers with authorization/budgeting, unit/contract/e2e tests, and documentation/OpenAPI updates. ChangesAudio API Support
Sequence Diagram(s)sequenceDiagram
participant Client
participant Handler
participant audioService
participant Router
participant Authorizer
participant Provider
Client->>Handler: POST /v1/audio/speech or /v1/audio/transcriptions
Handler->>audioService: CreateSpeech / CreateTranscription
audioService->>Router: resolve provider & concrete model
audioService->>Authorizer: authorize resolved selector
audioService->>Provider: CreateSpeech / CreateTranscription
Provider->>audioService: AudioResponse (Data + ContentType)
audioService->>Client: HTTP blob with Content-Type
🎯 4 (Complex) | ⏱️ ~45 minutes
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryThis PR adds OpenAI-compatible audio support to the gateway. It changes:
Confidence Score: 5/5This looks safe to merge.
Reviews (2): Last reviewed commit: "test(e2e): add audio scenarios S110-S114..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+29
to
+33
| raw, err := p.client.DoRaw(ctx, p.prepareRequest(llmclient.Request{ | ||
| Method: http.MethodPost, | ||
| Endpoint: "/audio/speech", | ||
| Body: req, | ||
| })) |
There was a problem hiding this comment.
CompatibleProvider is embedded by the Azure provider, so Azure now implements audio using these OpenAI paths. A normal Azure OpenAI resource expects deployment-scoped audio URLs such as /openai/deployments/{deployment}/audio/... with api-version; this code sends /audio/speech with only the request mutator applied. Routing an Azure audio model through the new endpoint will hit the wrong upstream path and return 404 instead of creating speech.
|
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
internal/server/audio_service_test.go (1)
56-252: 🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick winAdd tests for uncovered audio response normalization/error branches.
Please add cases for: (1) provider returns
nilresponse →502, (2) emptyContentType→ defaults toapplication/octet-stream, and (3) missing multipartfileon transcription →400. These are core branches in the new service behavior.As per coding guidelines,
**/*_test.go: “Add or update tests for behavior changes to cover … response normalization, error handling, default configuration …”.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/server/audio_service_test.go` around lines 56 - 252, Add three unit tests to cover audio response normalization and error branches: (1) for speech, add a test using audioMockProvider where speechResp is nil and assert handler.AudioSpeech (or svc.CreateSpeech) returns HTTP 502 and provider is not nil; (2) for speech, add a test where speechResp.ContentType == "" and assert the response Content-Type defaults to "application/octet-stream" after calling handler.AudioSpeech; (3) for transcriptions, add a test where the multipart form omits the "file" part and assert handler.AudioTranscriptions returns HTTP 400; use the existing test helpers/fixtures (audioMockProvider, mockProvider, NewHandler, audioService, recordingModelAuthorizer) and assert on rec.Code, rec.Header().Get("Content-Type"), and that mock.captured... fields remain nil where appropriate.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@internal/server/audio_service.go`:
- Around line 141-143: The handler currently only reads
form.Value["timestamp_granularities[]"] which rejects clients that send
"timestamp_granularities" without brackets; update the MultipartForm handling
(where c.MultipartForm() is called and granularities is set) to accept both keys
by checking form.Value for "timestamp_granularities[]" and, if absent or empty,
also for "timestamp_granularities", merging or choosing the non-empty value so
granularities contains values from either variant before further processing.
- Around line 100-103: In prepare (around the s.provider.(selectorResolver)
block) don't overwrite selector unless
resolver.ResolveModel(core.NewRequestedModelSelector(model, providerHint))
returns found == true and resolveErr == nil; if resolveErr != nil return or
propagate that error instead of silently ignoring it, and if found == false keep
the original selector (or handle as a not-found error) to avoid authorizing
against the wrong selector; update the logic that currently assigns selector =
resolved to only run when both found is true and resolveErr is nil, and surface
resolveErr when present.
In `@internal/server/handlers.go`:
- Around line 483-492: The OpenAPI annotations for the AudioTranscriptions
handler incorrectly advertise JSON-only output; update the swagger comments
around the AudioTranscriptions route handler (the annotations above the
AudioTranscriptions function in handlers.go) to list all produced content types
(e.g., application/json, text/plain, application/x-subrip, text/vtt) and add
corresponding `@Success` entries or use a generic `@Success` 200 {string} for
non-JSON response_format options (json remains {object} map[string]interface{});
ensure the `@Produce/`@Success annotations reflect each response_format option so
generated clients know when they will receive plain text/SRT/VTT vs JSON.
In `@tests/contract/openai_audio_test.go`:
- Around line 24-59: Extend the tests by adding table-driven cases (similar to
TestOpenAIReplayCreateSpeech) that exercise the transcription request
translation and provider-specific mapping: call audio.CreateSpeech and the
transcription method (the function under test that sends transcription requests)
with core.AudioSpeechRequest and the transcription request struct containing
fields instructions, speed, language, prompt, temperature, and
timestamp_granularities; mock the outbound HTTP replay (look for
replayKey(http.MethodPost, "/audio/speech") and the transcription POST route) to
capture the outgoing payload and assert that the provider-bound JSON/body
contains the expected translated keys/values (including defaults and format
mapping paths), verifying both presence and correct mapping for
provider-specific fields and default handling. Ensure tests are table-driven,
include cases for empty/default values and explicit values, and assert
translation logic for CreateSpeech, transcription request translation functions,
and any format->ContentType mapping paths.
---
Outside diff comments:
In `@internal/server/audio_service_test.go`:
- Around line 56-252: Add three unit tests to cover audio response normalization
and error branches: (1) for speech, add a test using audioMockProvider where
speechResp is nil and assert handler.AudioSpeech (or svc.CreateSpeech) returns
HTTP 502 and provider is not nil; (2) for speech, add a test where
speechResp.ContentType == "" and assert the response Content-Type defaults to
"application/octet-stream" after calling handler.AudioSpeech; (3) for
transcriptions, add a test where the multipart form omits the "file" part and
assert handler.AudioTranscriptions returns HTTP 400; use the existing test
helpers/fixtures (audioMockProvider, mockProvider, NewHandler, audioService,
recordingModelAuthorizer) and assert on rec.Code,
rec.Header().Get("Content-Type"), and that mock.captured... fields remain nil
where appropriate.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 541f0bd8-7769-4b30-8337-21433c9ec720
📒 Files selected for processing (11)
internal/core/audio.gointernal/core/audio_test.gointernal/core/interfaces.gointernal/llmclient/client.gointernal/providers/openai/audio.gointernal/providers/router.gointernal/server/audio_service.gointernal/server/audio_service_test.gointernal/server/handlers.gointernal/server/http.gotests/contract/openai_audio_test.go
Comment on lines
+100
to
+103
| if resolver, ok := s.provider.(selectorResolver); ok { | ||
| if resolved, _, resolveErr := resolver.ResolveModel(core.NewRequestedModelSelector(model, providerHint)); resolveErr == nil { | ||
| selector = resolved | ||
| } |
Contributor
There was a problem hiding this comment.
Respect ResolveModel’s found flag and error before authorizing.
prepare currently overwrites selector even when found == false, and it silently ignores ResolveModel errors. That can authorize against the wrong selector and hide resolution failures.
🔧 Suggested fix
if resolver, ok := s.provider.(selectorResolver); ok {
- if resolved, _, resolveErr := resolver.ResolveModel(core.NewRequestedModelSelector(model, providerHint)); resolveErr == nil {
- selector = resolved
- }
+ resolved, found, resolveErr := resolver.ResolveModel(core.NewRequestedModelSelector(model, providerHint))
+ if resolveErr != nil {
+ return nil, resolveErr
+ }
+ if found {
+ selector = resolved
+ }
}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@internal/server/audio_service.go` around lines 100 - 103, In prepare (around
the s.provider.(selectorResolver) block) don't overwrite selector unless
resolver.ResolveModel(core.NewRequestedModelSelector(model, providerHint))
returns found == true and resolveErr == nil; if resolveErr != nil return or
propagate that error instead of silently ignoring it, and if found == false keep
the original selector (or handle as a not-found error) to avoid authorizing
against the wrong selector; update the logic that currently assigns selector =
resolved to only run when both found is true and resolveErr is nil, and surface
resolveErr when present.
Comment on lines
+483
to
+492
| // @Produce json | ||
| // @Security BearerAuth | ||
| // @Param file formData file true "Audio file to transcribe" | ||
| // @Param model formData string true "Model ID" | ||
| // @Param language formData string false "Input language (ISO-639-1)" | ||
| // @Param prompt formData string false "Optional text to guide the model" | ||
| // @Param response_format formData string false "json, text, srt, verbose_json, or vtt" | ||
| // @Param temperature formData string false "Sampling temperature (0-1)" | ||
| // @Success 200 {object} map[string]interface{} "Transcription in the requested response_format" | ||
| // @Failure 400 {object} core.OpenAIErrorEnvelope |
Contributor
There was a problem hiding this comment.
Document non-JSON transcription outputs in OpenAPI annotations.
AudioTranscriptions can return text formats, but annotations currently advertise JSON-only output and object schema. This can break generated clients and contract expectations.
🔧 Suggested fix
// `@Accept` mpfd
// `@Produce` json
+// `@Produce` text/plain
+// `@Produce` application/octet-stream
// `@Security` BearerAuth
@@
-// `@Success` 200 {object} map[string]interface{} "Transcription in the requested response_format"
+// `@Success` 200 {file} file "Transcription payload in the requested response_format"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| // @Produce json | |
| // @Security BearerAuth | |
| // @Param file formData file true "Audio file to transcribe" | |
| // @Param model formData string true "Model ID" | |
| // @Param language formData string false "Input language (ISO-639-1)" | |
| // @Param prompt formData string false "Optional text to guide the model" | |
| // @Param response_format formData string false "json, text, srt, verbose_json, or vtt" | |
| // @Param temperature formData string false "Sampling temperature (0-1)" | |
| // @Success 200 {object} map[string]interface{} "Transcription in the requested response_format" | |
| // @Failure 400 {object} core.OpenAIErrorEnvelope | |
| // `@Produce` json | |
| // `@Produce` text/plain | |
| // `@Produce` application/octet-stream | |
| // `@Security` BearerAuth | |
| // `@Param` file formData file true "Audio file to transcribe" | |
| // `@Param` model formData string true "Model ID" | |
| // `@Param` language formData string false "Input language (ISO-639-1)" | |
| // `@Param` prompt formData string false "Optional text to guide the model" | |
| // `@Param` response_format formData string false "json, text, srt, verbose_json, or vtt" | |
| // `@Param` temperature formData string false "Sampling temperature (0-1)" | |
| // `@Success` 200 {file} file "Transcription payload in the requested response_format" | |
| // `@Failure` 400 {object} core.OpenAIErrorEnvelope |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@internal/server/handlers.go` around lines 483 - 492, The OpenAPI annotations
for the AudioTranscriptions handler incorrectly advertise JSON-only output;
update the swagger comments around the AudioTranscriptions route handler (the
annotations above the AudioTranscriptions function in handlers.go) to list all
produced content types (e.g., application/json, text/plain,
application/x-subrip, text/vtt) and add corresponding `@Success` entries or use a
generic `@Success` 200 {string} for non-JSON response_format options (json remains
{object} map[string]interface{}); ensure the `@Produce/`@Success annotations
reflect each response_format option so generated clients know when they will
receive plain text/SRT/VTT vs JSON.
- propagate ResolveModel errors in the audio service instead of authorizing an unresolved selector when the registry is not ready or the selector is malformed (matches the inference orchestrator) - accept the unbracketed timestamp_granularities form field in addition to the canonical timestamp_granularities[]; still forwarded bracketed upstream - annotate AudioTranscriptions as also producing text/plain (text/srt/vtt) - add service-layer tests for the nil-response 502, empty content-type default, and missing-file 400 branches - add contract tests asserting outbound request translation for speech (instructions/speed/voice/format) and transcription multipart fields (language/prompt/temperature/timestamp_granularities[] + file part) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- add release E2E curl scenarios for the audio endpoints: TTS binary output, STT round-trips (json and text formats), and the missing-voice (400) and unknown-model (404) negative paths; each is self-contained - remove tests/e2e/release-e2e-findings.md, a point-in-time run report whose scenarios and bug fix are already codified in the matrix and tests Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
internal/server/audio_service.go (1)
141-144: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick win
io.ReadAll(file)isn’t unbounded: globalBodyLimitapplies to this route
POST /v1/audio/transcriptionsis registered under the server that installs Echo’s globalmiddleware.BodyLimit(parseBodySizeLimitBytes(cfg.BodySizeLimit)), so oversized multipart requests should be rejected before the handler reads the upload.- If the PR’s “25MB audio” requirement must be enforced specifically for the uploaded audio part, add a per-part size bound/validation in
transcriptionRequestFromForminstead of relying only on the server-wide request limit.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/server/audio_service.go` around lines 141 - 144, The handler currently calls io.ReadAll(file) which relies on the global Echo BodyLimit middleware but does not enforce a per-file size cap; update transcriptionRequestFromForm to validate the uploaded audio part's size (enforce the PR's 25MB audio limit) before calling io.ReadAll by reading the part with a size bound (e.g., via a LimitedReader or checking the part's header/size) and return a clear error if the part exceeds the per-file limit; keep existing global middleware but ensure the per-part validation occurs in transcriptionRequestFromForm and reference the uploaded file handling path where io.ReadAll(file) is used.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@internal/server/audio_service.go`:
- Around line 141-144: The handler currently calls io.ReadAll(file) which relies
on the global Echo BodyLimit middleware but does not enforce a per-file size
cap; update transcriptionRequestFromForm to validate the uploaded audio part's
size (enforce the PR's 25MB audio limit) before calling io.ReadAll by reading
the part with a size bound (e.g., via a LimitedReader or checking the part's
header/size) and return a clear error if the part exceeds the per-file limit;
keep existing global middleware but ensure the per-part validation occurs in
transcriptionRequestFromForm and reference the uploaded file handling path where
io.ReadAll(file) is used.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 13b96906-d1ab-43ba-9d95-9bfa3b8c52d2
📒 Files selected for processing (6)
internal/server/audio_service.gointernal/server/audio_service_test.gointernal/server/handlers.gotests/contract/openai_audio_test.gotests/e2e/release-e2e-findings.mdtests/e2e/release-e2e-scenarios.md
💤 Files with no reviewable changes (1)
- tests/e2e/release-e2e-findings.md
- add docs/advanced/audio-api.mdx covering the OpenAI-compatible /v1/audio/speech and /v1/audio/transcriptions endpoints, with examples, supported parameters, and the orchestrator-bypass limitations - add the page to the Advanced group in docs.json navigation - regenerate the swag spec (cmd/gomodel/docs/docs.go) and docs/openapi.json from the handler annotations so the audio routes render in the API Reference, including the text/plain transcription response Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/openapi.json`:
- Around line 3562-3575: The OpenAPI response schema for the transcription
endpoint incorrectly models "text/plain" as an object; update the "content"
entry for "text/plain" (alongside any "text/srt" and "text/vtt" entries) so its
"schema" has "type": "string" (remove "additionalProperties": true), while
leaving "application/json" as an object for json/verbose_json; ensure the
"content" block for text responses reflects plain-text string bodies rather than
JSON objects.
- Around line 6929-6955: Update the OpenAPI schema for core.AudioSpeechRequest
to mark the TTS-required fields as required by adding a "required" array that
lists "model", "input", and "voice"; locate the core.AudioSpeechRequest object
in the openapi.json and add the required property alongside its existing "type"
and "properties" so generated docs/SDKs and validation will treat model, input,
and voice as mandatory for AudioSpeechRequest.
- Around line 3524-3549: The multipart transcription schema is missing the
timestamp_granularities field and mis-typed temperature as a string; add a new
property "timestamp_granularities" (type: array of strings, with an items schema
and a description) and change "temperature" from "type: string" to a numeric
type (e.g., "type: number" with "format": "float" and appropriate bounds) in the
same schema block that currently defines "file", "model", "language", "prompt",
"response_format", and "temperature" so generated clients can send the full
supported request shape.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 4965ef2d-f5fc-4e61-b03a-22be0ba933ad
📒 Files selected for processing (4)
cmd/gomodel/docs/docs.godocs/advanced/audio-api.mdxdocs/docs.jsondocs/openapi.json
Comment on lines
+3524
to
+3549
| "properties": { | ||
| "file": { | ||
| "description": "Audio file to transcribe", | ||
| "type": "string", | ||
| "format": "binary" | ||
| }, | ||
| "model": { | ||
| "description": "Model ID", | ||
| "type": "string" | ||
| }, | ||
| "language": { | ||
| "description": "Input language (ISO-639-1)", | ||
| "type": "string" | ||
| }, | ||
| "prompt": { | ||
| "description": "Optional text to guide the model", | ||
| "type": "string" | ||
| }, | ||
| "response_format": { | ||
| "description": "json, text, srt, verbose_json, or vtt", | ||
| "type": "string" | ||
| }, | ||
| "temperature": { | ||
| "description": "Sampling temperature (0-1)", | ||
| "type": "string" | ||
| } |
Contributor
There was a problem hiding this comment.
Fix the transcription multipart schema to match the supported request contract.
This schema drops timestamp_granularities entirely and types temperature as a string, so generated clients cannot represent the full request shape the endpoint is supposed to accept.
Suggested OpenAPI fix
"response_format": {
"description": "json, text, srt, verbose_json, or vtt",
"type": "string"
},
"temperature": {
"description": "Sampling temperature (0-1)",
- "type": "string"
+ "type": "number"
+ },
+ "timestamp_granularities": {
+ "description": "Timestamp granularity options for verbose_json responses",
+ "type": "array",
+ "items": {
+ "type": "string"
+ }
}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/openapi.json` around lines 3524 - 3549, The multipart transcription
schema is missing the timestamp_granularities field and mis-typed temperature as
a string; add a new property "timestamp_granularities" (type: array of strings,
with an items schema and a description) and change "temperature" from "type:
string" to a numeric type (e.g., "type: number" with "format": "float" and
appropriate bounds) in the same schema block that currently defines "file",
"model", "language", "prompt", "response_format", and "temperature" so generated
clients can send the full supported request shape.
Address review feedback on the generated spec (fixed at the source so it survives regeneration): - transcription: document the timestamp_granularities[] form field and type temperature as a number (handlers.go @Param annotations) - transcription: model the text/plain response (text/srt/vtt) as a string rather than an object, via an openapi-postprocess patch; application/json stays an object - speech: mark model, input, and voice as required on core.AudioSpeechRequest via ensureRequiredProperty Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
cmd/gomodel/docs/docs.go (1)
4866-4892:⚠️ Potential issue | 🟠 Major | ⚡ Quick winAdd the missing required fields to
core.AudioSpeechRequest.The published schema still treats
model,input, andvoiceas optional. That drifts from the endpoint contract and will cause generated clients/docs to accept invalid payloads that the server rejects. Please regenerate this spec after fixing the source schema/post-processing so those properties appear underrequired.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@cmd/gomodel/docs/docs.go` around lines 4866 - 4892, The OpenAPI schema for core.AudioSpeechRequest is missing required entries for the fields model, input, and voice; update the source schema or post-processing that builds core.AudioSpeechRequest so that "required": ["model","input","voice"] is added and those properties are marked required, then regenerate the docs/spec; locate the schema builder or struct-to-schema mapper that emits core.AudioSpeechRequest (search for AudioSpeechRequest, model, input, voice) and modify the schema generation logic to include these three fields as required before running the spec generation step.
♻️ Duplicate comments (1)
internal/server/handlers.go (1)
490-493:⚠️ Potential issue | 🟠 Major | ⚡ Quick winModel non-JSON transcription outputs as strings, not objects.
This annotation still tells the generator that every 200 response is an object, but
text,srt, andvttreturn raw text bodies. Generated clients will deserialize those responses against the wrong schema. Please split the success schema so JSON formats stay object-shaped and non-JSON formats are documented as strings.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/server/handlers.go` around lines 490 - 493, The OpenAPI comment wrongly documents all 200 responses as objects; update the Swagger annotations around the transcription handler so JSON formats remain an object and non-JSON formats are documented as strings: keep the existing "`@Success` 200 {object} map[string]interface{}" for response_format values "json" and "verbose_json", and add a separate "`@Success` 200 {string} string" entry (with a descriptive message) for response_format values "text", "srt", and "vtt"; reference the existing annotations mentioning response_format, timestamp_granularities[], and map[string]interface{} to locate and split the success schema accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@cmd/gomodel/docs/docs.go`:
- Around line 4866-4892: The OpenAPI schema for core.AudioSpeechRequest is
missing required entries for the fields model, input, and voice; update the
source schema or post-processing that builds core.AudioSpeechRequest so that
"required": ["model","input","voice"] is added and those properties are marked
required, then regenerate the docs/spec; locate the schema builder or
struct-to-schema mapper that emits core.AudioSpeechRequest (search for
AudioSpeechRequest, model, input, voice) and modify the schema generation logic
to include these three fields as required before running the spec generation
step.
---
Duplicate comments:
In `@internal/server/handlers.go`:
- Around line 490-493: The OpenAPI comment wrongly documents all 200 responses
as objects; update the Swagger annotations around the transcription handler so
JSON formats remain an object and non-JSON formats are documented as strings:
keep the existing "`@Success` 200 {object} map[string]interface{}" for
response_format values "json" and "verbose_json", and add a separate "`@Success`
200 {string} string" entry (with a descriptive message) for response_format
values "text", "srt", and "vtt"; reference the existing annotations mentioning
response_format, timestamp_granularities[], and map[string]interface{} to locate
and split the success schema accordingly.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 3b6e4250-20d2-4150-8734-18761ff728a3
📒 Files selected for processing (4)
cmd/gomodel/docs/docs.godocs/openapi.jsoninternal/server/handlers.gotools/openapi-postprocess.mjs
This was referenced Jun 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds two first-class OpenAI-compatible audio endpoints to the gateway:
POST /v1/audio/speech— text-to-speech (JSON in → binary audio out)POST /v1/audio/transcriptions— speech-to-text (multipart upload → JSON/text out)These follow the existing optional-interface + thin-service pattern already used for files and batches, so providers without audio support simply don't implement the interface, and the OpenAI-compatible providers (openrouter, zai, vllm, oracle, azure, minimax) inherit it for free.
User-visible impact
response_format).modelselection andproviderhints work exactly as they do for chat/embeddings.Implementation notes
coreAudioSpeechRequest/AudioTranscriptionRequest/AudioResponsetypes, JSON decode helper, response content-type derivation, optionalAudioProviderinterfaceproviders/openaiCreateSpeech(JSON → binary viaDoRaw) andCreateTranscription(streamed multipart → bytes) onCompatibleProviderproviders(router)routeAudioCallhelper that type-assertsAudioProvider, with an unsupported-model guardserveraudioService: validate → authorize → budget → route → blob. Authorization runs on the registry-resolved selector so model-override and user-path rules match the inference orchestratorllmclientextractModelreports the model for audio speech requests in metrics hooksScope / deliberate limitations
Tests
-tags=contract): OpenAI replay tests for speech (default/wav/opus), speech validation, transcription (json/text), and upstream-error propagationAll pre-commit hooks pass (
test-race,lint,gofmt, perf guard).🤖 Generated with Claude Code
Summary by CodeRabbit