fix(openai): explicitly send tool_choice=auto when tools are provided#2813
Merged
Conversation
Some strict OpenAI-compatible gateways like LiteLLM reject requests where tool_choice is missing while tools are present, even though the OpenAI spec treats omission as equivalent to auto. This fix sends the default value explicitly to maintain compatibility. Fixes docker#2804
docker-agent
left a comment
docker-agent
left a comment
There was a problem hiding this comment.
Assessment: 🟢 APPROVE
The fix correctly addresses the LiteLLM/Bedrock gateway incompatibility by explicitly setting tool_choice: "auto" whenever tools are present — in both the Chat Completions and Responses API paths. The implementation is minimal, spec-compliant, and backwards-compatible.
Summary of analysis:
- Both code paths (
CreateChatCompletionStreamandCreateResponseStream) guard thetool_choicefield behindlen(tools) > 0, which is correct. - The SDK union param usage (
ChatCompletionToolChoiceOptionUnionParam{OfAuto: param.NewOpt("auto")}) is confirmed correct by the wire-format test that captures and deserializes the actual HTTP request body. - The Responses API path uses
responses.ToolChoiceOptionsAutoconstant — the correct typed approach. - Tests cover: tools present (verifies
tool_choiceis on the wire as bare string"auto"), and tools absent (verifiestool_choiceis omitted). - No high or medium severity issues found.
rumpl
previously approved these changes
May 18, 2026
Existing VCR cassettes were recorded before tool_choice was sent explicitly. Normalize the field in the matcher so old cassettes keep matching, matching the existing pattern used for max_tokens, reasoning, and thinkingConfig.
rumpl
approved these changes
May 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #2804.
The OpenAI spec treats an omitted
tool_choiceas equivalent to"auto", but strict OpenAI-compatible gateways (notably LiteLLM-based proxies) reject requests with a 400 when the field is absent:This change sends
tool_choice: "auto"explicitly in both the Chat Completions and Responses API paths whenevertoolsis non-empty. The behavior is:autopreserves the model's autonomy to decide when to call tools.When no tools are provided,
tool_choiceis correctly omitted (sending it with notoolswould be invalid).Changes
pkg/model/provider/openai/client.go— setToolChoiceexplicitly in bothCreateChatCompletionStreamandCreateResponseStreamwhen tools are present.pkg/model/provider/openai/tool_choice_test.go— verifies the Chat Completions wire format.pkg/model/provider/openai/tool_choice_responses_test.go— verifies the Responses API wire format.Tests assert the actual JSON request body (not just SDK types) to confirm the field is on the wire.
Validation
go build ./...✅go vet ./...✅go test ./pkg/model/provider/openai/...✅golangci-lint run ./pkg/model/provider/openai/...✅ (0 issues)