fix(kosong): clamp max_tokens to max_output_size for OpenAI-compatible providers - #1208
Open
slgao wants to merge 1 commit into
Open
fix(kosong): clamp max_tokens to max_output_size for OpenAI-compatible providers#1208slgao wants to merge 1 commit into
slgao wants to merge 1 commit into
Conversation
…e providers Third-party OpenAI-compatible providers (HuggingFace, Ollama, etc.) can have output limits below the generic CHAT_COMPLETIONS_MAX_OUTPUT_TOKENS_CEILING (131072). When max_output_size is explicitly configured, withMaxCompletionTokens now honours it as a hard upper bound instead of applying the generic ceiling, preventing 400 errors from providers whose actual limit is lower. Fixes MoonshotAI#1148.
🦋 Changeset detectedLatest commit: 52f9427 The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 52f942740f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| baseUrl: providerValue(provider.baseUrl, provider.env, 'OPENAI_BASE_URL'), | ||
| apiKey: providerApiKey(provider), | ||
| reasoningKey, | ||
| ...(maxOutputSize !== undefined ? { maxTokens: maxOutputSize } : {}), |
There was a problem hiding this comment.
When KIMI_MODEL_MAX_COMPLETION_TOKENS=0 or KIMI_MODEL_MAX_TOKENS=0, resolveCompletionBudget returns undefined to disable completion-token clamping, but this line now bakes maxOutputSize into every OpenAI-compatible provider config. OpenAILegacyChatProvider.generate() serializes constructor maxTokens as max_tokens even when applyCompletionBudget is skipped, so any OpenAI-compatible model alias with maxOutputSize will still send a cap despite the documented env opt-out. Keep maxOutputSize in the budget path or avoid wiring it when the opt-out is active.
Useful? React with 👍 / 👎.
xsyetopz
added a commit
to xsyetopz/kimi-code
that referenced
this pull request
Jul 31, 2026
Phase 1 — Non-PR issues (5): - MoonshotAI#94: Read tool status now reports both MAX_LINES and MAX_BYTES when both limits are hit (was: else-if chain only reported the first). Fixed in v1 (agent-core) and v2 (agent-core-v2). Added test for dual-limit case. - MoonshotAI#250: KIMI_MODEL_* env config no longer gives OpenAI/Anthropic models the Kimi default capabilities (image_in, thinking). DEFAULT_CAPABILITIES is now a per-provider-type map: kimi gets ['image_in','thinking'], openai/anthropic get []. Fixed in v1 (env-model.ts) and v2 (envOverlay.ts). Added 2 tests. - MoonshotAI#1972: Skill descriptions exceeding the 250-char model listing limit now emit a warning via onWarning during loadRoots, so the user knows the model sees a truncated version. Added 2 tests (warns / does not warn). - MoonshotAI#1982: Added disabled_skills config denylist. config.toml now accepts disabled_skills = ['skill-name']. Matching skills (case-insensitive) are hidden from getSkill, getPluginSkill, listSkills, listInvocableSkills, register, loadRoots, and the TUI slash menu. Wired through SessionSkillConfig, resolveSessionSkillConfig, and TOML serialization. Added 3 tests. - MoonshotAI#2224: Plugin-qualified skill names (e.g. 'superpowers:systematic-debugging') are now resolved by the native Skill tool. Falls back to getPluginSkill when bare-name getSkill misses and the name contains ':'. Fixed in v1 (skill-tool.ts) and v2 (skillTool.ts). Added 2 tests. Phase 2 — PR-matching issues (5): - MoonshotAI#1008 (PR MoonshotAI#1096): Web UI session archive relabeled from 'Archive' to 'Delete'. Confirm message now states the action is irreversible (en + zh). - MoonshotAI#1106 (PR MoonshotAI#1108): WebSearch and FetchURL tools now forward the abort signal from ExecutableToolContext through to fetch() calls. Updated provider interfaces (WebSearchProvider, UrlFetcher) and all 3 provider implementations (MoonshotWebSearch, MoonshotFetchURL, LocalFetchURL). Updated 2 tests. - MoonshotAI#1050 (PR MoonshotAI#1052): kosong streaming response loop now has an idle timeout (default 60s, configurable via idleTimeoutMs). Replaced bare for-await with manual iterator + Promise.race against timeout. Throws APITimeoutError on idle. Fixed in v1 (kosong/generate.ts) and v2 (agent-core-v2/generate.ts). Added 9 tests. - MoonshotAI#1148 (PR MoonshotAI#1208): max_tokens/max_output_tokens now clamped to 128k ceiling in openai-legacy generate() and openai-responses withMaxCompletionTokens() + generate(). Kimi provider left unchanged (upstream clamping is intentional). Added 4 tests. - MoonshotAI#1273 (PR MoonshotAI#1276): Added KimiCore.dispose() which iterates all live sessions and calls session.close() via Promise.allSettled. Wired into CoreProcessService.dispose(). Idempotent. Added 5 tests. Skipped (already fixed in fork): MoonshotAI#1539, MoonshotAI#244, MoonshotAI#1218. Checklist: .tmp/upstream-tracking/ISSUE-RESOLUTION-CHECKLIST.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issue
Resolves #1148
Problem
When using a third-party OpenAI-compatible provider (HuggingFace, Ollama, etc.) with a
max_output_sizelower thanCHAT_COMPLETIONS_MAX_OUTPUT_TOKENS_CEILING(131072), the agent sends amax_tokensvalue that exceeds the provider's actual limit, causing a 400 error.Root cause: two gaps in the OpenAI provider path:
provider-manager.tswas not forwardingmaxOutputSizeasmaxTokensto the OpenAI provider config (unlike the Anthropic case which already passesdefaultMaxTokens).withMaxCompletionTokensinopenai-legacy.tsalways applies the generic 128k ceiling, overriding an explicitly configuredmaxTokenswhen the budget cap is large.What changed
packages/kosong/src/providers/openai-legacy.ts_explicitMaxTokens: booleanfield, set whenoptions.maxTokensis provided at construction.withMaxCompletionTokens, when_explicitMaxTokensis true, the configured value is used as the ceiling instead ofCHAT_COMPLETIONS_MAX_OUTPUT_TOKENS_CEILING. This mirrors the existing_explicitMaxTokenspattern already used by the Anthropic provider.packages/agent-core/src/session/provider-manager.ts...(maxOutputSize !== undefined ? { maxTokens: maxOutputSize } : {})to theopenaicase intoKosongProviderConfig, mirroring the existingdefaultMaxTokensspread in theanthropiccase.packages/kosong/test/openai-legacy.test.tsmaxTokens: 65536(simulating a HuggingFace endpoint), callswithMaxCompletionTokens(1_048_576, ...), and asserts the resulting request body hasmax_tokens: 65536instead of the old131072.packages/agent-core/test/agent/config-state.test.ts'clamps the LLM completion cap to 128k for openai-compatible providers'test, which was written to document the old buggy behaviour (expect(requestMaxTokens).toBe(131072)formaxOutputSize: 384000). Now asserts384000with an updated comment.Checklist
gen-changesetsskill, or this PR needs no changeset.gen-docsskill, or this PR needs no doc update.