fix: consume the model metadata declared by the models.dev catalog - #2015
Conversation
Models served over the Anthropic protocol whose names carry no Claude marker (e.g. a catalog-imported Kimi K3) no longer inherit the latest Opus effort list, so the model selector stops offering levels the model does not accept. The models.dev catalog import now also parses reasoning_options and records the declared effort levels on the model alias, so K3 offers its real levels (low / high / max).
🦋 Changeset detectedLatest commit: 81e79bf The changes in this PR will be included in the next version bump. This PR includes changesets to release 5 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
commit: |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 120ac572b5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…models.dev catalog - Models declared status=deprecated in the catalog are no longer offered for import. - Per-model provider overrides on gateway providers (an npm package targeting an Anthropic SDK plus a usable endpoint) now land as alias protocol and base_url, so those models are served over the right protocol and endpoint; overrides without a usable URL are skipped. - A declared limit.input now sizes the context budget instead of the larger total context window (e.g. gpt-5: 272k instead of 400k). The model alias schema gains an optional base_url field (not accepted in overrides) that Anthropic wire resolution prefers over the provider-level base URL.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9ffeb5db9d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ack in catalog imports - reasoning_options 'none' is the model's off encoding: off_effort flows from the catalog through the model alias to the OpenAI wire providers, so turning thinking off sends 'none' instead of omitting the effort field; models with effort levels but no way to disable thinking are imported as always_thinking and no longer offer an Off option. - Bare Claude family aliases (e.g. sonnet-latest) recover the inferred Anthropic effort profile; v2 comment conventions restored. - Providers whose SDK the catalog does not type now fall back to the OpenAI-compatible wire (with a visible "guessed" note) instead of being refused; imports lacking a usable endpoint ask for one (--base-url on the CLI, a prompt in the TUI). Proprietary SDKs (Amazon Bedrock), unrecognized explicit types, and env-placeholder URLs are refused with a clear reason.
RealKai42
changed the title
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 39d1dd30ec
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- A JSON null tier in declared effort values is now read as the 'none' off-encoding (previously such models were wrongly imported as always-thinking with no way to turn reasoning off). - Alpha-status models are filtered out alongside deprecated ones. - Models whose per-model provider override targets a wire that cannot be expressed per-model (e.g. Claude on google-vertex, whose wire here is Gemini-mode Vertex, or gpt entries on an Anthropic provider) are skipped instead of being imported under the silently wrong protocol. - interleaved: true no longer pins reasoning_content: the provider's default three-field scan is wider and the pinned key only narrowed reasoning parsing for gateways answering with another field name.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bbf8c2c915
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…onor --base-url - catalogProviderNeedsBaseUrl now covers the Anthropic wire: a non-official Anthropic-compatible vendor without a concrete catalog endpoint (e.g. google-vertex-anthropic) must supply --base-url / the TUI prompt instead of silently falling back to the default Anthropic endpoint. - --base-url now takes precedence over the catalog-declared endpoint, and an empty --base-url is rejected instead of persisting a blank endpoint.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d83b4e4771
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…port A model that declares always_thinking (e.g. a catalog-imported gpt-5) no longer resolves to a dishonest off state via thinking.enabled=false or an SDK/ACP off request: resolution clamps to the model's default effort on every wire instead of letting upstream keep reasoning while the UI reports Off. The Anthropic warn-and-send path for unlisted effort levels is unchanged. Cohere's proprietary SDK joins Amazon Bedrock on the import-refusal list instead of being guessed as OpenAI-compatible.
- An explicit but unrecognized catalog type is now refused before npm/id inference, so a future catalog protocol is never silently miswired through the OpenAI fallback. - User-supplied --base-url values for Anthropic-wire providers get the same trailing-/v1 normalization as catalog endpoints, avoiding /v1/v1/messages requests. - The TUI import prompt rejects env-placeholder base URLs like the CLI does.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2f9ac6c62b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…tion Wire-type inference, the OpenAI-compatible fallback, proprietary-SDK refusal, endpoint adaptation, and the base-URL requirement are now produced together by resolveCatalogImport, one pure resolver consumed by both the CLI and the TUI — replacing the cooperating predicates (inferWireType, isGuessedWireType, catalogProviderNeedsBaseUrl) whose permutations kept producing edge cases. No behavior change.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ed0569a146
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…y same-wire override endpoints - A configured thinking.effort = "off" no longer bypasses the always-on clamp: it is treated as absent and the model default applies, mirrored on both engines. - The previously public inferWireType stays as a deprecated compatibility wrapper over resolveCatalogImport so existing SDK consumers do not break on a patch release. - Catalog model overrides that stay on the provider's wire but declare their own endpoint now persist it on the alias (and the v1 OpenAI wire branches honor alias-level base URLs like the Anthropic branch).
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6da7351fbd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- max_context_tokens once again means the total context window (used by completion budgeting); a model's declared input limit is tracked as max_input_tokens, which compaction, context-splice and usage-ratio checks prefer — fixing the over-clamping introduced when the input cap was stored as the context budget. - A catalog endpoint declared only as an env placeholder now always produces needs-base-url (official SDK included), so credentials are never sent to the public vendor host by default. - api-only per-model overrides are honored as same-wire endpoint changes; overrides targeting another known but inexpressible wire (e.g. google-genai on an OpenAI gateway) are skipped; same-wire models whose declared endpoint is an unusable placeholder are skipped instead of silently rerouted.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 81e79bf4b4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
1 participant
Related Issue
None — the problem is explained below.
Problem
The model selector offered thinking effort levels the selected model does not support. Importing "Kimi For Coding" from the models.dev catalog and selecting K3 showed Off / Low / Medium / High / Xhigh / Max, while K3 only accepts low / high / max — picking one of the phantom levels sent an effort value upstream that the model does not recognize.
Two causes stacked up:
reasoning_options, but the catalog import ignored that field, so nosupport_effortswas recorded on the imported alias.What changed
reasoning_optionseffort values into the alias'ssupport_efforts(thenonepseudo-level is dropped; declaring levels implies thinking support), so imported models advertise their real levels — K3 offers Off / Low / High / Max.claude-latest); clearly non-Claude names served over the Anthropic protocol no longer inherit Claude effort levels. Wire-path profile inference for unknown models is unchanged.Checklist
gen-changesetsskill, or this PR needs no changeset.gen-docsskill, or this PR needs no doc update.Follow-up: consume more of the metadata the catalog declares
A review of every field models.dev returns surfaced three more declared-but-ignored cases of the same class, fixed in the second commit:
status: "deprecated"(124 models): deprecated models are no longer offered for import; previously they landed in the picker and config with no signal.provideroverrides (188 models): gateway providers (zenmux, opencode, azure, …) serve some models over a different protocol/endpoint than the provider default. Overrides that explicitly target an Anthropic SDK and have a usable URL now land as aliasprotocol+base_url(a new alias field the Anthropic wire resolution prefers over the provider-level URL). Overrides without a usable endpoint — env-placeholder URLs or no URL at all (google-vertex) — and directions the alias schema cannot express are skipped, so nothing regresses.limit.input(1107 models): when declared below the total context window (e.g. gpt-5: 400k window, 272k input cap), it now sizes the context budget so compaction kicks in before upstream rejects the prompt.Tests added for all three fixes, including an end-to-end pass over a real models.dev snapshot (zenmux overrides, google-vertex non-regression, gpt-5 input cap, deprecated filtering).
Follow-up 2: review feedback and the OpenAI-compatible fallback
Addressed all three review comments, and a fourth same-class gap found during live smoke testing:
noneis now honored as the thinking-off encoding (Codex P2): models whose declared effort levels includenone(421 across the catalog, e.g. xai grok) carry anoff_effortthat the OpenAI chat-completions and Responses wires send when thinking is turned Off — previously they omitted the field, so models that reason by default kept reasoning. Models with effort levels but no toggle and nonone(925, e.g. gpt-5) are imported asalways_thinkingand no longer offer an Off option.sonnet-latest/opus-latest-style names on Anthropic-compatible providers again get the inferred effort list; clearly non-Claude names (k3, GLM, …) still get none.--base-urlon the CLI, a prompt in the TUI). Amazon Bedrock, unrecognized explicittypes, and env-placeholder URLs are refused with a clear reason.Verified end-to-end against the live models.dev catalog (CLI imports for kimi-for-coding / zenmux / openai / xai / openrouter / bedrock) and live requests through all four protocol wires, including
off → reasoning_effort: 'none'being accepted and actually disabling reasoning on a real endpoint.Follow-up 3: alignment with the reference models.dev consumer (opencode)
A side-by-side study of how opencode (the creator of models.dev) consumes the catalog surfaced four more divergences, all fixed:
nulleffort tier: models.dev's schema allows anullvalue in effortvalues, equivalent to'none'(e.g. sarvam-105b[null, low, medium, high]). It is now read as thenoneoff-encoding — previously the value was dropped and such models were wrongly imported as always-thinking.vertexaiwire is Gemini-mode, not Anthropic-over-Vertex; gpt entries on an Anthropic provider) were previously imported under the provider's protocol, guaranteed wrong at runtime. They are no longer offered.interleaved: trueno longer pinsreasoning_content: the provider already scansreasoning_content/reasoning_details/reasoninginbound and writesreasoning_contentoutbound, so the pinned key only narrowed parsing for gateways that answer with another field name (the pin predates the default scan; it was necessary when introduced and became a negative optimization later the same day).Verified against the live catalog: sarvam imports with
off_effort=none, google-vertex imports only its Gemini models, freemodel gpt entries are skipped.Follow-up 4: consolidation refactor
The review rounds exposed that the import decision logic had grown into cooperating predicates (
inferWireType+isGuessedWireType+catalogProviderNeedsBaseUrl+catalogBaseUrl+adaptBaseUrlForWire), whose permutations kept producing edge cases. They are now replaced by a single pure resolver,resolveCatalogImport(entry, userBaseUrl?), which returns a discriminated decision —ok(wire, guessed flag, adapted endpoint) /needs-base-url/invalid(typed reason) — consumed by both the CLI and the TUI. All error/reason surfaces (proprietary SDK, unknown explicit type, empty or placeholder URL) are produced in one place. Behavior is unchanged; this is verified by the migrated predicate test matrix plus a fresh live-catalog smoke pass.All 15 Codex review comments across the rounds have been addressed and resolved; CI is green.