chore: update model catalog from bot issues - #1026
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Codex (@codex) review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 733a7dc49f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "Codex (@codex) address that feedback".
| "databricks-meta-llama-3-1-8b-instruct": ["databricks"], | ||
| "openai/gpt-oss-120b": ["groq", "together", "baseten"], | ||
| "openai/gpt-oss-20b": ["groq"], // NOTE: We use groq pricing for this and Together pricing for the 120B model | ||
| "openai/gpt-oss-20b": ["groq", "together"], |
There was a problem hiding this comment.
Keep the shared model pinned to Groq pricing
When this model is selected through Groq, the catalog now applies Together's $0.05/$0.20 rates even though Groq's rates remain $0.075/$0.30. model_list.json stores only one price per model, and this change both replaces that price and widens the direct endpoint mapping to Together; the previous mapping deliberately pinned this ID to Groq for pricing. Consequently, Groq usage for this model is underestimated by one third. Keep the direct mapping and catalog rates pinned to Groq, or introduce provider-specific pricing before routing the same ID through both providers.
Useful? React with 👍 / 👎.
The daily bot-issues resolver (fix_bot_issue.ts) added models straight from a ticket without consulting the sync-exclusion list, so models we removed kept coming back every day (e.g. #1026 re-added realtime, Vertex MaaS, and the ENDPOINT_NOT_FOUND Databricks endpoints). Wire isModelExcludedFromSync into resolveIssueCommand: excluded ids are dropped from the actionable set and the issue is closed as unsupported when nothing else remains. Extend MANUAL_SYNC_EXCLUDED_MODELS with the entries trimmed from the catalog PR: - databricks-gpt-5-4-mini, databricks-gemini-3-6-flash (ENDPOINT_NOT_FOUND, not deployed on the account) - the 9 Vertex MaaS ids (real models, but added deliberately via a dedicated PR with the gateway OpenAPI routing change, not by the daily automation) Verified: resolve-issue on an excluded model now returns action "unsupported" and leaves the catalog untouched. Tests: 44 pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Gemini Live uses a bidirectional API and is rejected by generateContent
("not supported for generateContent"), so it is not a chat/completions model —
same rationale as the gpt-realtime-2.1 entries already excluded. Add it to
MANUAL_SYNC_EXCLUDED_MODELS so the LiteLLM sync stops re-adding it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The daily bot-issues resolver (fix_bot_issue.ts) added models straight from a ticket without consulting the sync-exclusion list, so models we removed kept coming back every day (e.g. #1026 re-added realtime, Vertex MaaS, and the ENDPOINT_NOT_FOUND Databricks endpoints). Wire isModelExcludedFromSync into resolveIssueCommand: excluded ids are dropped from the actionable set and the issue is closed as unsupported when nothing else remains. Extend MANUAL_SYNC_EXCLUDED_MODELS with the entries trimmed from the catalog PR: - databricks-gpt-5-4-mini, databricks-gemini-3-6-flash (ENDPOINT_NOT_FOUND, not deployed on the account) - the 9 Vertex MaaS ids (real models, but added deliberately via a dedicated PR with the gateway OpenAPI routing change, not by the daily automation) Verified: resolve-issue on an excluded model now returns action "unsupported" and leaves the catalog untouched. Tests: 44 pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…d Databricks) Now that this branch carries the permanent sync-exclusion list, drop the 11 entries it otherwise both adds and excludes: gpt-realtime-2.1(+mini), the 7 Vertex MaaS ids present, and databricks-gpt-5-4-mini / databricks-gemini-3-6-flash (ENDPOINT_NOT_FOUND). Leaves the day's legitimate additions (kimi-k3, mythos, non-ENF databricks, bedrock). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The daily bot dropped `openrouter` from 8 models' provider lists (gpt-4o, o1, o3-mini, o4-mini, gpt-4-turbo, gpt-4.1-nano, mistral-small-2603, grok-4.5). Keep openrouter for now — the correct fix is a slug-keyed openrouter entry (e.g. `x-ai/grok-4.5`), coming via a `sync-openrouter` automation, rather than dropping openrouter support. Restores available_providers (and grok-4.5's index.ts entry); leaves the batch's other field changes intact. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Automated daily batch of model catalog updates from bot issues.
Included issues
Summary
databricks-gpt-5-4-minidatabricks-gpt-5-4-mini2
publishers/meta/models/llama-4-maverick-17b-128e-instruct-maaspublishers/meta/models/llama-4-scout-17b-16e-instruct-maaspublishers/meta/models/llama-4-maverick-17b-128e-instruct-maaspublishers/meta/models/llama-4-scout-17b-16e-instruct-maas2
3
4
grok-4.5grok-4.5-latestgrok-4.5grok-4.5-latest2
gpt-realtime-2.1gpt-realtime-2.1-minigpt-realtime-2.1gpt-realtime-2.1-mini2
3
4
publishers/mistralai/models/mistral-medium-3publishers/mistralai/models/mistral-small-2503publishers/mistralai/models/codestral-2publishers/mistralai/models/mistral-medium-3publishers/mistralai/models/mistral-small-2503publishers/mistralai/models/codestral-22
3
4
5
publishers/openai/models/gpt-oss-120b-maaspublishers/openai/models/gpt-oss-20b-maaspublishers/openai/models/gpt-oss-120b-maaspublishers/openai/models/gpt-oss-20b-maas2
3
4
thinkingmachines/inklingthinkingmachines/inkling2
openai/gpt-oss-20bopenai/gpt-oss-20b2
3
databricks-gemini-3-6-flashdatabricks-gemini-3-6-flash2
accounts/fireworks/models/kimi-k3accounts/fireworks/models/kimi-k32
3
Verified metadata
#975: [BOT ISSUE] Databricks: add missing databricks-gpt-5-5-pro, databricks-gpt-5-4-mini, databricks-gemini-3-5-flash
Verification notes
No LLM verification step ran; model metadata was already complete in the issue.
sync_models vs proposed update
sync_models cross-check found differences. Official provider verification was used for the applied values, and sync_models discrepancies are listed below for review.
#980: [BOT ISSUE] Vertex: add missing Meta Llama 4 Maverick and Scout MaaS entries
Verification notes
Verification
sync_models vs proposed update
sync_models cross-check found differences. Official provider verification was used for the applied values, and sync_models discrepancies are listed below for review.
#990: [BOT ISSUE] xAI: fix stale cached input pricing for grok-4.5 and grok-4.5-latest ($0.50 → $0.30)
Verification notes
No LLM verification step ran; model metadata was already complete in the issue.
sync_models vs proposed update
sync_models cross-check found differences. Official provider verification was used for the applied values, and sync_models discrepancies are listed below for review.
#991: [BOT ISSUE] OpenAI: add missing gpt-realtime-2.1 and gpt-realtime-2.1-mini
Verification notes
Verification
sync_models vs proposed update
sync_models cross-check found differences. Official provider verification was used for the applied values, and sync_models discrepancies are listed below for review.
#994: [BOT ISSUE] Vertex: add missing Mistral MaaS entries (mistral-medium-3, mistral-small-2503, codestral-2)
Verification notes
Verification
sync_models vs proposed update
sync_models cross-check found differences. Official provider verification was used for the applied values, and sync_models discrepancies are listed below for review.
#996: [BOT ISSUE] Vertex: add missing OpenAI GPT-OSS MaaS entries (gpt-oss-120b-maas, gpt-oss-20b-maas)
Verification notes
Verification
sync_models vs proposed update
sync_models cross-check found differences. Official provider verification was used for the applied values, and sync_models discrepancies are listed below for review.
#999: [BOT ISSUE] Together: add together to available_providers for thinkingmachines/inkling
Verification notes
Verification
sync_models vs proposed update
sync_models cross-check found differences. Official provider verification was used for the applied values, and sync_models discrepancies are listed below for review.
#1001: [BOT ISSUE] Together: stale pricing for openai/gpt-oss-20b ($0.075/$0.30 → $0.05/$0.20)
Verification notes
No LLM verification step ran; model metadata was already complete in the issue.
sync_models vs proposed update
sync_models cross-check found differences. Official provider verification was used for the applied values, and sync_models discrepancies are listed below for review.
#1011: [BOT ISSUE] Databricks: add missing databricks-gemini-3-6-flash, databricks-glm-5-2, and databricks-inkling
Verification notes
No LLM verification step ran; model metadata was already complete in the issue.
sync_models vs proposed update
sync_models cross-check found differences. Official provider verification was used for the applied values, and sync_models discrepancies are listed below for review.
#1025: [BOT ISSUE] Fireworks: add missing accounts/fireworks/models/kimi-k3
Verification notes
Verification
sync_models vs proposed update
sync_models cross-check found differences. Official provider verification was used for the applied values, and sync_models discrepancies are listed below for review.