Skip to content

fix: rewrite Vertex AI Model Garden provider to publisher for capability lookups#2767

Merged
dgageot merged 1 commit into
docker:mainfrom
dgageot:fix-vertex-model-garden-capabilities-2740
May 12, 2026
Merged

fix: rewrite Vertex AI Model Garden provider to publisher for capability lookups#2767
dgageot merged 1 commit into
docker:mainfrom
dgageot:fix-vertex-model-garden-capabilities-2740

Conversation

@dgageot
Copy link
Copy Markdown
Member

@dgageot dgageot commented May 12, 2026

Fixes #2740.

When a model runs on Vertex AI Model Garden, the OpenAI/Anthropic clients underneath compute their capability-lookup ID as Provider + "/" + Model. With provider: google, that yields google/claude-sonnet-4-20250514 — which doesn't exist in models.dev. The lookup silently fails back to "no capabilities", so attachments and other capability-gated features get dropped.

This rewrites ModelConfig.Provider from google to the lower-cased provider_opts.publisher (e.g. anthropic, meta, mistral) on a clone of the config before handing it to the underlying client. The original config is never mutated, so routing/telemetry keep their google view.

Tests cover anthropic/meta publishers, case folding, whitespace trimming, missing/empty publisher (no-op), input-mutation safety, and a regression test pinning the resulting ID to anthropic/claude-sonnet-4-20250514.

@dgageot dgageot requested a review from a team as a code owner May 12, 2026 10:15
Copy link
Copy Markdown

@docker-agent docker-agent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟡 NEEDS ATTENTION

Comment thread pkg/model/provider/vertexai/modelgarden_test.go Outdated
Comment thread pkg/model/provider/vertexai/modelgarden_test.go
@dgageot dgageot force-pushed the fix-vertex-model-garden-capabilities-2740 branch from e70d921 to 5b6cfcb Compare May 12, 2026 10:22
gtardif
gtardif previously approved these changes May 12, 2026
@dgageot dgageot force-pushed the fix-vertex-model-garden-capabilities-2740 branch from 5b6cfcb to 084d34b Compare May 12, 2026 10:32
@dgageot
Copy link
Copy Markdown
Member Author

dgageot commented May 12, 2026

Addressed both review comments in 084d34b:

[MEDIUM] Meta double-prefix lookup ID — fixed. The remap now distinguishes Anthropic from other publishers:

  • publisher: anthropicProvider = "anthropic" (Vertex Anthropic uses bare model names; models.dev keys them under anthropic/<model>).
  • Other publishers → Provider = "google-vertex" (Vertex's OpenAI-compat path uses <publisher>/<model> model names; models.dev keys them under google-vertex/<publisher>/<model>).

Renamed the helper to withModelsDevProvider and added a dedicated modelsDevProvider mapping function with a doc comment explaining the asymmetry. Added TestWithModelsDevProvider_CapabilityLookupIDs with cases for Anthropic, Meta and Mistral that assert the final composed ID exactly (so Meta now produces google-vertex/meta/llama-..., not meta/meta/llama-...).

[LOW] testify usage — fixed. The new tests now use require.NotNil, require.NotSame, and assert.Equal per AGENTS.md.

@dgageot dgageot merged commit a9fd54b into docker:main May 12, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Vertex AI Model Garden: attachment capability lookup fails due to provider mismatch

3 participants