Non-vision models blocked from passing images to vision-capable MCP tools

[andrea-tomassi]

Description

When using a non-vision model, image attachments are rejected at the backend level before the LLM can process them. The model receives an error text instead of the image, so it can't delegate the image to a vision-capable MCP tool (like zai-mcp-server_analyze_image).

This is inconsistent: a vision model and a non-vision model are both capable of calling the same MCP tool to analyze an image — neither uses its own "vision" for the actual analysis. But only the vision model can receive the image reference in the first place.

Steps to Reproduce

1. Select a non-vision model (e.g., one without image in its modalities.input)
2. Upload an image via drag-and-drop or the attach button (this works — frontend gating doesn't exist)
3. Submit the prompt
4. The LLM receives: ERROR: Cannot read "image.png" (this model does not support image input). Inform the user.
5. The LLM cannot call zai-mcp-server_analyze_image (or any other vision tool) because the image reference is gone

Root Cause

In packages/opencode/src/provider/transform.ts, the unsupportedParts() function (lines 391–427) replaces image/file parts with an error text when the model lacks capabilities.input.image:

// line 414-416
const modality = mimeToModality(mime)         // → "image"
if (!modality) return part                     
if (model.capabilities.input[modality]) return part  // passes through

// line 418-421 — image stripped, replaced with error
return {
  type: "text" as const,
  text: `ERROR: Cannot read ${name} (this model does not support ${modality} input). Inform the user.`,
}

This is called at line 430 on every message before dispatch.

Meanwhile, the frontend (packages/app/src/components/prompt-input.tsx) has zero awareness of model capabilities — the attach button is always active, drag-and-drop always works, paste always works. This creates a misleading UX where you can upload but the backend silently strips it.

Expected Behavior

A non-vision model should still receive the image as context (even if only as a text reference to the file), so it can dispatch the image to a vision-capable MCP tool (e.g., zai-mcp-server_analyze_image). The model itself doesn't need to support image input — it just needs to know the image exists and have a reference to pass to a tool.

Suggested Fix

Instead of replacing unsupported image parts with a hard error, pass the file metadata as a text part (e.g., [Attached image: "image.png" (image/png)]) so the LLM can still reason about it and route it to tools that handle images.

Environment

• OpenCode version: current dev branch
• Reproducible: always

[github-actions]

This issue might be a duplicate of existing issues. Please check:

• [FEATURE]: Auto-describe images via vision fallback when active model lacks vision support #24948: Auto-describe images via vision fallback when active model lacks vision support — same root problem (non-vision models can't receive images), proposes a similar fallback approach
• [FEATURE]: Auto image-to-text transcription via multimodal model for non-multimodal providers #22828: Auto image-to-text transcription via multimodal model for non-multimodal providers — overlapping concern about routing image data when the active model lacks vision
• [FEATURE]: provide full image path in the error message when model does not support read image. #23508: Provide full image path in error message when model does not support image — directly overlaps with the suggested fix (passing file metadata as text instead of a hard error)
• [BUG] Non-vision models fail even when user only sends text - OpenCode auto-reads clipboard images #26668: Non-vision models fail even when user only sends text — related non-vision model image-stripping behavior

If your issue is distinct, please clarify what differentiates it from the above. Otherwise, consider adding a comment to the most relevant existing issue instead.

[andrea-tomassi]

Thanks for flagging these. #29216 is distinct from each:

vs #24948 and #22828 (auto vision fallback / auto transcription)

Both propose a server-side automation — automatically call a separate vision model to transcribe the image into text, then feed that text to the non-vision model. This is a heavyweight feature with config, model selection, and a processing pipeline.

#29216 is much simpler and more fundamental: just don't strip the image. The non-vision model doesn't need to read the image itself — it only needs the image reference (file path / MIME / filename) so it can decide to dispatch the image to a vision-capable MCP tool. This is the tool-delegation pattern — the model says "I can't see this, let me call analyze_image on it." An auto-fallback does the work silently; the tool-delegation pattern gives the model agency.

vs #23508 (provide full image path in error)

The closest in spirit, but #23508 asks for a better error message. #29216 argues there shouldn't be an error at all. The image should pass through as context the model can reason about, not as an error it must apologize for. The difference: #23508 says "tell me where the image is" → #29216 says "don't destroy the image reference in the first place."

vs #26668 (clipboard auto-read)

Unrelated bug — that's about clipboard images being auto-attached even for text-only messages. #29216 is about intentionally uploaded images being stripped.

---

Summary: All four existing issues assume the model must either natively support images or get a text transcription. They all work within the assumption that the unsupportedParts() rejection is correct behavior. #29216 challenges that assumption: the model doesn't need to support images — it just needs an un-destroyed reference so it can route to tools.

The fix is also simpler than #24948/#22828: change unsupportedParts() to emit a text part with file metadata instead of a hard error — no model routing, no config, no transcription pipeline needed.