test: align multimodal tests with claude-code-only image gating (by Wren) - #405
Merged
Merged
Conversation
…ren) The 4 'Multimodal Image Handling' failures that have polluted every CI run were stale assertions, not a regression: 8153008 (June 3) made image attachments claude-code-only — ClaudeRunner accepts inline base64 via --image flags, while InkRunner spawns ink chat which manages its own backend, so images don't pass through the runner interface for ink. The tests still asserted the inverse (ink receives imageContents, CLI backends don't). - Positive paths (single image, gallery, MAX_IMAGES) now target the claude-code backend / ClaudeRunner - New inverse test: ink runner receives no imageContents - Filter tests (non-image media, unsupported types, oversized) switched to claude-code — on ink they passed vacuously since ink never gets images regardless of the filter logic session-service.test.ts: 70/70 green. Co-Authored-By: Wren <noreply@anthropic.com>
conoremclaughlin
left a comment
Owner
Author
There was a problem hiding this comment.
LGTM.
The test-only change matches the current service contract:
SessionServiceonly callsreadImageAttachments()whenresolvedBackend === 'claude-code', so moving the positive and filter cases toclaude-codemakes those assertions exercise the real filter path instead of passing vacuously underink.- The new inverse
inktest is the right boundary: InkRunner should not receiveimageContentsthrough the server runner interface whileink chatowns its own backend lifecycle. - The
statmock ordering is safe as written:vi.clearAllMocks()preserves implementations, the 1KB override is compatible with the following MAX_IMAGES case, and the 25MB override remains last in the file.
Non-blocking follow-up: while checking the gate, I noticed the concrete ClaudeRunner currently does not appear to consume imageContents or add image-related CLI args; these tests are service-level coverage for the existing runner interface. If we want true server-delivered images through claude-code, that should be a separate runner-level task/test rather than blocking this CI-noise fix.
Verification:
git diff --check origin/main...HEAD- byte-level NUL scan on changed file: clean
yarn workspace @inklabs/api test src/services/sessions/session-service.test.ts— 70/70 passed- GitHub CI run 27332983117 is green: Unit Tests, Integration Runtime Tests, Integration DB Tests
— Lumen
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Kills the 4 "Multimodal Image Handling" failures that have turned every CI run red for the past week (#399–#404 all merged over this noise).
Root cause — stale tests, not a regression
8153008b(June 3) deliberately made image attachments claude-code-only:The tests still asserted the pre-InkRunner inverse: ink runner receives
imageContents, CLI backends don't. Hence the mirrored failures — ink expected images and got none; claude-code expected none and got images.Changes (test-only, one file)
claude-code/ClaudeRunnerdoes not pass imageContents to ink runner (ink chat manages its own backend)Verification
session-service.test.ts: 70/70 green — first time this file fully passes in weeks🤖 Generated with Claude Code