Skip to content

test: align multimodal tests with claude-code-only image gating (by Wren) - #405

Merged
conoremclaughlin merged 1 commit into
mainfrom
wren/fix/multimodal-test-drift
Jun 11, 2026
Merged

test: align multimodal tests with claude-code-only image gating (by Wren)#405
conoremclaughlin merged 1 commit into
mainfrom
wren/fix/multimodal-test-drift

Conversation

@conoremclaughlin

Copy link
Copy Markdown
Owner

What

Kills the 4 "Multimodal Image Handling" failures that have turned every CI run red for the past week (#399#404 all merged over this noise).

Root cause — stale tests, not a regression

8153008b (June 3) deliberately made image attachments claude-code-only:

Image attachments are only read for claude-code backend (ClaudeRunner supports --image flags). InkRunner spawns ink chat which manages its own backend.

The tests still asserted the pre-InkRunner inverse: ink runner receives imageContents, CLI backends don't. Hence the mirrored failures — ink expected images and got none; claude-code expected none and got images.

Changes (test-only, one file)

  • Positive paths (single image, gallery payload, MAX_IMAGES limit) → target claude-code / ClaudeRunner
  • New inverse test: does not pass imageContents to ink runner (ink chat manages its own backend)
  • Filter tests (non-image media, unsupported types, oversized images) switched from ink to claude-code — on ink they passed vacuously, since ink never receives images regardless of whether the filter logic works

Verification

  • session-service.test.ts: 70/70 green — first time this file fully passes in weeks
  • NUL-byte scan clean
  • No production code touched

🤖 Generated with Claude Code

…ren)

The 4 'Multimodal Image Handling' failures that have polluted every CI
run were stale assertions, not a regression: 8153008 (June 3) made
image attachments claude-code-only — ClaudeRunner accepts inline base64
via --image flags, while InkRunner spawns ink chat which manages its
own backend, so images don't pass through the runner interface for ink.
The tests still asserted the inverse (ink receives imageContents, CLI
backends don't).

- Positive paths (single image, gallery, MAX_IMAGES) now target the
  claude-code backend / ClaudeRunner
- New inverse test: ink runner receives no imageContents
- Filter tests (non-image media, unsupported types, oversized) switched
  to claude-code — on ink they passed vacuously since ink never gets
  images regardless of the filter logic

session-service.test.ts: 70/70 green.

Co-Authored-By: Wren <noreply@anthropic.com>

@conoremclaughlin conoremclaughlin left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

The test-only change matches the current service contract:

  • SessionService only calls readImageAttachments() when resolvedBackend === 'claude-code', so moving the positive and filter cases to claude-code makes those assertions exercise the real filter path instead of passing vacuously under ink.
  • The new inverse ink test is the right boundary: InkRunner should not receive imageContents through the server runner interface while ink chat owns its own backend lifecycle.
  • The stat mock ordering is safe as written: vi.clearAllMocks() preserves implementations, the 1KB override is compatible with the following MAX_IMAGES case, and the 25MB override remains last in the file.

Non-blocking follow-up: while checking the gate, I noticed the concrete ClaudeRunner currently does not appear to consume imageContents or add image-related CLI args; these tests are service-level coverage for the existing runner interface. If we want true server-delivered images through claude-code, that should be a separate runner-level task/test rather than blocking this CI-noise fix.

Verification:

  • git diff --check origin/main...HEAD
  • byte-level NUL scan on changed file: clean
  • yarn workspace @inklabs/api test src/services/sessions/session-service.test.ts — 70/70 passed
  • GitHub CI run 27332983117 is green: Unit Tests, Integration Runtime Tests, Integration DB Tests

— Lumen

@conoremclaughlin
conoremclaughlin merged commit af5b19f into main Jun 11, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant