Improve chat image references

[TraderSamwise]

Summary

• replace raw chat attachment metadata with numbered image reference tokens
• render image history parts as compact text chips instead of inline previews
• keep Codex suggested prompt input out of parsed chat prompts

Tests

• yarn vitest run lib/parsed-transcript.test.ts components/MessageBlock.test.ts (app)
• yarn vitest run src/agent-output-parser.test.ts
• yarn typecheck (root and app)
• yarn build
• pre-push: yarn typecheck && yarn lint && yarn test

Summary by CodeRabbit

• New Features

  • Image references now render as labeled tokens (not full inline images) for clearer display.
  • Transcript parsing produces stable numbered labels for attached images.

• Improvements

  • Better parsing of varied image attachment formats and more accurate extraction of image metadata.
  • Improved normalization to separate footer/status text from active prompts.

• Tests

  • Added tests covering image reference parsing and transcript/normalization behavior.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
app	Ready	Preview, Comment	Jun 5, 2026 3:28pm

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f48a59c4-be9b-46b3-aa37-646f0e2888cb

📥 Commits

Reviewing files that changed from the base of the PR and between 011ab69 and 9bb43af.

📒 Files selected for processing (2)

• src/agent-output-parser.test.ts
• src/agent-output-parser.ts

🚧 Files skipped from review as they are similar to previous changes (1)

• src/agent-output-parser.ts

---

📝 Walkthrough

Walkthrough

This PR adds an image_reference part type and parser to extract labeled image references from transcript text, renders those references as tokens in messages, and refines prompt-vs-status normalization (Codex heuristics) with accompanying tests.

Changes

Image Reference Handling

Layer / File(s)	Summary
Image Reference Contract and Parsing Infrastructure app/lib/events.ts, app/lib/parsed-transcript.ts, app/lib/parsed-transcript.test.ts	HistoryImageReferencePart added. Regexes detect inline/multi-line "Attached image files" and "Viewed Image" patterns; parser emits image_reference parts with stable per-attachment labels via shared imageLabels. Tests verify parsing, flattened format handling, and label stability across blocks.
Message Block UI Rendering app/components/MessageBlock.tsx	MessageBlock now renders image_reference parts via ImageReferenceToken instead of resolving/displaying Image. Imports simplified and MessageBlock signature destructures only message. imagePartLabel helper added for display labels.
Block Classification Refinement src/agent-output-parser.ts, src/agent-output-parser.test.ts	Normalization now accepts tool and includes helpers to detect footer-like/in-progress status and Codex-suggested prompts. Adds a pass that retypes certain "prompt" blocks to "status" when followed by footer-like text under contextual conditions. Tests cover Codex and Claude scenarios.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Possibly related PRs

• TraderSamwise/aimux#57: Also modifies parser normalization heuristics in src/agent-output-parser.ts, relating to Codex/Claude block classification changes.
• TraderSamwise/aimux#54: Related changes to image handling in app/components/MessageBlock.tsx and attachment URL normalization; overlaps with image rendering/reference handling.

Poem

🐰 I found the images in the thread,

numbered and labeled, light as lead,
tokens hop where URLs hid,
labels steady, never slid,
chorus of Codex kept in stead.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Improve chat image references' directly aligns with the main objective: replacing raw attachment metadata with numbered image reference tokens and rendering them as compact text chips instead of inline previews.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/chat-image-reference-tokens

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

app/components/MessageBlock.tsx (1)

55-55: ⚡ Quick win

Remove dead serviceEndpoint from MessageBlock props.

MessageBlock no longer uses serviceEndpoint, but Props still requires it, so callers must keep passing an unused prop.

Proposed diff

 interface Props {
   message: ChatMessage;
-  serviceEndpoint: ServiceEndpoint;
 }

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app/components/MessageBlock.tsx` at line 55, Remove the unused
serviceEndpoint property from the Props type used by MessageBlock: update the
Props interface/type declaration to drop serviceEndpoint and adjust any
imports/CallSites to stop passing it (or update their prop objects) so
MessageBlock({ message }: Props) no longer requires that unused field; ensure
type references to Props (and any tests or parent components) are updated
accordingly to prevent type errors.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/agent-output-parser.ts`:
- Around line 310-323: The condition that retypes a prompt to "status"
incorrectly calls isCodexSuggestedPrompt(current.text) for all tools; restrict
that check to only run for Codex-sourced prompts by adding a tool check (e.g.,
current.tool === 'codex' or current.tool?.name === 'codex') in the same if
expression inside the loop in agent-output-parser (the block that uses
looksLikeFooterStatus, looksLikeInProgressStatus, and isCodexSuggestedPrompt);
update the combined conditional so isCodexSuggestedPrompt(...) is evaluated only
when the current item's tool identifies it as Codex, leaving other tools'
prompts unaffected.

---

Nitpick comments:
In `@app/components/MessageBlock.tsx`:
- Line 55: Remove the unused serviceEndpoint property from the Props type used
by MessageBlock: update the Props interface/type declaration to drop
serviceEndpoint and adjust any imports/CallSites to stop passing it (or update
their prop objects) so MessageBlock({ message }: Props) no longer requires that
unused field; ensure type references to Props (and any tests or parent
components) are updated accordingly to prevent type errors.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c1598242-7805-4d12-b909-f499d6ffbf81

📥 Commits

Reviewing files that changed from the base of the PR and between 6c494cb and 011ab69.

📒 Files selected for processing (6)

• app/components/MessageBlock.tsx
• app/lib/events.ts
• app/lib/parsed-transcript.test.ts
• app/lib/parsed-transcript.ts
• src/agent-output-parser.test.ts
• src/agent-output-parser.ts