Skip to content

Add image attachment support - #55

Merged
TraderSamwise merged 5 commits into
masterfrom
codex/aimux-next
Jun 2, 2026
Merged

Add image attachment support#55
TraderSamwise merged 5 commits into
masterfrom
codex/aimux-next

Conversation

@TraderSamwise

@TraderSamwise TraderSamwise commented Jun 2, 2026

Copy link
Copy Markdown
Owner

Summary

  • route image content through the relay proxy for remote clients
  • add validated image attachment uploads to the project metadata server
  • add web/native image picking and pending attachment UI to the chat composer

Verification

  • YARN_IGNORE_ENGINES=1 yarn build
  • YARN_IGNORE_ENGINES=1 yarn vitest src/metadata-server.test.ts
  • YARN_IGNORE_ENGINES=1 yarn typecheck
  • YARN_IGNORE_ENGINES=1 yarn --cwd app typecheck
  • YARN_IGNORE_ENGINES=1 yarn --cwd app test lib/api.test.ts components/MessageBlock.test.ts
  • YARN_IGNORE_ENGINES=1 yarn --cwd app lint
  • YARN_IGNORE_ENGINES=1 yarn --cwd app expo export --platform web --output-dir /tmp/aimux-attachment-web-export

Summary by CodeRabbit

  • New Features

    • Attach images to agent chat: pick, preview thumbnails, remove items, and send attachment-only or text+attachment messages
    • Upload image attachments to server and reference them from agent input
  • Bug Fixes / Improvements

    • Relay-mode image URLs now resolve to proxied HTTPS URLs
    • Stronger validation for uploaded images (format, size, base64) and safer attachment ID handling
  • Tests

    • Added coverage for attachment uploads, relay routing, and attachment-related agent inputs
  • Chores

    • Added image picker dependency for native/web platforms

@vercel

vercel Bot commented Jun 2, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
app Ready Ready Preview, Comment Jun 2, 2026 12:07pm

@coderabbitai

coderabbitai Bot commented Jun 2, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: aa5340c2-7faf-41ba-b693-7b0f6eac27f9

📥 Commits

Reviewing files that changed from the base of the PR and between a886d8b and de2fdeb.

📒 Files selected for processing (1)
  • app/components/screens/AgentChatScreen.tsx
🚧 Files skipped from review as they are similar to previous changes (1)
  • app/components/screens/AgentChatScreen.tsx

📝 Walkthrough

Walkthrough

Adds cross-platform image picking, client upload APIs, server-side validated attachment storage, relay-aware proxied image URLs, chat UI attach/preview/send flow, and tests exercising uploads and agent input with attachments.

Changes

Image Attachments for Agent Chat

Layer / File(s) Summary
Relay image URL resolution
app/lib/daemon-url.ts, app/components/MessageBlock.tsx, app/components/MessageBlock.test.ts
Adds getRelayHttpUrl() and getRelayServiceUrl() and updates resolveImageUrl() to return relay-proxied HTTPS URLs for relative image paths in relay mode; tests updated to assert proxied URL behavior and restore env state.
Platform-specific image picker implementations
app/lib/image-picker.ts, app/lib/image-picker.web.ts, app/lib/image-picker.native.ts, app/package.json
Defines PickedImageAttachment and implements pickImageAttachment() for web (hidden file input + FileReader) and native (expo-image-picker with permissions); adds package dependency and a fallback stub for unsupported platforms.
Server-side attachment upload and storage
src/attachment-store.ts
Implements createUploadedAttachment() with MIME normalization, filename/base64 sanitization and validation, max size limit, UUID-based storage paths, SHA256 checksum, metadata JSON, and exported getAttachmentRecord() with ID validation.
Metadata server endpoints and formatting
src/metadata-server.ts
Adds POST /attachments to create uploads and extends POST /agents/input to accept attachmentIds, validate/resolve attachments, format text with an "Attached image files" section, and forward to lifecycle.
Server tests for uploads and agent input
src/metadata-server.test.ts
Adds tests for successful upload & served bytes, malformed base64 rejection, agent input containing attachments (including attachment-only fallback), and error cases for missing/unsafe IDs.
Client API for uploads and agent input with attachments
app/lib/api.ts, app/lib/api.test.ts
Adds uploadImageAttachment() and SendAgentInputOptions (attachmentIds?) to include attachments in sendAgentInput(); tests updated to cover relay routing for agents/input and attachments and direct HTTP upload with Authorization.
Chat screen UI and message send flow with attachments
app/components/screens/AgentChatScreen.tsx
Adds pendingAttachments state, attach-image button, thumbnail preview strip with removal, attachment-aware send flow that uploads pending images before calling sendAgentInput, and error/restore behavior.

Sequence Diagram(s)

sequenceDiagram
  participant Client as AgentChatScreen
  participant ImagePicker
  participant API as app/lib/api
  participant Relay as RelayProxy
  participant Server as MetadataServer
  participant Store as AttachmentStore
  Client->>ImagePicker: pickImageAttachment()
  ImagePicker-->>Client: PickedImageAttachment (base64, preview)
  Client->>API: uploadImageAttachment(base64...)
  API->>Server: POST /attachments (direct project HTTP or via relay)
  Server->>Store: createUploadedAttachment(data)
  Store-->>Server: AttachmentRecord (id, path, meta)
  Server-->>API: 201 AttachmentRecord
  Client->>API: sendAgentInput(text?, attachmentIds)
  API->>Server: POST /agents/input (may go via Relay)
  Server->>Server: formatAgentInputWithAttachments(...)
  Server->>Lifecycle: lifecycle.sendAgentInput(formattedText)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • TraderSamwise/aimux#54: Overlaps changes to MessageBlock.tsx and its tests around relay/non-absolute image URL handling.
  • TraderSamwise/aimux#10: Related prior changes touching relay-mode resolveImageUrl behavior and tests.

Poem

A rabbit finds images in a sunny glade,
Packs them in packets, no bytes delayed,
Through relays they hop, thumbnails in tow,
Chats now sparkle with every pictorial show —
🐇📸✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title 'Add image attachment support' accurately captures the main objective of the changeset, which introduces end-to-end image attachment functionality across UI, API, and server layers.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/aimux-next

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

app/components/screens/AgentChatScreen.tsx

ESLint skipped: missing config or dependency (missing-dependency). The ESLint configuration references a package that is not available in the sandbox.


Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
app/lib/image-picker.native.ts (1)

33-37: ⚡ Quick win

MediaTypeOptions is deprecated; use the MediaType string array.

ImagePicker.MediaTypeOptions has been deprecated in expo-image-picker in favor of a single MediaType or an array (e.g. ['images']). It still works on SDK 54 (~17.0.x) but logs a deprecation warning and will eventually be removed.

♻️ Proposed change
   const result = await ImagePicker.launchImageLibraryAsync({
-    mediaTypes: ImagePicker.MediaTypeOptions.Images,
+    mediaTypes: ["images"],
     base64: true,
     quality: 1,
   });

The deprecation is confirmed in the expo-image-picker changelog: ImagePicker.MediaTypeOptions have been deprecated. Use a single MediaType or an array of MediaTypes instead. Please confirm the array form against the version pinned in this repo before merging.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app/lib/image-picker.native.ts` around lines 33 - 37, The call to
ImagePicker.launchImageLibraryAsync uses the deprecated
ImagePicker.MediaTypeOptions.Images; update the options to use the new MediaType
array form instead—modify the mediaTypes field passed to
ImagePicker.launchImageLibraryAsync (the same call that assigns to result) to
use a string or string-array like ['images'] (or the equivalent MediaType value
supported by the pinned expo-image-picker version) so the deprecated
MediaTypeOptions accessor is removed.
src/metadata-server.ts (1)

2681-2683: ⚡ Quick win

Consider improving the error message for clarity.

The error message "text is required" is shown when both text and attachments are empty. While technically correct (since text is required when there are no attachments), a clearer message would be "text or attachments are required" to better reflect the actual validation logic.

💬 Suggested improvement
         if (!text.trim() && attachmentIds.length === 0) {
-          send(res, 400, { ok: false, error: "text is required" });
+          send(res, 400, { ok: false, error: "text or attachments are required" });
           return;
         }

Note: This change would require updating the test expectation at line 1402 in src/metadata-server.test.ts.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/metadata-server.ts` around lines 2681 - 2683, Update the validation error
message emitted when both text and attachmentIds are empty: replace the current
send(res, 400, { ok: false, error: "text is required" }) with a clearer message
such as "text or attachments are required" so the response matches the condition
if (!text.trim() && attachmentIds.length === 0); also update the corresponding
test expectation in src/metadata-server.test.ts (the assertion around line 1402)
to expect the new message.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@app/components/screens/AgentChatScreen.tsx`:
- Around line 246-280: The handleSendMessage function re-uploads attachments on
retry because the catch restores the original attachments array; change the flow
to track successfully uploaded attachment IDs (the uploaded array populated by
uploadImageAttachment) and on error restore only the remaining un-uploaded
attachments via setPendingAttachments (or persist uploaded IDs and restore those
IDs so retries call sendAgentInput with attachmentIds instead of re-uploading).
Specifically, update the upload loop and error handling around
uploadImageAttachment and sendAgentInput so that uploaded holds attachment
IDs/results to reuse, setDraft(text) only for unsent text, and
setPendingAttachments receives only attachments that were not yet uploaded;
alternatively split upload and send into separate functions that retry
sendAgentInput with existing attachment IDs when available.

In `@app/lib/image-picker.web.ts`:
- Around line 22-26: The promise awaiting the file picker can hang if the native
dialog is dismissed because input.onchange never fires; update the picker logic
to also detect dialog cancel by using a window focus handler: when initiating
the picker (before input.click()), add a one-time
window.addEventListener('focus', onFocus) that checks input.files and, if none
selected, resolves the promise with null; ensure you clean up the focus listener
and the input.onchange handler in both success and cancel paths so the Promise
always resolves and no listeners leak (refer to the existing input.onchange,
input.click and the local file promise).

---

Nitpick comments:
In `@app/lib/image-picker.native.ts`:
- Around line 33-37: The call to ImagePicker.launchImageLibraryAsync uses the
deprecated ImagePicker.MediaTypeOptions.Images; update the options to use the
new MediaType array form instead—modify the mediaTypes field passed to
ImagePicker.launchImageLibraryAsync (the same call that assigns to result) to
use a string or string-array like ['images'] (or the equivalent MediaType value
supported by the pinned expo-image-picker version) so the deprecated
MediaTypeOptions accessor is removed.

In `@src/metadata-server.ts`:
- Around line 2681-2683: Update the validation error message emitted when both
text and attachmentIds are empty: replace the current send(res, 400, { ok:
false, error: "text is required" }) with a clearer message such as "text or
attachments are required" so the response matches the condition if (!text.trim()
&& attachmentIds.length === 0); also update the corresponding test expectation
in src/metadata-server.test.ts (the assertion around line 1402) to expect the
new message.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 72ef28fe-6ec6-444f-846a-c881bed66470

📥 Commits

Reviewing files that changed from the base of the PR and between af2e76b and 3c0054a.

⛔ Files ignored due to path filters (1)
  • app/yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (13)
  • app/components/MessageBlock.test.ts
  • app/components/MessageBlock.tsx
  • app/components/screens/AgentChatScreen.tsx
  • app/lib/api.test.ts
  • app/lib/api.ts
  • app/lib/daemon-url.ts
  • app/lib/image-picker.native.ts
  • app/lib/image-picker.ts
  • app/lib/image-picker.web.ts
  • app/package.json
  • src/attachment-store.ts
  • src/metadata-server.test.ts
  • src/metadata-server.ts

Comment thread app/components/screens/AgentChatScreen.tsx
Comment thread app/lib/image-picker.web.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@app/components/screens/AgentChatScreen.tsx`:
- Around line 250-253: Add a sendBusy guard to prevent duplicate sends: at the
start of handleSendMessage() check if (sendBusy) return, then immediately
setSendBusy(true) before any async work and clear it in a finally block
(setSendBusy(false)); also update the keyboard handler that calls
handleSendMessage to check sendBusy before invoking it. Reference:
handleSendMessage, sendBusy, setSendBusy, keyboard handler (the Enter key
handler) and mirror the same pattern used in handleAttachImage.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 89a59404-fa6e-42d7-af3e-9ae2e6ed785a

📥 Commits

Reviewing files that changed from the base of the PR and between 3c0054a and a886d8b.

📒 Files selected for processing (2)
  • app/components/screens/AgentChatScreen.tsx
  • app/lib/image-picker.web.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • app/lib/image-picker.web.ts

Comment thread app/components/screens/AgentChatScreen.tsx Outdated
@TraderSamwise
TraderSamwise merged commit ee8dff4 into master Jun 2, 2026
3 checks passed
@TraderSamwise
TraderSamwise deleted the codex/aimux-next branch June 2, 2026 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant