feat: voice prompting — push-to-talk dictation into the composer (web first)

[radroid]

Summary

T3 Code has no voice input today — there is no way to dictate a prompt or attach a voice note on any surface (web, mobile, desktop). This issue proposes adding push-to-talk voice prompting, transcribed to text and dropped into the composer.

Motivation: the primary phone workflow here is the web app over the Tailscale HTTPS origin (see #19). Typing a long prompt on a phone keyboard is the main friction point in that flow; speaking it is not.

Current state (verified against main)

There is no microphone capture, speech-to-text, or audio-attachment path anywhere in the tree. What exists is audio-adjacent but unrelated:

Thing	What it actually is
apps/web/src/browser/browserRecording.ts	MediaRecorder over canvas.captureStream(12) — video capture of the browser-preview tool. No audio track, no getUserMedia.
thread/realtime/* in CodexAdapter.ts:1163-1241, CodexSessionRuntime.ts:530-537	Pass-through plumbing for the Codex app-server protocol's realtime events (outputAudio/delta, transcript/delta, sdp). Translated to canonical thread.realtime.* events in packages/contracts/src/providerRuntime.ts:157-161 — but no client subscribes to them and there is no RPC to start a realtime session. Dead-ends at the contract boundary.
NSMicrophoneUsageDescription in the built .app Info.plist	Electron packaging boilerplate, not a T3 feature.
"VoiceOver" mentions	Accessibility labels.

Composer affordances today are attachments / line-break / model / config / stop — no mic button (apps/mobile/src/features/threads/ThreadComposer.tsx:881-916).

Upstream context — this has to be fork-owned

Upstream pingdotgg/t3code#653 "[Discussion] Voice support" was closed as NOT PLANNED (2026-03-15), despite a working demo PR (local NVIDIA Parakeet 0.6B v2 STT + Rust sidecar). Maintainer response: "out of scope, there are apps that do this already." Commenters asked for it to be opt-in if it ever landed.

So there is no upstream implementation to wait for or rebase onto — if we want it, the fork builds and maintains it.

Related upstream issues, still open, all output-side rather than input-side:

• #3964 — read AI responses aloud via TTS
• #4587 — render agent-generated audio files inline
• #728 — macOS TCC mic/camera permissions for apps launched from the integrated terminal (relevant if capture ever runs desktop-side)

Proposed scope (v1)

Surface first: web, because that is the phone-over-Tailscale path.

• Mic button in the composer footer, push-to-talk (hold to record) plus a tap-to-toggle mode.
• Capture via getUserMedia + MediaRecorder (opus/webm). Secure-context requirement is already satisfied by the Tailscale HTTPS origin.
• Transcribe, then insert the text into the composer for review — do not auto-send. Dictation errors on a coding prompt are expensive.
• Opt-in: off by default, enabled from a settings row (mirror the notifyOnNeedsInput pattern in SettingsPanels.tsx).
• Visible recording state + level meter, and a hard cap on clip length.

Deliberately out of scope for v1: TTS/read-aloud, realtime bidirectional voice sessions, storing the audio as a durable "voice note" artifact, and mobile-native capture. Mobile app parity is a follow-up once the web shape is settled.

Open decisions

1. Where transcription runs. Options: (a) server-side call to a cloud STT model — simplest, best accuracy, needs a key and sends audio off-box; (b) on-device SpeechRecognition Web Speech API — free, but effectively unusable on iOS Safari, which is the target device; (c) local model + sidecar, as the upstream demo did — no network, but ships a binary and burns CPU/RAM. Leaning (a) for v1, behind an opt-in setting, with the provider abstracted so (c) can land later.
2. Whether audio is retained at all after transcription, or discarded immediately. Retaining turns this into a true "voice note" feature and pulls in persistence + a contracts change; discarding keeps it a dictation feature. Leaning discard for v1.
3. Whether the desktop app gets it via the embedded web surface for free, or needs its own capture path (see upstream macOS: apps launched from integrated terminal can't request TCC permissions (mic, camera) pingdotgg/t3code#728 for the macOS TCC wrinkle).

Seam discipline

The composer is one of the hottest fork seams we have — docs/t3x/SEAMS.md already carries rows for ComposerPrimaryActions.tsx (risk 276), ChatComposer.tsx (612), ChatView.tsx (9540), and ThreadComposer.tsx (480), and the outbox work already drifted against an upstream gate once.

Per the ledger's tripwire rule: put the capture/transcription logic in fork-owned files registered through apps/server/src/t3x/index.ts and the per-surface aggregator. Budget for exactly one new upstream edit — the mic button mount point in the composer footer — and update the SEAMS.md row and header totals in the same commit that adds it.

[radroid]

Closing this — same reasoning upstream gave in pingdotgg/t3code#653, and having now costed it out, I think they were right.

Why building it isn't worth it

The category that makes an in-app feature redundant isn't Wispr Flow (~$12–15/mo) — it's the free, system-wide, offline tools. They paste into whatever app has focus, so they already dictate into T3's composer without a line of T3 code, and they work in the terminal, the editor, and everywhere else at the same time:

• Windows Voice Typing (Win+H) — already installed, $0, ~85–90% accuracy with Enhanced Speech Recognition enabled.
• Handy — MIT, fully offline, push-to-talk, Windows/macOS/Linux. Whisper on GPU, or Parakeet V3 on CPU. (README notes Whisper crashes on some Windows configs; Parakeet V3 is the safe CPU default but weaker on accents/noise.)
• OpenWhispr, whisper-local — same shape, also free and offline.

On phones, iOS and Android keyboard dictation covers the Tailscale web-app workflow that originally motivated this issue, also at $0.

The cost that actually killed it

The API bill was never the problem. At 20 min of dictation/day, Groq whisper-large-v3-turbo ($0.04/hr) runs **$0.30/month**, comfortably inside budget — though its 10-second minimum billing per request bites for push-to-talk, where most clips are shorter than that. OpenAI is $0.003–0.006/min, i.e. ~$1.30–2.60/month for the same usage. Local models are $0.

The real cost is seam maintenance. Per docs/t3x/SEAMS.md, the composer is the hottest surface the fork touches — ChatView.tsx at risk 9540, ChatComposer.tsx at 612, ComposerPrimaryActions.tsx at 276 — and the outbox work has already drifted against a new upstream gate once. Adding a mic button means a permanent, recurring rebase tax on the highest-churn file in the tree, paid every sync, forever, to duplicate something Win+H does for free. The ledger's own tripwire says re-isolate rather than add row 35; this is a case for not adding the row at all.

What would reopen this

Not "we want voice" — that's solved. Only this: if system dictation turns out to drop or mangle text in T3's composer specifically, because the composer is a custom editor (ComposerPromptEditor.tsx on web, T3ComposerEditorView.swift on iOS) rather than a plain <textarea>. That would be an input-handling bug in the editor, filed as such and fixed there — not a voice feature to build.

Worth a two-minute test with Win+H against the composer before this is fully settled.