feat(audio): switch playback to <audio src=…> via /api/calls/:id/audio#27
Merged
feat(audio): switch playback to <audio src=…> via /api/calls/:id/audio#27
Conversation
- Replace base64-over-WS + Web Audio decode pipeline with a single persistent hidden HTMLAudioElement, wired through MediaElementAudioSourceNode → GainNode → AudioContext.destination so the existing volume slider and gain graph keep working. - WS client no longer parses or holds an audio callback; on CAL it dispatches callReceived(call) to Redux. A new listener middleware drives the player off the calls slice. - Per-call playback flow: audio.src = /api/calls/:id/audio, preload auto, play() on canplay, advance queue on ended/error. - Download buttons in BookmarksPanel and SearchPanel are now plain <a download> anchors pointing at the same authenticated endpoint. - Shared-call page uses <audio src=/api/shared/:token/audio> directly. - Autoplay-unlock now also primes the persistent <audio> element inside the user-gesture handler (play().then(pause)) so subsequent programmatic play() calls succeed on Mobile Edge / Mobile Safari. - Tests updated to drive the new flow; 188 frontend tests pass. Pairs with backend PR #26 dropping audioData from CAL frames; merge that one first, then this.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Switch the SPA audio pipeline from base64-over-WebSocket + Web Audio
decodeAudioDatato a single persistent hiddenHTMLAudioElementthat streams each call from the existingGET /api/calls/:id/audioendpoint, authenticated via theos_sessioncookie introduced in #25.Pairs with #26 — merge order matters
This PR depends on backend PR #26 ("drop embedded audio from CAL messages"). Merge #26 first, then this immediately. Either side works on its own (the frontend simply ignores any base64
audiofield the backend still sends, and the backend with this frontend just stops shipping bytes nobody reads), but the clean cut is the goal.User-visible fix
Mobile Edge on Android now plays AAC calls. Edge ships AAC support in
HTMLMediaElement(via Android MediaCodec) but not in Web Audio'sdecodeAudioData— the old path threwEncodingErrorand the existing fallback was hidden behind a.catch(() => {}), so calls silently never played. Routing every call through the platform<audio>element fixes this without touching the gain graph the volume slider depends on.Side benefit: the client no longer holds a base64 string + decoded
AudioBufferper call in memory.Architecture
HTMLAudioElementis created during the existing first-user-gesture audio-unlock and wired throughMediaElementAudioSourceNode → GainNode → AudioContext.destination. The volume slider, mute, and beep all continue to flow through the same gain node.CALit dispatchescallReceived(call)to Redux.audioListenerMiddlewaresubscribes to the calls slice and drives the player.audio.src = "/api/calls/" + call.id + "/audio",preload = "auto",play()oncanplay;ended/erroradvance the queue.audio.play().then(pause).catch(noop)on the persistent element inside the user-gesture handler so subsequent programmaticplay()calls succeed on mobile.Dropped code paths
decodeAudioDataand the in-memoryAudioBufferqueue.base64ToArrayBufferhelper and theaudiofield parse in the WS client.Bloballocation andURL.createObjectURLfor playback.playViaAudioElementWeb-Audio-failure fallback (now the only path)..catch(() => {})hack.Download / share
BookmarksPanelandSearchPaneldownload buttons are now plain<a href="/api/calls/:id/audio" download="…">anchors. The browser handles the streaming download with the cookie attached automatically.SharedCallpage uses<audio src="/api/shared/:token/audio">directly.Verification
pnpm exec tsc --noEmitclean.pnpm test --run— 188 / 188 pass across 20 suites.rg 'decodeAudioData|base64ToArrayBuffer' frontend/src— 0 hits.rg 'createObjectURL.*[Aa]udio|audioCallback|setAudioCallback' frontend/src— 0 hits.rg 'playViaAudioElement|EncodingError' frontend/src— 0 hits.Self-review
dangerouslySetInnerHTML. URLs are constructed from numericcall.id/ opaque share tokens.os_sessioncookie is httpOnly; the SPA never reads or writes it. No bearer header is added to the<audio>request — the cookie does the work.publicAccess=truedeployments still work. The server'sOptionalJWTOrSessionAuthfalls through to anonymous when no auth is presented, and the<audio>element will simply omit the cookie for unauthenticated sessions, which is exactly what public access expects.Files
8 files changed, +277 / −426. Net deletion: 149 LOC.