fix(voice): don't strand playback-segment counter on interrupted-while-paused frame

[enriqueespaillat-gyde]

Summary

ParticipantAudioOutput.captureFrame can permanently desync its playback-segment counter when a frame arrives while the output is paused (interruption detection) and is then interrupted. Afterwards every waitForPlayout() on that output blocks forever, which can hang the agent.

Root cause

captureFrame calls super.captureFrame(frame) — which bumps the base AudioOutput's playbackSegmentsCount — before the pause/interrupt gate:

super.captureFrame(frame);            // playbackSegmentsCount++

if (!this.playbackEnabledFuture.done) {
  this.audioSource.clearQueue();
  await Promise.race([this.playbackEnabledFuture.await, this.interruptedFuture.await]);
  if (this.interruptedFuture.done) {
    return;                           // pushedDuration is never incremented
  }
}

When a frame bails at that early return (interrupted while paused), the segment was already counted, but:

• pushedDuration is never incremented, so
• flush() short-circuits on if (!this.pushedDuration) return and never starts waitForPlayoutTask, so
• onPlaybackFinished is never called and playbackFinishedCount never catches up.

playbackSegmentsCount is now permanently ahead of playbackFinishedCount, so the next segment's waitForPlayout() (while (playbackFinishedCount < playbackSegmentsCount) await ...) never resolves.

Why it can hang the agent

The stalled turn's reply pipeline awaits audioOutput.waitForPlayout() and so never reaches _markGenerationDone(). Because the speech handle is also never interrupted, AgentActivity.mainTask's await speechHandle.waitIfNotInterrupted([speechHandle._waitForGeneration()]) has neither of its two exit conditions and parks forever — the agent goes silent until the participant disconnects. This is easiest to hit when a tool-call-only turn (no spoken text) immediately follows a barge-in. (Same hang surface as the mainTask issues #836 / #1089, but a distinct root cause in the audio output.)

Fix

Register the playback segment (super.captureFrame) only after the pause/interrupt gate is cleared, so a frame that bails never counts a segment.

Test

Adds a ParticipantAudioOutput captureFrame segment accounting suite to _output.test.ts:

• a frame interrupted while paused must not register a segment, and a following waitForPlayout() resolves (this fails before the fix — playbackSegmentsCount is left at 1 and waitForPlayout() hangs);
• the normal (non-paused) path still registers and pushes the frame.

[changeset-bot]

🦋 Changeset detected

Latest commit: abf1c8c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 33 packages

Name	Type
@livekit/agents	Patch
@livekit/agents-plugin-anam	Patch
@livekit/agents-plugin-assemblyai	Patch
@livekit/agents-plugin-baseten	Patch
@livekit/agents-plugin-bey	Patch
@livekit/agents-plugin-cartesia	Patch
@livekit/agents-plugin-cerebras	Patch
@livekit/agents-plugin-deepgram	Patch
@livekit/agents-plugin-elevenlabs	Patch
@livekit/agents-plugin-fishaudio	Patch
@livekit/agents-plugin-google	Patch
@livekit/agents-plugin-hedra	Patch
@livekit/agents-plugin-hume	Patch
@livekit/agents-plugin-inworld	Patch
@livekit/agents-plugin-lemonslice	Patch
@livekit/agents-plugin-liveavatar	Patch
@livekit/agents-plugin-livekit	Patch
@livekit/agents-plugin-minimax	Patch
@livekit/agents-plugin-mistral	Patch
@livekit/agents-plugin-mistralai	Patch
@livekit/agents-plugin-neuphonic	Patch
@livekit/agents-plugin-openai	Patch
@livekit/agents-plugin-perplexity	Patch
@livekit/agents-plugin-phonic	Patch
@livekit/agents-plugin-resemble	Patch
@livekit/agents-plugin-rime	Patch
@livekit/agents-plugin-runway	Patch
@livekit/agents-plugin-sarvam	Patch
@livekit/agents-plugin-silero	Patch
@livekit/agents-plugin-tavus	Patch
@livekit/agents-plugins-test	Patch
@livekit/agents-plugin-trugen	Patch
@livekit/agents-plugin-xai	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 3 additional findings.

[enriqueespaillat-gyde]

@toubatbrian this is an issue we run into often, im not entirely confident on the fix. Essentially causes the agent to hang if interrupted mid tool call.

[toubatbrian]

Essentially causes the agent to hang if interrupted mid tool call.

@enriqueespaillat-gyde Just to clarify, is this before or after this fix? From my local testing on your PR, I think this is the right fix.

[enriqueespaillat-gyde]

Essentially causes the agent to hang if interrupted mid tool call.

@enriqueespaillat-gyde Just to clarify, is this before or after this fix? From my local testing on your PR, I think this is the right fix.

I think the only problem for us - is that it doesn't fully fix the issue. I think I got closer to the root cause here - #1732

I'm less confident on the right fix though.