In the AgentCall class, we currently wait for the counterparty to complete their part of the dialog before starting STT, the LLM, and finally TTS.
A better architecture would stream STT and buffer output text before running inference.
We would start inference at a 250 ms speech gap but cancel (debounce) the inference run if the counterparty starts talking again. Finally, if the counterparty did a 500 ms break, we would run TTS and stream the audio directly to RTP.
In the
AgentCallclass, we currently wait for the counterparty to complete their part of the dialog before starting STT, the LLM, and finally TTS.A better architecture would stream STT and buffer output text before running inference.
We would start inference at a 250 ms speech gap but cancel (debounce) the inference run if the counterparty starts talking again. Finally, if the counterparty did a 500 ms break, we would run TTS and stream the audio directly to RTP.