feat(vad): bundle optimized silero vad and deprecate the plugin

[chenghao-mou]

Why

Silero VAD is the default endpointing implementation for voice agents, but lived behind a separate livekit-plugins-silero install step. That extra hop made the standard quickstart longer than it needed to be, and the plugin's onnxruntime-based loader paid the full model load cost in every job process (no fork-time sharing).

This PR moves Silero VAD into livekit-agents core, backed by livekit-local-inference. The plugin stays installable as a deprecated shim until v2.0, and existing call sites continue to work — they transparently route to the new implementation when settings are compatible.

This PR also introduces changes to follow the official silero settings, similar to #5788:

• removed exp filter
• changed the default min_silence_duration from 0.55s to 0.1s.

Code example

Before

from livekit.agents import Agent, AgentSession, JobContext, JobProcess, WorkerOptions, cli
from livekit.plugins import deepgram, openai, silero


def prewarm(proc: JobProcess) -> None:
    # Heavy ONNX session construction — must live behind prewarm so each
    # job process doesn't pay the load on every conversation start.
    proc.userdata["vad"] = silero.VAD.load()


async def entrypoint(ctx: JobContext) -> None:
    session = AgentSession(
        vad=ctx.proc.userdata["vad"],
        stt=deepgram.STT(),
        llm=openai.LLM(),
        tts=openai.TTS(),
    )
    await session.start(agent=Agent(instructions="..."), room=ctx.room)


if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint, prewarm_fnc=prewarm))

After

from livekit.agents import Agent, AgentSession, JobContext, WorkerOptions, cli, inference
from livekit.plugins import deepgram, openai


async def entrypoint(ctx: JobContext) -> None:
    session = AgentSession(
        vad=inference.VAD(model="silero"),
        stt=deepgram.STT(),
        llm=openai.LLM(),
        tts=openai.TTS(),
    )
    await session.start(agent=Agent(instructions="..."), room=ctx.room)


if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))

No prewarm_fnc, no silero plugin import, no proc.userdata shuttle. Weights are loaded once in the forkserver and inherited by every job process via COW.

API change

Before	After
pip install livekit-agents livekit-plugins-silero	pip install livekit-agents — Silero is bundled
from livekit.plugins.silero import VAD vad = VAD.load(min_silence_duration=0.4)	from livekit.agents import inference vad = inference.VAD(model="silero", min_silence_duration=0.4)
silero.VAD.load() did a heavy onnxruntime session construction → expected to live behind a prewarm hook	inference.VAD(model="silero") is a cheap wrapper; weights are loaded once at forkserver-preload time, inherited via COW
Per-job: ~6 MB Silero ONNX loaded into every job process	Per-fork: weights resident in the forkserver, COW-shared with each job (Linux); spawn platforms unchanged
Plugin owned its own VAD/VADStream/OnnxModel (~650 LOC)	Core owns the wrapper; plugin keeps a frozen copy as a deprecation shim
silero.VAD.load(force_cpu=False, sample_rate=16000) ran onnxruntime; with custom onnx_file_path, used a user-supplied model	silero.VAD.load(...) transparently delegates to inference.VAD(model="silero", ...) when settings are compatible; 8 kHz + onnx_file_path still routes to the legacy onnxruntime path
vad: NotGivenOr[vad.VAD] = NOT_GIVEN in AgentSession.__init__ — vad=None was illegal per type, even though the code accepted it	vad: NotGivenOr[vad.VAD | None] = NOT_GIVEN — vad=None now type-legal as an explicit "no VAD" signal
No way to invoke Silero VAD without importing the plugin	from livekit.agents.inference import VAD

Behaviour change

Before	After
Worker startup: each forked job pays the full Silero ONNX load (~tens of ms)	Forkserver preload runs livekit.agents.inference._warmup once → init_vad() + init_eot() page native weights into the forkserver. Jobs fork with weights already resident.
import livekit.plugins.silero — silent	Emits a single DeprecationWarning pointing to inference.VAD(model="silero"); v2.0 removal target
silero.VAD.load(force_cpu=False) honored the user's GPU request via onnxruntime	When delegating to native, force_cpu=False is ignored (native lib is CPU-only) → now emits a WARNING explaining the kwarg is ignored and pointing at onnx_file_path as the legacy escape hatch
AgentSession with default args → session.vad is None	No change on this branch — default stays None.

Migration

If you currently use…	Do nothing?	Recommended update
silero.VAD.load() with default settings	Works (delegates) — deprecation warning prints once at plugin import	from livekit.agents import inference; inference.VAD(model="silero")
silero.VAD.load(sample_rate=8000)	Works — still routes to legacy onnxruntime	No native equivalent; stay on plugin until v2.0 then migrate if 16 kHz is acceptable
silero.VAD.load(onnx_file_path=...)	Works — still routes to legacy onnxruntime	Stay on plugin until v2.0; the bundled native model is fixed
silero.VAD.load(force_cpu=False)	Works but a warning now fires noting force_cpu is ignored on the delegated native path	Use inference.VAD(model="silero") (and accept CPU) or keep the plugin form with onnx_file_path=... to keep the legacy path

[theomonnom]

Should we follow the same defaults as Silero? IMO we should use 0.1 now. It shouldn't have any side effect and it is closer to the truth

[chenghao-mou]

I'd love to, but I wasn't sure if you are going to merge #5788 first.

[theomonnom]

just closed mine

[longcw]

tested locally and it works well. could you add a comparison in the pr description?

[devin-ai-integration]

Devin Review found 1 new potential issue.

View 13 additional findings in Devin Review.

[devin-ai-integration]

🟡 Auto-created VAD for Speechmatics STT uses drastically lower min_silence_duration default (0.1s vs 0.55s)

In _resolve_vad_for_model, the auto-created VAD for Speechmatics STT models changed from SileroVAD.load() (which had min_silence_duration=0.55) to inference.VAD(model="silero") (which defaults to min_silence_duration=0.1). This 5.5× reduction means END_OF_SPEECH events fire much sooner for Speechmatics STT users who don't provide their own VAD. While the audio EOT detector (_maybe_apply_vad_silence_override in livekit-agents/livekit/agents/voice/audio_recognition.py:669) bumps this to at least 0.25s when the audio turn detector is active, users with text-based turn detectors (e.g., MultilingualModel) will experience the full 0.1s default — significantly different from the previous 0.55s behavior.

Suggested change

	from .vad import VAD
	vad_instance = VAD(model="silero")
	from .vad import VAD
	vad_instance = VAD(model="silero", min_silence_duration=0.55)

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

Devin Review found 1 new potential issue.

View 15 additional findings in Devin Review.

[devin-ai-integration]

🟡 Duplicate livekit-local-inference dependency entry in pyproject.toml

This PR adds a second "livekit-local-inference>=0.2.5" entry on line 37, while one already exists on line 34. While most Python build tools deduplicate dependencies gracefully, this is clearly unintentional and could cause confusion or subtle issues with tooling that doesn't handle duplicates.

Suggested change

"livekit-local-inference>=0.2.5",

---

Was this helpful? React with 👍 or 👎 to provide feedback.