Skip to content

Add Krisp Viva frame processor plugin - #2026

Closed
rosetta-livekit-bot[bot] wants to merge 1 commit into
mainfrom
snowiest-sunburn-arced
Closed

Add Krisp Viva frame processor plugin#2026
rosetta-livekit-bot[bot] wants to merge 1 commit into
mainfrom
snowiest-sunburn-arced

Conversation

@rosetta-livekit-bot

@rosetta-livekit-bot rosetta-livekit-bot Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add @livekit/agents-plugin-krisp with the Krisp VIVA FrameProcessor facade, auth providers, plugin registration, README, API report, and changeset.
  • Port license-mode adaptive frame-size buffering and the source regression tests to Vitest.
  • Update the basic agent example comments from legacy background voice cancellation to Krisp Viva.

Source diff coverage

File-by-file classification
  • examples/voice_agents/basic_agent.py: adapted to examples/src/basic_agent.ts; updated the target example comments/import guidance from BVC/noise-cancellation to Krisp Viva.
  • livekit-plugins/livekit-plugins-krisp/README.md: adapted to plugins/krisp/README.md; translated install/import/API examples to npm/TypeScript and documented the JS backend package/runtime-auth behavior.
  • livekit-plugins/livekit-plugins-krisp/examples/krisp_agent_example.py: not applicable; target has no per-plugin Krisp example directory counterpart. Parity is represented through README usage and basic_agent.ts, without inventing new examples.
  • livekit-plugins/livekit-plugins-krisp/examples/krisp_minimal_example.py: not applicable for the same reason: no target counterpart and no existing JS Krisp example to modify.
  • livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/__init__.py: ported to plugins/krisp/src/index.ts; exports the public processor/auth namespace and registers the plugin.
  • livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/_krisp.py: adapted to plugins/krisp/src/krisp.ts; ports license-mode frame buffering, mono passthrough, sample-rate adaptation, runtime noiseSuppressionLevel, and SDK lifecycle as far as the Node SDK backend permits.
  • livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/auth.py: ported to plugins/krisp/src/auth.ts; provides LiveKit Cloud and Krisp license auth providers plus JS aliases auth.livekitCloud / auth.krispLicense.
  • livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/krisp_instance.py: not applicable as a deleted source file; its responsibilities are replaced by the new facade/license implementation in plugins/krisp/src/krisp.ts.
  • livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_filter.py: adapted to plugins/krisp/src/viva_filter.ts; ports provider resolution, default LiveKit Cloud backend selection, deprecated modelPath / sampleRate / frameDurationMs handling, forwarding hooks, enabled state, close, and runtime suppression-level APIs.
  • livekit-plugins/livekit-plugins-krisp/pyproject.toml: adapted to plugins/krisp/package.json, tsconfig.json, tsup.config.ts, and API Extractor config; creates the JS package with equivalent workspace metadata.
  • pyproject.toml: not applicable; source-only Python workspace/pytest formatting changes have no target counterpart. Target workspace already includes plugins/*.
  • tests/test_krisp_frame_buffer.py: ported to plugins/krisp/src/krisp.test.ts; translates the buffering regression tests to Vitest without adding unrelated tests.
  • uv.lock: not applicable; Python lockfile changes map to target package.json / pnpm-lock.yaml updates only as needed by the new JS package.

Backend gap

The Python source PR depends on published Python backend wheels (livekit-plugins-krisp-internal for LiveKit Cloud and krisp_audio for license mode). I could not find an equivalent published Node backend package in this target repo or npm. This PR ports the public JS facade, provider resolution, FrameProcessor lifecycle hooks, and license-mode buffering against expected Node module shapes, but real runtime cloud/license audio processing requires those Node backend packages to exist:

  • LiveKit Cloud path dynamically loads @livekit/agents-plugin-krisp-internal and expects it to export KrispVivaFilterFrameProcessor.
  • License path dynamically loads krisp-audio and expects a Krisp SDK API equivalent to the Python SDK.

Without those backend packages installed, constructors throw explicit runtime errors rather than silently bypassing noise reduction.

Verification

  • pnpm test -- plugins/krisp: passed.
  • pnpm --filter @livekit/agents-plugin-krisp lint: passed.
  • pnpm --filter @livekit/agents-plugin-krisp... build: passed.
  • pnpm --filter @livekit/agents-plugin-krisp api:check: passed.
  • pnpm --filter livekit-agents-examples... build: passed.
  • pnpm --filter livekit-agents-examples lint: completed with pre-existing warnings in examples/src/testing/survey_agent.test.ts.
  • pnpm lint: passed with existing warnings across the repo.
  • pnpm exec turbo run build --concurrency=1: passed. A prior plain pnpm build attempt was killed with exit 137 while building unrelated packages in parallel.
  • pnpm --filter livekit-agents-examples test: failed in existing example tests unrelated to the Krisp changes (src/drive-thru/test_agent.test.ts, src/testing/agent_task.test.ts, and unhandled FakeLLM errors reported from src/testing/survey_agent.test.ts).

Ported from livekit/agents#5914

Original PR description

simplified default usage for cloud users now looks like

  await session.start(
        agent=MyAgent(),
        room=ctx.room,
        room_options=room_io.RoomOptions(
            audio_input=room_io.AudioInputOptions(
                noise_cancellation=krisp.KrispVivaFilterFrameProcessor(noise_suppression_level=90)
            ),
        ),
    )

@rosetta-livekit-bot
rosetta-livekit-bot Bot requested a review from a team as a code owner July 13, 2026 09:26
@changeset-bot

changeset-bot Bot commented Jul 13, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: a81e470

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 37 packages
Name Type
@livekit/agents-plugin-krisp Patch
@livekit/agents Patch
@livekit/agents-plugin-anam Patch
@livekit/agents-plugin-anthropic Patch
@livekit/agents-plugin-assemblyai Patch
@livekit/agents-plugin-baseten Patch
@livekit/agents-plugin-bey Patch
@livekit/agents-plugin-cartesia Patch
@livekit/agents-plugin-cerebras Patch
@livekit/agents-plugin-deepgram Patch
@livekit/agents-plugin-did Patch
@livekit/agents-plugin-elevenlabs Patch
@livekit/agents-plugin-fishaudio Patch
@livekit/agents-plugin-google Patch
@livekit/agents-plugin-hedra Patch
@livekit/agents-plugin-hume Patch
@livekit/agents-plugin-inworld Patch
@livekit/agents-plugin-lemonslice Patch
@livekit/agents-plugin-liveavatar Patch
@livekit/agents-plugin-livekit Patch
@livekit/agents-plugin-minimax Patch
@livekit/agents-plugin-mistral Patch
@livekit/agents-plugin-mistralai Patch
@livekit/agents-plugin-neuphonic Patch
@livekit/agents-plugin-openai Patch
@livekit/agents-plugin-perplexity Patch
@livekit/agents-plugin-phonic Patch
@livekit/agents-plugin-resemble Patch
@livekit/agents-plugin-rime Patch
@livekit/agents-plugin-runway Patch
@livekit/agents-plugin-sarvam Patch
@livekit/agents-plugin-silero Patch
@livekit/agents-plugin-soniox Patch
@livekit/agents-plugin-tavus Patch
@livekit/agents-plugin-trugen Patch
@livekit/agents-plugin-xai Patch
@livekit/agents-plugins-test Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@rosetta-livekit-bot
rosetta-livekit-bot Bot requested a review from lukasIO July 13, 2026 09:27

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

View 1 additional finding in Devin Review.

Open in Devin Review

Comment on lines +337 to +341
close(): void {
this.session = null;
this.inBuf = new Int16Array(0);
this.outBuf = new Int16Array(0);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Native SDK resource is never released when the processor is closed

The native Krisp SDK is acquired but never released when the processor is closed (close() at plugins/krisp/src/krisp.ts:337-341), because the release logic lives in a separate destroy() method that nothing ever calls, so the SDK leaks for the lifetime of the process.

Impact: The native Krisp Audio SDK global state is never cleaned up, leaking native resources each time a session ends.

SDK acquire/release lifecycle mismatch

When using the Krisp license auth path, KrispLicenseSDKManager.acquire() is called during KrispLicenseFrameProcessor.create() at plugins/krisp/src/krisp.ts:207, incrementing a reference count. The matching KrispLicenseSDKManager.release() call exists only in destroy() at plugins/krisp/src/krisp.ts:343-349.

However, the framework's teardown path calls close(), not destroy():

  • agents/src/voice/room_io/_input.ts:178 calls this.frameProcessor?.close()
  • plugins/krisp/src/viva_filter.ts:211-213 (the facade) calls this.inner?.close()
  • plugins/krisp/src/krisp.ts:337-341 (close()) only nulls the session and buffers — it never calls KrispLicenseSDKManager.release()

The destroy() method is not part of the FrameProcessor base class interface and is never invoked by any caller in the codebase. As a result, referenceCount never decrements and globalDestroy() is never called.

Suggested change
close(): void {
this.session = null;
this.inBuf = new Int16Array(0);
this.outBuf = new Int16Array(0);
}
close(): void {
this.session = null;
this.inBuf = new Int16Array(0);
this.outBuf = new Int16Array(0);
if (this.sdkAcquired) {
KrispLicenseSDKManager.release();
this.sdkAcquired = false;
}
}
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@lukasIO lukasIO closed this Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant