Skip to content

fix: honor ignore on enter for toolsets - #2011

Merged
toubatbrian merged 4 commits into
mainfrom
nestled-bushiest-woofing
Jul 15, 2026
Merged

fix: honor ignore on enter for toolsets#2011
toubatbrian merged 4 commits into
mainfrom
nestled-bushiest-woofing

Conversation

@rosetta-livekit-bot

@rosetta-livekit-bot rosetta-livekit-bot Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Ports livekit/agents#6361 to JS by honoring ToolFlag.IGNORE_ON_ENTER after flattening ToolContext, so ignored tools nested inside a Toolset are hidden during onEnter replies and nested tool-response follow-ups, then restored for later user turns.

Also adds ignoreOnEnter to createEndCallTool, matching the Python option for hiding the end-call tool during greetings.

Python parity

The final implementation matches the current Python lifecycle and cleanup behavior:

  • Pipeline replies filter a copied flattened ToolContext while preserving nested Toolset lifecycle and routing.
  • Realtime replies use a temporary flattened copy, leaving the provider's structured session context available for restoration.
  • Temporary realtime setup runs inside the restoration try/finally, so providers that mutate and then reject cannot strand the filtered context.
  • Tool-choice and tool-context restoration each use independent catch-and-log handling (failed to reset tool_choice / failed to reset tools), so one cleanup failure cannot skip the other or mask a primary generation error.
  • createEndCallTool({ ignoreOnEnter: true }) maps to ToolFlag.IGNORE_ON_ENTER; the default remains false.
Source diff coverage
Source file Classification Target coverage
livekit-agents/livekit/agents/beta/tools/end_call.py Adapted agents/src/beta/tools/end_call.ts: camelCase option and factory-function API.
livekit-agents/livekit/agents/llm/tool_context.py Ported agents/src/llm/tool_context.ts: flattened exclusion plus an internal flattened copy for temporary realtime updates.
livekit-agents/livekit/agents/voice/agent_activity.py Adapted agents/src/voice/agent_activity.ts: pipeline/realtime filtering, setup-failure restoration, and independent cleanup failure logging.
tests/test_agent_session.py Adapted Production-path pipeline and realtime AgentSession coverage for greeting, nested tool response, later-turn restoration, provider mutation/rejection, and cleanup failures.
tests/test_tools.py Ported/adapted _exclude behavior and createEndCallTool({ ignoreOnEnter: true }).

Production-path verification

The focused suite runs real AgentSession pipeline and realtime paths with ordinary, ignored, and two-level nested toolsets. It verifies:

  • Greeting and nested tool-response generations omit ignored tools.
  • A later user turn restores all ignored tools.
  • Realtime provider state is restored after a temporary update mutates and rejects.
  • A tool-choice cleanup throw does not skip tool restoration or replace a primary generation error.
  • A tool restoration throw is logged/contained and does not replace the primary error.
  • Successful-path cleanup failures remain contained without context leakage when restoration applies.

Final focused result on head 280e7bdbb78d196db1f9c82c8a2efaec2ca4ac12: 3 files passed, 86 tests passed.

Additional local checks passed:

  • pnpm build:agents.
  • pnpm build.
  • pnpm --filter @livekit/agents typecheck.
  • Changed-file eslint --quiet.
  • pnpm format:check.
  • git diff --check.

The existing full-agents run still has the unrelated AMD timing/classification failures, and API Extractor remains blocked by its existing unsupported generated export * as ___ syntax.

Cue voice E2E

Fresh voice verification was run at literal local/remote head 280e7bdbb78d196db1f9c82c8a2efaec2ca4ac12 after a successful full repository build.

  • Session: sid_6c92d326915e.
  • Mode/transport: voice mode over LiveKit transport.
  • Providers: LiveKit Inference with openai/gpt-4.1-mini, deepgram/nova-3, and cartesia/sonic-3.
  • Greeting: one tool batch executed exactly allowed_greeting_probe and nested allowed_nested_probe; ignored_restored_probe was absent. Cue then waited for the committed assistant message after playout.
  • Later real voiced user turn: persisted STT and USER events contain “Call the ignored restored probe now, then tell me it completed.” The agent executed ignored_restored_probe, then committed the assistant confirmation after playout.
  • Both command results resolved successfully with exit code 0.
  • Events: ~/.cue-cli/sessions/sid_6c92d326915e/events.jsonl — SHA-256 3cb7530fdef5468f17a42209dc48d064b77961f41cb2d976fa1da48ecf33eaf6.
  • Greeting result: recordings/001_run/result.json — SHA-256 2de039d446a27bc963609df92837808a27d74e0e1099446bc44bea585ae3136d.
  • Restored-turn result: recordings/002_run/result.json — SHA-256 563df770a12640a2da7be3062312e6859f6f84ad344d4e0f4e6131e2bf57c971.
  • Session WAV: stereo PCM 16-bit/48 kHz, 24.794 s, 4,760,400 bytes — SHA-256 2d17e9acf7f2ac6c952533efdc746574f54af9f29cee9c6b753f59151cd2c9ab.
  • Greeting WAV: stereo PCM 16-bit/48 kHz, 4.616 s, 886,232 bytes — SHA-256 9f5e2bbf92393c414844f22fb11c511c7ef05be718ad169717abf61f0445c0af.
  • Restored-turn WAV: stereo PCM 16-bit/48 kHz, 9.866 s, 1,894,364 bytes — SHA-256 917ec6d6535e47da6203a17f195e54c351a0d7bf081423579d307adc27cf9386.

The temporary nested-toolset harness and worker were removed after the persisted artifacts were inspected.

Changeset

The existing patch changeset for @livekit/agents is retained.


Ported from livekit/agents#6361

Original PR description

Closes #6354

ToolFlag.IGNORE_ON_ENTER was only honored for bare FunctionTool/RawFunctionTool instances in the on_enter reply filter — tools nested in a Toolset (e.g. EndCallTool) passed through unfiltered, so the model could fire a speculative end_call during the greeting turn (with delete_room=True, an instant hangup at pickup).

The filter now flattens toolsets and hides flagged tools via ToolContext._exclude, which drops them from the callable/LLM-visible set while keeping the owning toolset in place so executor routing and lifecycle are unaffected. Rather than threading the ignored set through the reply tasks, _on_enter_ignored_tools reads the on_enter contextvar directly; the reply task and its tool-response follow-ups inherit it through asyncio.create_task.

The realtime path is covered too: it only overrides its session tools when generate_reply is given tools=, so inside on_enter it now falls back to the agent's full set and hides flagged tools even without an explicit tools=.

EndCallTool gains an ignore_on_enter parameter (defaults to False, preserving current behavior) to opt into hiding end_call during the greeting.

@rosetta-livekit-bot
rosetta-livekit-bot Bot requested a review from a team as a code owner July 9, 2026 12:27
@changeset-bot

changeset-bot Bot commented Jul 9, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 280e7bd

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 36 packages
Name Type
@livekit/agents Patch
@livekit/agents-plugin-anam Patch
@livekit/agents-plugin-anthropic Patch
@livekit/agents-plugin-assemblyai Patch
@livekit/agents-plugin-baseten Patch
@livekit/agents-plugin-bey Patch
@livekit/agents-plugin-cartesia Patch
@livekit/agents-plugin-cerebras Patch
@livekit/agents-plugin-deepgram Patch
@livekit/agents-plugin-did Patch
@livekit/agents-plugin-elevenlabs Patch
@livekit/agents-plugin-fishaudio Patch
@livekit/agents-plugin-google Patch
@livekit/agents-plugin-hedra Patch
@livekit/agents-plugin-hume Patch
@livekit/agents-plugin-inworld Patch
@livekit/agents-plugin-lemonslice Patch
@livekit/agents-plugin-liveavatar Patch
@livekit/agents-plugin-livekit Patch
@livekit/agents-plugin-minimax Patch
@livekit/agents-plugin-mistral Patch
@livekit/agents-plugin-mistralai Patch
@livekit/agents-plugin-neuphonic Patch
@livekit/agents-plugin-openai Patch
@livekit/agents-plugin-perplexity Patch
@livekit/agents-plugin-phonic Patch
@livekit/agents-plugin-resemble Patch
@livekit/agents-plugin-rime Patch
@livekit/agents-plugin-runway Patch
@livekit/agents-plugin-sarvam Patch
@livekit/agents-plugin-silero Patch
@livekit/agents-plugin-soniox Patch
@livekit/agents-plugin-tavus Patch
@livekit/agents-plugins-test Patch
@livekit/agents-plugin-trugen Patch
@livekit/agents-plugin-xai Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@rosetta-livekit-bot
rosetta-livekit-bot Bot requested a review from longcw July 9, 2026 12:27

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

toubatbrian and others added 3 commits July 15, 2026 11:17
Exercise full pipeline and realtime sessions so nested on-enter filtering remains scoped through tool responses without leaking into later user turns.

Co-authored-by: Cursor <cursoragent@cursor.com>
Protect temporary realtime setup so providers that mutate before rejecting cannot leave ignored tools removed.

Co-authored-by: Cursor <cursoragent@cursor.com>
Restore tool choice and tools independently so cleanup errors cannot mask the primary generation failure or skip later cleanup.

Co-authored-by: Cursor <cursoragent@cursor.com>
@toubatbrian
toubatbrian merged commit bad25fd into main Jul 15, 2026
9 checks passed
@toubatbrian
toubatbrian deleted the nestled-bushiest-woofing branch July 15, 2026 20:51
@github-actions github-actions Bot mentioned this pull request Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant