Skip to content

feat(realtime): configurable pipeline.max_history_items - #10331

Merged
mudler merged 1 commit into
masterfrom
feat/realtime-max-history-config
Jun 14, 2026
Merged

feat(realtime): configurable pipeline.max_history_items#10331
mudler merged 1 commit into
masterfrom
feat/realtime-max-history-config

Conversation

@localai-bot

Copy link
Copy Markdown
Collaborator

Problem

Composed realtime pipelines (VAD+STT+LLM+TTS) get MaxHistoryItems = 0 (unlimited) by default — only self-contained any-to-any models (liquid-audio) get the 6-item cap. So a long-running pipeline session grows every turn and feeds the entire conversation to the LLM until its context window fills. There's no way to cap it from config.

Fix

Add an optional pipeline.max_history_items to the model config. resolveMaxHistoryItems uses the explicit value when set (including 0 = unlimited), otherwise falls back to defaultMaxHistoryItems (6 for realtime-audio, 0 for legacy pipelines). Wired into session init; the existing trimRealtimeItems does the trimming.

name: gpt-realtime
pipeline:
  vad: silero-vad-ggml
  transcription: parakeet-cpp-tdt-0.6b-v3
  llm: gemma-4-e2b-it-qat-q4_0
  tts: vits-piper-it_IT-paola-sherpa
  max_history_items: 20   # cap trailing items fed to the LLM per turn

Tests cover the resolver (explicit value, explicit 0, fallback-to-default, nil). Existing defaultMaxHistoryItems/trimRealtimeItems tests unchanged.

🤖 Generated with Claude Code

Composed realtime pipelines (VAD+STT+LLM+TTS) defaulted to unlimited history,
so a long-running session grew every turn and fed the whole conversation to the
LLM until its context window filled. Add an optional pipeline.max_history_items
to cap the trailing items per turn; explicit value (including 0=unlimited) wins
over the per-model-type default. Self-contained any-to-any models keep their
6-item default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mudler
mudler merged commit 7d2a762 into master Jun 14, 2026
55 of 59 checks passed
@mudler
mudler deleted the feat/realtime-max-history-config branch June 14, 2026 16:13
@localai-bot localai-bot added the enhancement New feature or request label Jun 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants