Skip to content

Dedupe repeat reminders never reach the model for oversized tool results (>50K chars) #2177

Description

@vincentllm

Summary

The escalating repeat reminders injected by the tool-call deduplicator (streak 3/5/8, system-reminder texts) never reach the model when the tool result exceeds 50,000 characters. The reminder is appended at the tail of the tool output, and the oversized-result offload path keeps only a 2K head preview — so the reminder is silently discarded exactly in the large-output scenario where repeat loops are most likely. The only guardrail that still fires is the force stop at streak 12.

Both engines are affected (v1 agent-core and v2 agent-core-v2).

Environment

kimi-code 0.29.0, headless (kimi -p), default v1 harness — print mode routes to the v2 engine only when KIMI_CODE_EXPERIMENTAL_FLAG is truthy (apps/kimi-code/src/cli/experimental-v2.ts; the run had no experimental flags). Independently, the run's wire metadata protocol_version = "1.4" matches agent-core's AGENT_WIRE_PROTOCOL_VERSION = '1.4' (v2 writes '1.5'). Model: kimi-for-coding.

Observed behavior

Task: a 458KB / 5500-line log file with a marker string near the end; the model is asked to read the file and answer with the marker. The file exceeds the Read tool's 50KB limit, so answering requires paging.

Observed: 12 consecutive identical Read {"path":"big.log"} calls, zero assistant text, turn force-stopped after the 12th call. Session wire log shows:

  • Every tool result contained the plain hint next_step: Use Read with output_path to page through the full output. — ignored 12 times.
  • No tool result at streak 3/5/8/12 contained any dedupe system-reminder text.
  • Output per step was constant (~16 tokens = the repeated tool call); the turn ended with no final answer.

Control experiment (same task, same model, same endpoint, only the thinking effort varied):

group n doom loop answered correctly
thinking=off 5 + 1 calibration 6/6 0/6
thinking=low 5 0/5 5/5

One-sided Fisher exact test: formal replication off 5/5 vs low 0/5 → p = 1/252 = 0.00397 (descriptive, including the calibration run: p = 1/462 = 0.00216). Caveats: the two batches were not randomized/interleaved, and Fisher only tests the proportion difference — but with thinking enabled the model's own recorded thought explicitly planned pagination (…line offset near the end) and its first call was Read {"path":"big.log","line_offset":-100}.

So: with thinking off, the model does not paginate; the dedupe escalation that was intended to intervene in the loop is invisible; and the turn dies at the force stop. (Whether a visible reminder would actually have recovered the loop is exactly what a post-fix A/B should measure.)

Root cause (source-confirmed, commit 188c0fcb)

v1 (packages/agent-core):

  1. deduper.finalizeResult()appendReminder() appends the reminder to the end of the tool output — agent/turn/tool-dedup.ts L57-L75
  2. The finalizeToolResult hook then calls budgetToolResultForModel()agent/turn/index.ts L944-L976
  3. For results >50K chars it persists the full text and renders a header + text.slice(0, 2000) preview — agent/turn/tool-result-budget.ts L19-L36, L81

The reminder sits at character ~89K; the preview keeps characters 0-2K. The reminder never reaches the model (nor the wire log).

v2 (packages/agent-core-v2) has the same ordering: reminders are appended inside onDidExecuteTool (toolExecutorService.ts L585-L619) and truncateForModel runs after the hooks with the same head-only preview (toolResultTruncationService.ts L85-L104).

In both engines stopTurn is preserved across truncation, so the streak-12 force stop still fires — only the reminders are lost.

Minimal public reproduction

  1. Generate the log (any 5500-line file >50KB with markers at lines 1/2750/5450):
lines = ["2026-07-24T00:00:00Z INFO boot MARKER-START-ALPHA-9911"]
for i in range(2, 5501):
    if i == 2750:
        lines.append("2026-07-24T06:00:00Z WARN mid-point MARKER-MIDDLE-BETA-5522")
    elif i == 5450:
        lines.append("2026-07-24T11:59:50Z WARN shutdown imminent MARKER-END-OMEGA-7733")
    else:
        lines.append(f"2026-07-24T00:{i%60:02d}:00Z ERROR worker-{i%8} request failed code=E_L2_{i%20} retry=1 latency={i%1000}ms")
open("big.log", "w").write("\n".join(lines) + "\n")
  1. Prompt (verbatim): Read big.log and tell me: what is the marker string near the end of the file? Answer with the exact marker string.

  2. Thinking off (model config in the profile's config file): capabilities = ["thinking"], support_efforts = ["low", "high"], [thinking] effort = "off".

  3. Run: kimi -p "<prompt>" (headless, no experimental flags).

Trimmed wire excerpt (paths/ids abbreviated; full file available on request):

#1  llm.request  step=0.1  thinkingEffort=off
#1  tool.call    Read {"path": "big.log"}
#1  tool.result  "Tool output exceeded 50000 characters; showing a preview only.
                 ... next_step: Use Read with output_path to page through the full output.
                 [preview] 1\t...MARKER-START-ALPHA-9911 ..."   <- no system-reminder
#3  tool.call    Read {"path": "big.log"}     (identical args)
#3  tool.result  same preview body            <- still no system-reminder (expected: reminder 1)
...                                            (#4-#11 identical)
#12 tool.call    Read {"path": "big.log"}
#12 tool.result  same preview body            <- no reminder 3; turn force-stopped here,
                                                0 assistant text messages in the whole wire

Expected

Escalating reminders (streak 3/5/8) must be visible to the model regardless of result size, so a repeat loop can be recovered before the force stop.

Possible fixes

  1. Prepend the reminder ahead of the result output (smallest change; survives any head-only truncation) — implemented in the linked draft PR.
  2. Run the budget/offload step before dedupe finalize, so reminders are appended after truncation.
  3. Render the preview as head + tail instead of head-only.
  4. Return a hard error on cross-step repeats ("this call already returned a result; page via output_path").
  5. Make the truncation hint actionable: Read already supports paging parameters (e.g. line_offset, used successfully in the thinking=low run), but the hint does not say so — include the parameter names and a concrete paging example in the hint text.

Longer term, consider delivering control messages (like these reminders) out-of-band from raw tool data, so their visibility does not depend on text head/tail placement within the tool output.

A 2×2 A/B (thinking on/off × reminder truncated/visible) after the fix lands would quantify how much of the doom-loop prevention comes from the reminder itself.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions