Skip to content

Fix truncated translate-and-adapt; route through shared Haiku client#660

Merged
mircealungu merged 1 commit into
masterfrom
wt/translate-truncation
Jul 2, 2026
Merged

Fix truncated translate-and-adapt; route through shared Haiku client#660
mircealungu merged 1 commit into
masterfrom
wt/translate-truncation

Conversation

@mircealungu

Copy link
Copy Markdown
Member

Problem

A user tried to import a fanfiction chapter (The Changing of the Guard, ~4k words / 24k chars) via translate-and-adapt (English → German B2) and it failed. Server logs showed the Anthropic call returning Error parsing Anthropic JSON: Unterminated string — the LLM reply was cut off at max_tokens mid-JSON, then fell back to DeepSeek (also capped) and produced a broken/incomplete article.

Root cause: not a length limit and not abuse — the output caps (max_tokens=4000 Anthropic, 8000 DeepSeek) were too small for a full chapter's translation, which runs at least as long as the input. There was no truncation detection, so a cut-off reply became unparseable JSON.

Changes

  • Raise the Anthropic cap to 16000 (Haiku 4.5's output ceiling is 64k) and route the call through the shared haiku_client instead of a duplicated hand-rolled requests.post — model (models.SIMPLIFICATION), key, endpoint, and truncation handling now live in one place. Removes ~40 lines of duplication.
  • Detect truncation in haiku_completion (stop_reason == "max_tokens"None, fail-soft) so a cut-off reply becomes a clean fallback/failure instead of garbage JSON. Left haiku_completion_or_raise (the batch crawl path) unchanged on purpose — long crawled articles can legitimately hit the cap there, and raising would shrink simplified-article inventory.
  • DeepSeek: raise to its 8192 ceiling (16000 would 400) and detect finish_reason == "length".
  • Endpoint: return 422 with an actionable message ("try importing a single chapter") instead of an opaque 500 "Translation failed".

The reported chapter should now translate on the primary Anthropic path (16k leaves ample room). The truncation detection is the safety net for genuinely huge single chapters.

Not included / follow-up

  • Chunking long works into per-chapter LLM calls is the durable fix for arbitrarily long content — tracked separately.
  • A weekly per-user character cooldown (the original idea) is not needed: this was a truncation bug on a normal-sized chapter, not a cost/abuse problem. It pairs naturally with chunking later, if wanted.

Testing

Import + wiring smoke-tested (HAIKU_MODEL resolves via models.ANTHROPIC_HAIKU). No automated tests cover these methods (they call live LLM APIs). Recommend re-importing the fanfiction URL to confirm end-to-end.

🤖 Generated with Claude Code

User-submitted articles (e.g. a fanfiction chapter, ~4k words) failed to
translate-and-adapt: the LLM output was capped at max_tokens=4000 (Anthropic)
/ 8000 (DeepSeek), too small for a full chapter's translation, which runs at
least as long as the input (German et al. 20-30% longer). The reply was cut
off mid-JSON and failed to parse ("Unterminated string"), surfacing as a
generic failure.

- Raise the Anthropic cap to 16000 (Haiku 4.5 ceiling is 64k) and route the
  call through the shared haiku_client instead of a duplicated hand-rolled
  POST, so model (models.SIMPLIFICATION), key, and endpoint live in one place.
- Detect truncation in haiku_completion (stop_reason "max_tokens" -> None,
  fail-soft) so a cut-off reply becomes a clean fallback/failure instead of
  unparseable JSON. Left haiku_completion_or_raise (batch crawl path)
  unchanged to avoid shrinking simplified-article inventory.
- DeepSeek: raise to its 8192 ceiling and detect finish_reason "length".
- Endpoint returns 422 with an actionable message (try a single chapter)
  instead of an opaque 500.

Chunking long works into per-chapter LLM calls is the durable fix, tracked
separately.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mircealungu mircealungu merged commit 4c8fd32 into master Jul 2, 2026
1 of 3 checks passed
@mircealungu mircealungu deleted the wt/translate-truncation branch July 2, 2026 17:47
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

ArchLens - No architecturally relevant changes to the existing views

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant