Fix truncated translate-and-adapt; route through shared Haiku client#660
Merged
Conversation
User-submitted articles (e.g. a fanfiction chapter, ~4k words) failed to
translate-and-adapt: the LLM output was capped at max_tokens=4000 (Anthropic)
/ 8000 (DeepSeek), too small for a full chapter's translation, which runs at
least as long as the input (German et al. 20-30% longer). The reply was cut
off mid-JSON and failed to parse ("Unterminated string"), surfacing as a
generic failure.
- Raise the Anthropic cap to 16000 (Haiku 4.5 ceiling is 64k) and route the
call through the shared haiku_client instead of a duplicated hand-rolled
POST, so model (models.SIMPLIFICATION), key, and endpoint live in one place.
- Detect truncation in haiku_completion (stop_reason "max_tokens" -> None,
fail-soft) so a cut-off reply becomes a clean fallback/failure instead of
unparseable JSON. Left haiku_completion_or_raise (batch crawl path)
unchanged to avoid shrinking simplified-article inventory.
- DeepSeek: raise to its 8192 ceiling and detect finish_reason "length".
- Endpoint returns 422 with an actionable message (try a single chapter)
instead of an opaque 500.
Chunking long works into per-chapter LLM calls is the durable fix, tracked
separately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
ArchLens - No architecturally relevant changes to the existing views |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A user tried to import a fanfiction chapter (
The Changing of the Guard, ~4k words / 24k chars) via translate-and-adapt (English → German B2) and it failed. Server logs showed the Anthropic call returningError parsing Anthropic JSON: Unterminated string— the LLM reply was cut off atmax_tokensmid-JSON, then fell back to DeepSeek (also capped) and produced a broken/incomplete article.Root cause: not a length limit and not abuse — the output caps (
max_tokens=4000Anthropic,8000DeepSeek) were too small for a full chapter's translation, which runs at least as long as the input. There was no truncation detection, so a cut-off reply became unparseable JSON.Changes
haiku_clientinstead of a duplicated hand-rolledrequests.post— model (models.SIMPLIFICATION), key, endpoint, and truncation handling now live in one place. Removes ~40 lines of duplication.haiku_completion(stop_reason == "max_tokens"→None, fail-soft) so a cut-off reply becomes a clean fallback/failure instead of garbage JSON. Lefthaiku_completion_or_raise(the batch crawl path) unchanged on purpose — long crawled articles can legitimately hit the cap there, and raising would shrink simplified-article inventory.finish_reason == "length".500 "Translation failed".The reported chapter should now translate on the primary Anthropic path (16k leaves ample room). The truncation detection is the safety net for genuinely huge single chapters.
Not included / follow-up
Testing
Import + wiring smoke-tested (
HAIKU_MODELresolves viamodels.ANTHROPIC_HAIKU). No automated tests cover these methods (they call live LLM APIs). Recommend re-importing the fanfiction URL to confirm end-to-end.🤖 Generated with Claude Code