Fix hollow-response loop in claude-cli backend (re-targeted to v8, clean diff, with tests)

[christophepub]

Re-opens #1062 with the maintainer feedback applied:

• ✅ Re-targeted to v8 (fresh branch off upstream/v8, no stale main noise)
• ✅ Diff limited to graphify/llm.py (1 file changed in the source)
• ✅ Unit tests added in tests/test_llm_parser.py covering the four parser failure modes + argv-shape assertions

Closing #1062. Same scope, clean diff.

Problem

graphify extract --backend claude-cli against a multi-modal corpus comes back with ~30-50% of semantic chunks flagged as hollow responses and triggers adaptive bisection. On an 800-file repo this turned a ~15 min run into ~44 min and consumed 2-3× more claude subscription quota than necessary.

Symptoms in the log:

[graphify] LLM returned invalid JSON, skipping chunk: Expecting value: line 1 column 1 (char 0)
[graphify] claude-cli returned a hollow response; treating as truncation so adaptive retry can bisect the chunk.
[graphify] chunk of 39 truncated at depth 0, splitting into halves of 19 and 20

The hollow-detection path works as designed — the issue is what causes Claude to return content that fails json.loads.

Root cause (two compounding bugs)

1. _parse_llm_json only strips fences at offset 0

if raw.startswith("```"):
    raw = raw.split("```", 2)[1]
    ...

Claude (and most chat models) frequently prepends a short preamble before the JSON:

Here are the extracted entities:

```json
{"nodes": [...], "edges": [...]}
```

raw.startswith("```") returns False, the fence-stripping is skipped entirely, json.loads fails on the preamble text, the chunk is dropped, the hollow detector re-routes it to bisection. Each bisected half is another claude -p call that may exhibit the same failure. Cost compounds.

2. _call_claude_cli uses --append-system-prompt

--append-system-prompt adds graphify's extraction prompt on top of Claude Code's default interactive-agent system prompt ("use markdown formatting", "output text to communicate with the user"). These conflict with graphify's "return raw JSON only" instruction, and the default prompt wins about half the time — producing the preambles and markdown fences from issue 1.

The claude CLI exposes --system-prompt (replace) since at least 2.1.x, which is the right primitive for a headless extraction backend.

Fix

Three complementary changes in graphify/llm.py:

1. Robust JSON extraction in _parse_llm_json — strips fences regardless of position, with a balanced-brace fallback that scans for the first complete {...} object anywhere in the response. Handles preambles, trailing prose, and prose-wrapped JSON without fences. Diagnostic log on terminal failure includes the first 200 chars so future format drift is debuggable.

2. Switch claude-cli to --system-prompt — eliminates the conflict at the source. Claude receives only graphify's extraction prompt and returns clean JSON on the first call. Side benefit: cache-creation tokens per call drop ~19% (47k vs 58k in my measurements) because Claude Code's default system prompt is no longer materialized.

3. GRAPHIFY_CLAUDE_CLI_MODEL env var — claude-cli defaults to Opus, which is overkill for the structured JSON extraction graphify performs. Setting GRAPHIFY_CLAUDE_CLI_MODEL=haiku lets users cut quota usage 3-5× for the semantic pass. Default behaviour unchanged when the env var is unset.

The three fixes are complementary: 2 dramatically reduces the rate of malformed responses; 1 keeps graphify robust against the residual cases (soft refusals, model confusion) and benefits every other backend too; 3 unlocks cheaper builds, which is only safe because 1+2 make Haiku's more frequent markdown-wrapping recoverable.

Tests (tests/test_llm_parser.py, 10 cases)

Parser:

• test_preamble_then_fence_is_parsed — the primary bug
• test_prose_wrapped_json_without_fence_is_parsed — balanced-brace fallback
• test_raw_json_still_works — regression check on the happy path
• test_total_refusal_returns_empty_fragment — graceful degradation
• test_fence_with_uppercase_language_tag — ```JSON
• test_fence_without_closing_backticks — truncation case
• test_empty_response_returns_empty_fragment

argv shape (mocked subprocess, same pattern as the existing test_claude_cli_backend.py):

• test_uses_system_prompt_not_append
• test_model_env_var_adds_model_flag
• test_no_model_flag_when_env_var_unset

19/19 tests pass (9 pre-existing in test_claude_cli_backend.py + 10 new)

Evidence

Test run on a 43-file modes/ directory (Markdown docs):

Metric	Before	After
Hollow responses	~30-50% of chunks	0
Bisections triggered	several per run	0
Output tokens	inflated by preambles	clean
Cache-creation tokens / call	~58k	~47k (-19%)

Validated end-to-end on an 800-file repo (mixed code + docs):

Metric	Before	After
Wall time	44 min	4 min (incremental)
Output tokens	269k	19k (-93%)
Final graph	4 248 nodes	4 279 nodes (+31 previously silently dropped)

GRAPHIFY_CLAUDE_CLI_MODEL=haiku also validated on a small corpus: graph structure identical (118 nodes / 193 edges vs Opus baseline 118 / 192), output tokens -82%.

Trade-offs

• --system-prompt replaces Claude Code's default prompt entirely. For the -p headless extraction use case this is desirable. Subscription auth is unaffected (verified).
• The balanced-brace scanner in strategy 2 is O(n) and only runs if both json.loads(stripped) and fence-stripping fail — no perf impact on the common path.
• --no-session-persistence is unchanged. A follow-up could explore session reuse to reclaim more cache budget, but that's orthogonal.