bug: `cora scan` aborts with "failed to parse LLM JSON response" on large batches

[ajianaz]

Bug

cora scan fails with Error: failed to parse LLM JSON response: expected value at line 1 column 1 when a scan batch is sent to the LLM. The failure occurs mid-scan (after "Reviewing (batch N/M)…") and aborts the entire run with no per-file results.

Steps to reproduce

1. Point cora at any working OpenAI-compatible provider (reproduced with two: freemodel/gpt-5.4 default in auth.toml, and zai/glm-5.1 via --provider zai --model glm-5.1).
2. Run a full-project scan with enough files to trigger batching:

cora scan --format compact

Repo: codecoradev/cira (SvelteKit + Rust monorepo, 98 files detected).

Expected behavior

Scan completes (or skips the failing batch) and emits findings for reviewed files.

Actual behavior

Scan aborts on the first non-trivial batch:

🔍 98 files to review…
  Reviewing (batch 1/13)…
  Reviewing (batch 2/13)…
Error: failed to parse LLM JSON response: expected value at line 1 column 1

Exit non-zero. No SARIF/compact output written. --verbose adds no detail about which response failed to parse.

Environment

• cora: 0.6.0
• OS: macOS 15 (Darwin 25.5.0, arm64)
• Providers reproduced:
  • freemodel / gpt-5.4 (default in ~/.cora/auth.toml, base https://api.freemodel.dev/v1)
  • zai / glm-5.1 (base https://api.z.ai/api/coding/paas/v4)
• No .cora.yaml in the project (default config).
• Formats tried: compact, json — both fail.

Workaround

Scanning a small directory (≤ ~15 files, single batch) succeeds with the same provider/model and produces correct findings. Reproduced reliably:

# Works (single batch, ~3 files)
mkdir -p /tmp/cora-test && cp apps/api/src/*.rs /tmp/cora-test/ && cp apps/api/src/models/*.rs /tmp/cora-test/
cora scan --format compact --provider zai --model glm-5.1 --path /tmp/cora-test
# → Scanned 3 files — Found 4 issues. ✅

# Works (11 files, 1 batch)
cp apps/api/src/routes/*.rs /tmp/cora-test/
cora scan --format compact --provider zai --model glm-5.1 --path /tmp/cora-test
# → 11 matches in 6F ✅

# Fails (17 files, 2 batches)
cora scan --format compact --provider zai --model glm-5.1 --path <dir with 17 .ts/.svelte files>
# Reviewing (batch 1/2)… Reviewing (batch 2/2)… Scanned 17 files — Found 11 issues. ✅ (this one passed)

# Fails (98 files, 13 batches) — the full-project case
cora scan --format compact
# → Error: failed to parse LLM JSON response: expected value at line 1 column 1 ❌

The failure threshold seems tied to batch size / total payload, not to a specific file — the same files that scan cleanly in small groups abort the run when included in a 98-file scan.

Likely cause

When a batch is large, the model returns a response that isn't valid JSON (truncated, wrapped in prose, or an error body from the provider). cora then hard-fails the whole scan instead of:

1. Logging the raw offending response (only shown with --verbose, and even then minimal), and
2. Skipping / retrying the batch with a smaller chunk, and
3. Emitting partial results for the batches that succeeded.

Suggestions

• Capture and surface the raw LLM response (or its first N bytes) when JSON parsing fails, so users can tell whether it's a truncation, a rate-limit page, or a provider error.
• Make per-batch parse failures non-fatal: skip the batch, log a warning with the file list, continue.
• Optionally auto-retry a failed batch with a smaller chunk size.
• Consider a --max-batch-size / --batch-files N flag so users can work around provider token limits.

Happy to test a fix or provide more trace if useful. Great tool — the small-directory scans are excellent.