fix(llm): isolate Stage 2 batch failures and keep unanalysed findings

[nyxst4ck]

Summary

Fixes the two coupled Stage 2 resilience bugs, which the reporter noted should be considered together because fixing the abort without fixing the drop would make the scanner less safe:

• Stage 2: one failed batch aborts the entire meta-analyzer pass and silently falls back to static #9 — one failed batch aborts the entire meta-analyzer pass and silently falls back to static
• Stage 2: apply_filter silently drops findings for files the LLM never analyzed (false negatives) #11 — apply_filter silently drops findings for files the LLM never analyzed (false negatives)

Changes

llm_analyzer_base.py — per-batch failure isolation. arun_batches now gathers with return_exceptions=True: a transient error (timeout, 429, oversized-chunk 400) is logged with the batch label and costs only its own batch instead of cancelling the whole fan-out. ValueError and NotImplementedError still propagate — they signal misconfiguration (missing API key, wrong response_schema), which skipping a batch cannot fix. This also benefits the two discovery analyzers (semantic_developer_intent, semantic_quality_policy), which previously lost all discovered findings when any single batch failed.

meta_analyzer.py — no verdict means conservative keep, not delete. With partial results possible, the keep/drop loop must distinguish "the LLM rejected this" from "the LLM never saw this". The node now partitions findings by whether a returned batch actually carried them (each Batch already tracks its findings): analysed findings go through the normal confirm-or-drop filter; unanalysed ones are kept via the existing _fallback_filtered path; a WARNING logs how many findings were kept unfiltered so the gap is visible rather than silent. When all batches fail, behaviour matches today's net result (everything kept via fallback).

Net effect, as suggested in #11: an infra failure can only ever cost enrichment on a file, never the finding itself.

Tests

• arun_batches: a failing batch doesn't abort the others; all-failed returns empty; ValueError still propagates (5 new tests in test_llm_analyzer_base.py).
• New tests/nodes/test_meta_analyzer.py: confirmed finding kept enriched + rejected finding dropped + unseen finding kept when one batch fails; all-batches-failed keeps everything; strict confirm-or-drop preserved when nothing fails.
• The two node-level tests are red on main and green with this fix.

pytest tests/ --ignore=tests/integration   # 598 passed; failures identical to main (pre-existing, unrelated)
ruff check / ruff format                   # clean on touched files

All commits are signed off per the DCO.

[rng1995]

Solid, well-reasoned fix. I traced both halves against the current code and the keep/drop semantics hold up, including the conservative (fail-closed) direction that matters for a security scanner.

Correctness — verified

• get_batches(files_with_findings, file_cache, findings) builds batches from the same Finding instances (it appends f directly / passes them through findings_in_range), and arun_batches returns those same Batch objects. So the id(f) partition in meta_analyzer is sound: a finding is "analysed" only if a batch that actually returned carried it.
• When no batch fails (len(batch_results) == len(batches)), the code reduces to apply_filter(findings, batch_results) — byte-for-byte the previous behavior, so there's no regression to the strict confirm-or-drop path. The added node test (test_no_failures_keeps_strict_confirm_or_drop) pins this.
• analysed and unanalysed are a disjoint partition of findings, so combining apply_filter(analysed, …) with _fallback_filtered(unanalysed) can't double-count or drop.
• apply_filter is pure confirm-or-drop keyed by (file, rule_id, start/end_line) and _fallback_filtered is keep-all, so routing un-analysed findings through the fallback genuinely prevents an infra failure from turning into a false negative. The all-batches-failed case collapses to today's "keep everything via fallback", as claimed.
• Re-raising ValueError from arun_batches is consistent with the caller's existing except ValueError: raise, so misconfiguration still surfaces instead of masquerading as a clean scan.

Tests
Good coverage of exactly the risky logic: per-batch isolation, all-failed → empty, ValueError propagation, and the three node-level keep/drop scenarios (confirmed kept + enriched, rejected dropped, unseen kept, all-failed keep-all, strict path preserved). Appreciate that the node tests are red on main / green with the fix.

Minor / optional (non-blocking)

• In arun_batches, isinstance(result, BaseException) + continue will also swallow asyncio.CancelledError (it derives from BaseException, and return_exceptions=True surfaces child cancellations as results). Consider narrowing the swallow to Exception, or explicitly re-raising asyncio.CancelledError, so cooperative cancellation isn't masked. Low severity since outer-task cancellation still propagates from the gather await itself.
• The id()-based partition is correct today but silently depends on get_batches not copying findings. A one-line comment noting that invariant (or matching on a stable finding key) would harden it against a future refactor. Worth noting the failure mode is itself fail-safe — if instances ever diverged, findings would be treated as unanalysed and kept, never dropped.

These are optional; the change is correct, conservative in the right direction, and well-tested. Approving.

[rng1995]

@nyxst4ck - Please resolve the conflicts, minor issues and merge the PR.

[nyxst4ck]

Resolved the merge conflict by merging current upstream/main into this branch, and addressed the two minor review notes while I was there:

• �syncio.CancelledError now propagates instead of being treated as a transient batch failure.
• Added a short comment documenting the Finding object identity invariant used for partial-batch partitioning.
• Kept both the upstream �pply_filter regression tests and this PR's partial-batch fallback tests in ests/nodes/test_meta_analyzer.py.

Validation:

• uv run --extra dev pytest tests/nodes/test_llm_analyzer_base.py tests/nodes/test_meta_analyzer.py (108 passed)