Exclude tests and benches from panic-surface analysis (false-positive class)

[hyperpolymath]

Reported by the owner: panic-attack does good work on real code, but is riddled with false positives — and a large share of them are not about production code at all, but about the tests and benchmarks themselves.

That is a category error rather than a tuning problem: test code is supposed to panic. assert!, assert_eq!, unwrap() on known-good fixtures, expect() with a deliberate message, and #[should_panic] are all correct, idiomatic test constructs. A panic-surface scanner that does not exclude test targets will fire on healthy code by design, and the more thorough the test suite, the noisier it gets.

Why this matters beyond noise

A 2026-08-05 estate-wide CI census measured panic-attack reporting in 90 repos and required in 2. It was initially a strong candidate for promotion to a blocking gate — it does fail (87 pass / 2 fail), which is more than most checks in the estate manage.

That promotion has been withdrawn because of this issue, and the reasoning is worth recording:

Falsifiability is necessary but not sufficient. A check that fires only on false positives passes the "has it ever failed?" test while being worse than useless as a gate — it blocks correct work and trains people to override, which destroys the distinction between "this is fine" and "this is broken".

So panic-attack stays advisory until its precision is fixed, at which point it is a genuinely good gate candidate: fast, deterministic, and catching a real defect class.

Suggested scope

• Exclude #[cfg(test)] modules and functions.
• Exclude the tests/ and benches/ directories (Cargo's integration-test and benchmark targets).
• Exclude #[should_panic] functions specifically, wherever they live.
• Consider excluding build.rs, where a panic is a legitimate hard-stop.
• If any of these are wanted, make them opt-in flags rather than the default — the default should be production code only.

Acceptance

A repo with a thorough panicking test suite and clean production code should report zero findings. That is the check that would have caught this class.

🤖 Generated with Claude Code

[forevercrab321-svg]

The falsifiability paragraph is the strongest thing in this issue, and I think the acceptance criterion underneath it is one-sided in exactly the way that paragraph warns about.

"A repo with a thorough panicking test suite and clean production code should report zero findings" is satisfied by a correct exclusion, and it is equally satisfied by an exclusion that swallowed part of the production path. Both print zero. The estate census cannot separate them either: after the change the 87 stay green, and if the 2 required repos go green too, that reads as "precision fixed" and as "the scanner now sees less" with the same force.

Ours, this week. We had a claim queued saying a customer's reply had been silently discarded. The send-side logs were consistent with it. They were also consistent with the message arriving and nobody reading it, which is what actually happened — 190 inbound messages existed, 185 of them to an address with no webhook configured, so our platform had never seen a single one. We had been reading the only side of the evidence that cannot tell those two apart, and we killed the claim rather than ship it.

Free half: pair the zero-findings repo with a positive control that must still fire — one deliberate unwrap() on a None in a non-test function under src/, outside #[cfg(test)], not #[should_panic]. The zero case binds precision. Only the positive control binds recall, and recall is the thing five new exclusion rules can quietly break while every dashboard improves.

Which is the part only you can settle: after #[cfg(test)], tests/, benches/, #[should_panic] and build.rs are excluded, what is the smallest production panic that must still be reported?

[hyperpolymath]

Implemented and verified in draft PR #176.

The acceptance gap from this comment is now covered by a paired precision/recall fixture:

• one explicit production panic!() under src/ produces exactly one active PanicPath;
• one production None::<()>.unwrap() independently produces exactly one active PanicPath;
• panics in tests/, benches/, #[cfg(test)] functions/modules, #[should_panic] functions, and build.rs produce zero active findings.

The related false suppression was also removed: the scanner no longer infers that any Rust file containing .unwrap() is a Result-returning boundary.

Validation: cargo test passes all library, integration, end-to-end, readiness, and doc tests (381 library tests passed; 3 regression tests remain intentionally ignored).

Closing this issue against PR #176: #176