Skip to content

Reviewers manufacture findings to avoid reporting none #833

Description

@williamthorsen

Reviewers manufacture findings to avoid reporting none

Problem

Code reviewers treat a no-findings review as a failure state: they pad clean changes with low-value findings, then dock the score to match. This is a disposition, not a knowledge gap — challenged, reviewers drop the padding immediately and without defending it, which means they recognized it as fluff when they wrote it. Absent a challenge, they emit by default.

The behavior recurs after two prior mitigations (#679's authored-choice gate, #804's actionability gate). The codeassembly KB corroborates it across 17+ notes spanning 2026-06-09 to 2026-06-20, including instances two days after #804 merged and one the same day this ticket was filed.

Context

Both prior PRs added per-finding filters — constraints on findings the reviewer has already decided to write. Neither addresses the premise that an empty review is a failure, which is the actual driver, so the behavior persists (e.g. the #816 review: six "None." buckets, a self-disqualifying S1, docked to 9/10, with the actionability gate already in force).

The "Recommendations/Suggestions don't count against score" carve-out compounded the problem: it gave low-value findings a consequence-free tier to live in, while reviewers docked the score anyway.

The only thing that reliably stops the padding is the user challenging the reviewer until it drops the fluff — in effect, asserting with authority that zero findings is acceptable. So the lever is to reproduce that assertion and the act of self-challenge, not to add another constraint-rule.

The orchestrated reviewer subagents already omit empty finding blocks structurally and are not implicated — they are the model, not a target. And because the artifacts repo holds the post-cleanup state, the KB, not the saved reviews, is the instrument for measuring recurrence.

This ticket adds guidance about not padding reviews and must not pad the guidance itself: guardrails accrete and bury the message they protect, so consolidate concisely rather than stack beside the existing rules.

Proposed solution

Act on the interactive review path only — review-branch, with the canonical statement in the artifact-conventions finding-scheme SSOT. Three reinforcing parts:

  1. Frame the default. State prominently, before findings are drafted, that a no-findings review is a complete, valid, mergeable result — the reviewer's expected default, not a lazy or unfinished one.
  2. Reproduce the challenge. A mandatory pre-save pass: re-read each finding and delete every one the reviewer would not defend to a skeptical author, explicitly permitted to terminate at zero.
  3. Make the score an honest quality signal. A no-findings review is 10/10; every authored finding that survives the self-challenge lowers it by severity (F heaviest to S lightest), with no score-neutral tier. This removes the "Recommendations/Suggestions don't count against score" carve-out, which gave padding a consequence-free home.

The frame grants permission to land on zero, the self-challenge is the action that gets there, and honest scoring removes both the incentive to invent a finding and the hiding place for one. They reinforce rather than restate — which is what the per-finding filters failed to do.

Acceptance criteria

Must have

  • review-branch states, prominently and before findings are drafted, that a no-findings review is a complete and valid result and the reviewer's default outcome.
  • review-branch includes a mandatory pre-save pass that drops any finding the reviewer would not defend to a skeptical author, explicitly allowed to end at zero findings.
  • Scoring signals quality honestly: a no-findings review is 10/10, and every authored finding that survives the self-challenge (F/W/T/R/S) lowers the score by its severity — no score-neutral tier. Legacy and pre-existing observations never affect the score.
  • New guidance is concise and consolidates with the existing finding-discipline rules: the zero-findings frame has one canonical form in the finding-scheme SSOT, cross-surface duplication is minimized, and the existing actionability-gate and scoring rules stay legible rather than buried.
  • Changes are confined to the interactive review path (review-branch and the artifact-conventions finding-scheme SSOT); the reviewer subagents are out of scope.

Should have

  • review-branch's output template includes a worked example of a clean, no-findings review scoring 10/10, so the template models a clean outcome and not only a populated finding.

Metadata

Metadata

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions