Skip to content

review-bench fixture maintenance: fold baseline-r1 adjudicated unseeded defects into truth/fixtures #36

Description

@SSFSKIM

Problem & intent

Baseline-r1 adjudication surfaced genuine unseeded defects and two imperfect baits in the X1 fixtures (tests/review-bench/). Before the next scored run-id (C2's re-run), fold the adjudicated reality back into the fixtures so recall/FP stay clean: either fix each unseeded defect in base/patch or promote it into truth.json; tighten the two baits that drew defensible flags. Full list: results/2026-07-26-baseline-r1/scores.json genuine_unseeded_excluded + notes.md.

Constraints

  • Do not change any seeded bug or its truth entry (comparability across run-ids).
  • The case3 fixture needs an explicit single-threaded statement (README/base comment) so the recurring TOCTOU finding is unambiguously refutable — both engines flagged it.
  • case5's bounded-retention bait: state the retention bound where a reviewer will see it.

Success criteria

  • A fresh engine run on the updated fixtures produces no finding that is neither truth-matched nor a deliberate bait.
  • scores.json for the next run-id needs no "genuine-unseeded" escape hatch entries for known items.

Open questions

  • none — mechanical fixture maintenance guided by the adjudication record.

Decision log

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpriority:P2issue-tracker board priority

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions