test(audit-gate): assert per-directory scan coverage and re-derive the file-count floors (#4746 Item F1) - #4766
Merged
Conversation
…e file-count floors (#4746 Item F1) `CheckedFilesTest`'s three blanket floors (`CHECKED_FILE_FLOOR`, `LIB_FILE_FLOOR`, `V4_FILE_FLOOR`) were calibrated a few dozen files below the measured count, so every deletion batch of the #4746 safe-to-delete lane had to lower them: 2003/1996 (#4749) -> 1993/1986 (#4751) -> 1984/1977 (#4754). The ratchet described the deletions instead of constraining them, and the file's own standing decision was that the first batch which would actually trip a floor must land F1 (per-directory assertions) rather than recalibrate again. Measured on `f8911169` the slack is 6 further file deletions on two of the three floors, so that batch is the next one. F1-a: pin `LIB_TOP_LEVEL_DIRS` (25 directories, measured) and assert it both against a census of what the scan enumerates and against an independent disk traversal. A filter such as `if "Peierls" not in p.parts` now fails by name at any library size, with no constant to move; a directory the deletion lane empties disappears from both sides at once, so no recalibration follows. F1-b: `LIB_DIR_FLOORS`, coarse per-directory floors for the eight directories at 40+ files plus the loose bucket (Concrete 843 -> 600, AmbientLattice 275 -> 180, ClusterExpansion 205 -> 150, TransferMatrix 109 -> 80, Peierls 105 -> 80, AmbientComplexAnalyticity 76 -> 55, Inequalities 71 -> 50, Conditioning 48 -> 35, loose 39 -> 25). Parity only requires one surviving file per directory; these keep the big ones populated. `assertGreaterEqual`, not the strict comparison the blanket floors use. F1-c: with directory loss now asserted directly, the blanket floors are derived from a budget rather than from today's measurement: 200 further deleted modules below the 2026-07-29 measurement (1964/1957/1994), rounded down to a multiple of ten, giving 1760/1750/1790. Their residual job -- the two ways the exact `git ls-files` comparison can be made vacuous -- does not need a tight floor. F1-d: two mutation tests. `iter_checked_files` filtered on "Peierls" loses the directory from the census while staying above the blanket floor; the same function thinned to three `Conditioning` files keeps parity and the blanket floors green and trips only the per-directory floor. No change to `scripts/audit_gate.py`; V1/V2/V4 read the same 1964/1964/1994 files as before. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2 tasks
phasetr
added a commit
that referenced
this pull request
Jul 28, 2026
…re-derive its declaration floor The canary's declaration-count floor was `assertGreater(count, 1000)` against a measured census of 1005 at main `ddf332d9`, i.e. four further deletions of Unicode-bearing declarations before the CI gate step turns red. #4746 batch 7 deletes seven of them (measured 998 on the batch-7 branch), so the floor is one batch away from forcing the same recalibration `test_audit_gate.py` needed in PR #4766 (#4746 Item F1). The count was never the Unicode guard. The guard is the universal in `run_canary`: every Lambda/beta/sigma-bearing declaration must find itself in its own defining file, which holds at any population size. The count's only residual job is anti-vacuity. So this commit moves the power to assertions that need no calibration and re-derives the number once, from purpose: - a fixture tree (through the scanner's own `build_tree`) holding exactly one declaration per canary character. Per-character coverage on the real tree rests on three incidental declarations -- beta on exactly one, `Inequalities/HighTemp/SimonLiebInfinite.lean:20`, sigma on two in `TransferMatrix/MarkedTraceClosedWalk.lean` -- so the claim "the scanner handles beta" was guaranteed by an accident of naming. It is now a fixture. - two mutations, the direction nothing tested before: no test proved `run_canary` can fail at all. A cleaned text with the Greek letter removed (the signature of the tokenizer defect that produced three bad deletion sweeps) must raise "cannot find itself", and a tree with no Unicode declarations must raise "canary degenerated". - `CANARY_DECL_FLOOR = 500`, `assertGreaterEqual`. Derivation, measured on `ddf332d9`: census 1005 (Lambda 1002, beta 1, sigma 2); 457 of those live inside the #4746 Item A path allow-list and 2 more inside `AmbientLattice/SpecialCases` (#4563). No batch is authorised to delete anything else, so the campaign cannot push the census below 1005 - 457 - 2 = 546 even if every allow-listed module were deleted, and a cascade can only promote modules already inside the allow-list. The measured remaining lane (102 fully-dead modules on `ddf332d9`) carries 79 canary-bearing declarations, so the realistic floor is 926. Expected recalibrations for the rest of the campaign: zero. Test-only. `scripts/dead_candidate_scan.py` is unchanged, so no scan verdict, report or exit code moves. 111 tests pass via both entry points, including the CI gate step `python3 scripts/dead_candidate_scan.py --self-test`. Design: `.self-local/reports/design-canary-floor-f1-2.md`. Part of #4746 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
phasetr
added a commit
that referenced
this pull request
Jul 28, 2026
…re-derive its declaration floor (#4746 Item F1-2) (#4769) * chore: open branch for #4746 Item F1-2 (canary declaration floor) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * test(dead-candidate-scan): pair the Unicode canary with fixtures and re-derive its declaration floor The canary's declaration-count floor was `assertGreater(count, 1000)` against a measured census of 1005 at main `ddf332d9`, i.e. four further deletions of Unicode-bearing declarations before the CI gate step turns red. #4746 batch 7 deletes seven of them (measured 998 on the batch-7 branch), so the floor is one batch away from forcing the same recalibration `test_audit_gate.py` needed in PR #4766 (#4746 Item F1). The count was never the Unicode guard. The guard is the universal in `run_canary`: every Lambda/beta/sigma-bearing declaration must find itself in its own defining file, which holds at any population size. The count's only residual job is anti-vacuity. So this commit moves the power to assertions that need no calibration and re-derives the number once, from purpose: - a fixture tree (through the scanner's own `build_tree`) holding exactly one declaration per canary character. Per-character coverage on the real tree rests on three incidental declarations -- beta on exactly one, `Inequalities/HighTemp/SimonLiebInfinite.lean:20`, sigma on two in `TransferMatrix/MarkedTraceClosedWalk.lean` -- so the claim "the scanner handles beta" was guaranteed by an accident of naming. It is now a fixture. - two mutations, the direction nothing tested before: no test proved `run_canary` can fail at all. A cleaned text with the Greek letter removed (the signature of the tokenizer defect that produced three bad deletion sweeps) must raise "cannot find itself", and a tree with no Unicode declarations must raise "canary degenerated". - `CANARY_DECL_FLOOR = 500`, `assertGreaterEqual`. Derivation, measured on `ddf332d9`: census 1005 (Lambda 1002, beta 1, sigma 2); 457 of those live inside the #4746 Item A path allow-list and 2 more inside `AmbientLattice/SpecialCases` (#4563). No batch is authorised to delete anything else, so the campaign cannot push the census below 1005 - 457 - 2 = 546 even if every allow-listed module were deleted, and a cascade can only promote modules already inside the allow-list. The measured remaining lane (102 fully-dead modules on `ddf332d9`) carries 79 canary-bearing declarations, so the realistic floor is 926. Expected recalibrations for the rest of the campaign: zero. Test-only. `scripts/dead_candidate_scan.py` is unchanged, so no scan verdict, report or exit code moves. 111 tests pass via both entry points, including the CI gate step `python3 scripts/dead_candidate_scan.py --self-test`. Design: `.self-local/reports/design-canary-floor-f1-2.md`. Part of #4746 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * test(dead-candidate-scan): pin CANARY_CHARS so the per-character fixture is not self-fulfilling Raised by the codex cross-check of the previous commit: the fixture derives both its declaration names and its expected `per_char` mapping from `dcs.CANARY_CHARS`, so it could not detect a shrink of that constant. Dropping `β` from it -- the cheapest way to silence the degeneracy abort once the library's single `β`-bearing declaration is renamed -- would have left every canary assertion green while the character stopped being checked at all. That is the same "derive the calibration from the quantity it calibrates" anti-pattern the suite already refuses elsewhere. Also records, in the floor's comment block, that 1005 - 457 - 2 = 546 is an *authorisation* bound on the #4746 deletion campaign and not an invariant this file can enforce: a deletion outside the path allow-list is caught by review, not by the number. 112 tests pass via both entry points. Part of #4746 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #4746 (Item F1). Test-only:
scripts/test_audit_gate.py. No change toscripts/audit_gate.py, no Lean rebuild.Why now
CheckedFilesTest's blanket floors were calibrated a few dozen files below themeasured count, so every batch of the safe-to-delete lane had to lower them:
2003 / 1996 (#4749) -> 1993 / 1986 (#4751) -> 1984 / 1977 (#4754). That is the
ratchet describing the deletions instead of constraining them, and the standing
decision recorded in the file said the first batch that would actually trip a
floor must land F1 -- per-directory assertions -- rather than recalibrate again.
Re-measured independently on
f8911169(this PR's base) withpython3 -c "import audit_gate as ag; print(len(ag.iter_checked_files()), len(ag.iter_lib_files()), len(ag.iter_v4_files()[0]))":assertGreater)iter_checked_files()iter_lib_files()iter_v4_files()The next ten-module batch trips two of the three floors, so this is that batch's
prerequisite.
What changed
F1-a — directory parity (no calibration).
LIB_TOP_LEVEL_DIRSpins the 25top-level
IsingModel/subdirectories, asserted twice: against a census of whatiter_checked_files()enumerates, and against an independent disk traversal(
LIB_DIR.iterdir()+rglob("*.lean")). The two are deliberately not the sameassertion -- deriving the expected set from the scan would be the
CAPSTONE_MIN_COUNT == len(read_capstones())anti-pattern the suite alreadyrefuses. Measured: 25 directories from the scan, 25 from disk, identical sets.
A filter such as
if "Peierls" not in p.partsnow fails by name at anylibrary size; when the lane empties a directory, scan and disk shrink
together, so no recalibration follows. Only a whole-top-level-directory deletion
(last one: #4703,
IsingModel/LatticeSystemBridge) costs an edit, and that editis the point.
F1-b — coarse per-directory floors. Parity only requires one surviving file
per directory, so
LIB_DIR_FLOORSadds a count floor for the eight directoriesat 40+ files plus the loose bucket, at roughly 60-70 % of the measurement:
ConcretePeierlsAmbientLatticeAmbientComplexAnalyticityClusterExpansionInequalitiesTransferMatrixConditioning<top level>(loose)assertGreaterEqual, unlike the strict blanket floors. Smaller directories getno count floor: parity is the whole guard for them and a floor of "3" would be
noise.
F1-c — one-time re-derivation of the blanket floors. With directory loss now
asserted directly, the floors come from a budget instead of from the current
measurement: 200 further deleted modules (about 100 for the derived remainder of
the Item A lane, plus the excluded #4563
SpecialCasescluster, plus room forcascade growth), rounded down to a multiple of ten.
CHECKED_FILE_FLOORLIB_FILE_FLOORV4_FILE_FLOORThe floors are kept rather than deleted because they are the only assertions
that survive the two ways the exact
git ls-filescomparison can be madevacuous: narrowing
EXTRA_CHECKED_ROOTSand the suite's ownROOTSin onecommit, and a pathspec that matches nothing (
set() - scannedis empty, so thesubset assertion passes trivially). Neither needs a tight floor. A new test
pins the floors' residual power as arithmetic: losing
Concretewholesale(843 of 1964 files) must still take the count under
CHECKED_FILE_FLOOR.F1-d — mutation pairing. Two new mutations in
MutationTest, per thisfile's rule that every check is tested in the fixture and the mutation
direction, plus a
LibraryCensusTestpinning the census helper on hand-builtpaths (nested files, loose files, paths outside the library, missing bucket).
Verification
All commands run at
d1f9ca7f.python3 scripts/test_audit_gate.py: 179 tests OK (skipped=1), 18.3 s-- 166 before, 18.6 s, so +13 tests at unchanged runtime.
python3 scripts/audit_gate.py --self-test: OK (same 179).python3 scripts/audit_gate.py --full: PASS -- V1 1964, V2 1964,V3 13 capstones (axiom union
{propext, Classical.choice, Quot.sound}),V4 1994. Run against the identical Lean tree of the base commit in the warm
build tree, since V3 needs oleans; the change is Python-test-only and V1/V2/V4
were additionally re-run on this branch's own tree with the same counts.
rg -n "[Japanese ranges]" scripts/test_audit_gate.py: no match (V4 is greenon the new file as part of the run above).
Injected-mutation check (that the new assertions really go red). Each
weakening was applied to
scripts/audit_gate.pyor to the pinned literal, thenew tests were run, and the file was restored (
git diffclean afterwards):iter_checked_filesfiltered on"TransferMatrix"test_the_scan_reaches_every_top_level_library_directory("Items in the second set but not the first: 'TransferMatrix'") andtest_every_large_directory_keeps_its_coarse_floor("TransferMatrix is down to 0 scanned files (floor 80)"); disk-parity test stays green, as it mustConditioningthinned to 3 files (45 removed)test_every_large_directory_keeps_its_coarse_flooramong the count assertions; directory parity and both blanket floors stay green -- exactly the gap F1-b exists to close"Peierls"removed fromLIB_TOP_LEVEL_DIRStest_the_floor_table_names_only_buckets_that_existLIB_TOP_LEVEL_DIRSGhostDirectoryThe two committed mutation tests are pinned by
load_mutated, which raises ifits target text stops matching, so they cannot go vacuous.
Effect on the lane
Headroom after this PR: blanket 204 / 207 / 204 file deletions,
Concrete243,and no count floor at all on the small directories (parity only). The remaining
Item A lane is about 100 modules, so no further recalibration is expected;
the next constant edit would be a deliberate top-level-directory deletion or a
shrink beyond the 200-module budget, which is a governance event by
construction.
Not included: the
.self-local/issues/4746.mdmirror entry for Item F1, whichis
dev-pr-clerk's file and is being edited concurrently on another branch.🤖 Generated with Claude Code