fix(pruning): make prune candidacy independent of repair-hint history

docs/REPLICATION_DESIGN.md

@@ -105,9 +105,17 @@ Parameter safety constraints (MUST hold):
 17. A `PaidNotify(K)` only whitelists key `K` after receiver-side proof verification succeeds; sender assertions never whitelist by themselves.
 18. Neighbor-sync paid hints are non-authoritative and carry no PoP; receivers MUST only whitelist by paid-list majority verification (`>= ConfirmNeeded(K)`) or close-group replica majority (Section 7.2 rule 4), never by hint claims alone. Paid-hint-only processing MAY enqueue record fetch only after authorization succeeds and `IsResponsible(self, K)` is true; otherwise it updates `PaidForList(self)` only.
 19. Storage-proof audits start only after `BootstrapDrained(self)` becomes true.
-20. Storage-proof audits target only peers derived from closest-peer lookups for sampled local keys, filtered through local authenticated routing state (`LocalRT(self)`), and further filtered to `(peer, key)` pairs for which mature `RepairProof` and `RepairOpportunity` hold; random global peers, never-synced peers, peers without a key-specific repair hint proof, peers whose key proof predates a close-group change, and peers whose proof was cleared by routing-table removal are never audited for that key.
+20. Responsible-chunk storage audits (the general anti-outsourcing audit tick, Section 15) target only peers derived from closest-peer lookups for sampled local keys, filtered through local authenticated routing state (`LocalRT(self)`), and further filtered to `(peer, key)` pairs for which mature `RepairProof` and `RepairOpportunity` hold; random global peers, never-synced peers, peers without a key-specific repair hint proof, peers whose key proof predates a close-group change, and peers whose proof was cleared by routing-table removal are never audited for that key. This repair-proof prerequisite applies to the responsible-chunk audit ONLY: stored-record prune candidacy and prune-confirmation target selection (rule 22) never depend on `RepairProof` or `RepairOpportunity` — prune confirmation concerns the peers currently closest to the KEY, while repair hints are sent to the peers closest to SELF, and nothing guarantees those sets overlap for an out-of-range key.
 21. Verification-request batching is mandatory for unknown-key neighbor-sync verification and preserves per-key quorum semantics: each key receives explicit per-key evidence, and missing/timeout evidence is unresolved per key.
-22. On every `NeighborSyncCycleComplete(self)`, node MUST run a prune pass using current `SelfInclusiveRT(self)`: for stored records where `IsResponsible(self, K)` is false, record `RecordOutOfRangeFirstSeen` if not already set and delete only when `now - RecordOutOfRangeFirstSeen >= PRUNE_HYSTERESIS_DURATION`, every currently-known `CloseGroup(K)` member has a mature `RepairProof(peer, K)` tied to the current close-group snapshot, and every currently-known `CloseGroup(K)` member returns a positive nonce-bound audit proof for the record. If any close-group peer lacks mature repair proof or does not prove storage, pruning is deferred and the retained local record continues participating in normal neighbor-sync repair. Prune-confirmation failures emit trust penalties after fresh responsibility confirmation and key-specific mature repair proof; first-time bootstrap claims use the same one-time `BOOTSTRAP_CLAIM_GRACE_PERIOD` tracking before abuse penalties apply. Clear `RecordOutOfRangeFirstSeen` when back in range. For `PaidForList` entries where `self ∉ PaidCloseGroup(K)`, record `PaidOutOfRangeFirstSeen` if not already set and delete only when `now - PaidOutOfRangeFirstSeen >= PRUNE_HYSTERESIS_DURATION`; clear `PaidOutOfRangeFirstSeen` when back in range. The two timestamps are independent.
+22. On every `NeighborSyncCycleComplete(self)`, node MUST run a prune pass over all stored records and `PaidForList` entries using the current local routing table. For stored records, retention responsibility is evaluated at the storage-retention width `CLOSE_GROUP_SIZE + STORAGE_ADMISSION_MARGIN` (9 at reference parameters):
+    - In range: clear `RecordOutOfRangeFirstSeen(self, K)`.
+    - Out of range: set `RecordOutOfRangeFirstSeen(self, K)` to `now` if not already set — immediately, even when an older retained commitment currently contains `K` (commitment retention vetoes deletion, never the start of hysteresis). The timestamp is process-local and need not survive restarts.
+    - Once continuously out of range for `>= PRUNE_HYSTERESIS_DURATION`, the record is a prune candidate UNCONDITIONALLY: candidacy never depends on repair-hint proofs, bootstrap state, audit budget, or prior neighbor-sync contact.
+    - Candidates are processed subject to operational scheduling: bootstrap state (rule 19) may defer the prune audit, the bounded per-pass challenge budget may defer the prune audit, and a retained answerable commitment vetoes deletion. These are deferrals only — they never remove candidate status or restart hysteresis.
+    - Prune confirmation is a direct current-close-group possession check: challenge the current strict `CloseGroup(K)` (taken directly from the local routing table, never filtered through `RepairProofs` or prior neighbor-sync hints) with the nonce-bound audit proof, and delete the local record only when all but one of the current close group (6 of 7 at reference parameters) return valid positive possession proofs. Missing, timed-out, bootstrapping, malformed, absent, and digest-mismatching responses never count positively. Below the threshold, the record, its timestamp, and its candidacy are retained and retried on later passes, and the retained record continues participating in normal neighbor-sync repair.
+    - Immediately before deletion, revalidate against the current routing table: the key must still be outside the retention width, must not be held by an answerable retained commitment, and the positive reports must still satisfy the CURRENT strict close group (membership churn invalidates stale reports).
+    - Prune-confirmation failures emit trust penalties only after fresh responsibility confirmation; first-time bootstrap claims use the same one-time `BOOTSTRAP_CLAIM_GRACE_PERIOD` tracking before abuse penalties apply.
+    For `PaidForList` entries where `self ∉ PaidCloseGroup(K)`, record `PaidOutOfRangeFirstSeen` if not already set and delete only when `now - PaidOutOfRangeFirstSeen >= PRUNE_HYSTERESIS_DURATION` and three quarters of the current paid close group (15 of 20 at reference parameters) confirm the key in their own paid lists; clear `PaidOutOfRangeFirstSeen` when back in range. The two timestamps are independent.
 23. Peers claiming bootstrap status are skipped for sync and audit without penalty for up to `BOOTSTRAP_CLAIM_GRACE_PERIOD` from first observation. After the grace period, each continued bootstrap claim emits `BootstrapClaimAbuse` evidence to `TrustEngine` (via `report_trust_event` with `ApplicationFailure(weight)`). If a peer stops claiming bootstrap and later claims bootstrap again, the repeated claim emits `BootstrapClaimAbuse` immediately; the grace period is not reset.
 24. Audit trust-penalty signals require responsibility confirmation: on audit failure, challenger MUST perform fresh local RT closest-peer lookups for each challenged key and only penalize the peer for keys where it is confirmed responsible.
 
@@ -361,15 +369,25 @@ This check is evaluated per-key at decision points:
 2. Post-cycle pruning eligibility (prune stored records where node is no longer responsible).
 3. Post-cycle paid-list retention eligibility (drop `PaidForList` entries for keys where node is no longer in `PaidCloseGroup(K)`).
 
+Distinct widths and mechanisms govern a stored record's lifecycle; they must not be conflated (reference profile values in parentheses):
+
+- **Storage admission**: how close self must be to accept a write. The client PUT self-closeness gate accepts up to the K-bucket width (20); neighbor-sync replica-hint admission uses the storage-admission width `CLOSE_GROUP_SIZE + STORAGE_ADMISSION_MARGIN` (9).
+- **Retained storage responsibility**: a stored record is retained while self is within the storage-retention width `CLOSE_GROUP_SIZE + STORAGE_ADMISSION_MARGIN` (9) of the key.
+- **Commitment inclusion**: storage commitments cover keys for which self is in the strict `CLOSE_GROUP_SIZE` (7) closest, so only a subset of physically stored chunks appears in commitments — being uncommitted is normal for retained-but-not-strictly-close records.
+- **Prune candidacy**: a stored record continuously outside the retention width (9) for `PRUNE_HYSTERESIS_DURATION` is a prune candidate, unconditionally.
+- **Prune-audit scheduling**: candidates are audited subject to bootstrap drain and the bounded per-pass challenge budget; a retained answerable commitment vetoes deletion. All of these are deferrals — none removes candidacy or restarts hysteresis.
+- **Prune deletion confirmation**: a direct possession check of the current strict `CloseGroup(K)` (7), requiring all-but-one (6 of 7) valid positive proofs; targets come straight from the local routing table.
+- **General anti-outsourcing storage audits** (Section 15): a separate mechanism that retains its mature repair-proof prerequisite (rule 20). Prune candidacy and prune-confirmation target selection never consult repair proofs.
+
 Post-cycle responsibility pruning (triggered by `NeighborSyncCycleComplete(self)`):
 
-1. For each locally stored key `K`, recompute `IsResponsible(self, K)` using current `SelfInclusiveRT(self)`:
+1. For each locally stored key `K`, recompute retention responsibility (storage-retention width) using current `SelfInclusiveRT(self)`:
    a. If in range: clear `RecordOutOfRangeFirstSeen(self, K)` (set to `None`).
-   b. If out of range: if `RecordOutOfRangeFirstSeen(self, K)` is `None`, set it to `now`. Once the hysteresis duration has elapsed, request a nonce-bound audit proof only from current `CloseGroup(K)` peers for which a mature `RepairProof(peer, K)` exists for the current close-group snapshot, subject to a bounded per-pass prune-confirmation budget. If any current close-group peer lacks mature repair proof or does not prove storage, defer pruning and emit trust penalties only after fresh responsibility confirmation plus key-specific mature repair proof; first-time bootstrap claims are penalized only after the bootstrap grace period, while repeated bootstrap claims are penalized immediately. Delete the local record only when `now - RecordOutOfRangeFirstSeen(self, K) >= PRUNE_HYSTERESIS_DURATION`, every current close-group peer has mature repair proof for `K`, and every current close-group peer proves storage for `K`.
+   b. If out of range: if `RecordOutOfRangeFirstSeen(self, K)` is `None`, set it to `now` — immediately, even if a retained commitment currently contains `K`. Once the hysteresis duration has elapsed, `K` is a prune candidate. Request a nonce-bound audit proof from the current strict `CloseGroup(K)` peers (taken directly from the local routing table, never filtered through repair proofs), subject to a bounded per-pass prune-confirmation budget; bootstrap state may defer the audit and a retained answerable commitment vetoes deletion, without removing candidacy. Delete the local record only when all but one of the current close group (6 of 7) return valid positive possession proofs AND an immediate pre-deletion revalidation confirms `K` is still out of range, not held by an answerable retained commitment, and the positive reports still satisfy the current close group. Anything less retains the record, its timestamp, and its candidacy for later passes. Trust penalties for failed prune proofs are emitted only after fresh responsibility confirmation; first-time bootstrap claims are penalized only after the bootstrap grace period, while repeated bootstrap claims are penalized immediately.
 2. For each key `K` in `PaidForList(self)`, recompute `PaidCloseGroup(K)` membership using current `SelfInclusiveRT(self)`:
    a. If `self ∈ PaidCloseGroup(K)`: clear `PaidOutOfRangeFirstSeen(self, K)` (set to `None`).
-   b. If `self ∉ PaidCloseGroup(K)`: if `PaidOutOfRangeFirstSeen(self, K)` is `None`, set it to `now`. Delete the entry only when `now - PaidOutOfRangeFirstSeen(self, K) >= PRUNE_HYSTERESIS_DURATION`.
-3. Paid-list entry pruning remains local-state-only and does not require remote confirmations.
+   b. If `self ∉ PaidCloseGroup(K)`: if `PaidOutOfRangeFirstSeen(self, K)` is `None`, set it to `now`. Delete the entry only when `now - PaidOutOfRangeFirstSeen(self, K) >= PRUNE_HYSTERESIS_DURATION` and three quarters of the current paid close group (15 of 20) confirm the key in their own paid lists.
+3. Paid-list entry pruning is gated on paid-list confirmations from the current paid close group; it never requires chunk-possession proofs (and chunk pruning never requires paid-list confirmations).
 
 Effect:
 
@@ -612,7 +630,7 @@ Each scenario should assert exact expected outcomes and state transitions.
 35. Neighbor-sync round-robin batch selection with cooldown skip:
 - With more than `NEIGHBOR_SYNC_PEER_COUNT` eligible peers, consecutive rounds scan forward from cursor, skip and remove cooldown peers, and sync the next batch of up to `NEIGHBOR_SYNC_PEER_COUNT` non-cooldown peers. Cycle completes when all snapshot peers have been synced, skipped (cooldown), or removed (unreachable).
 36. Post-cycle responsibility pruning with time-based hysteresis:
-- When a full neighbor-sync round-robin cycle completes, node runs one prune pass using current `SelfInclusiveRT(self)` (`LocalRT(self) ∪ {self}`): stored keys with `IsResponsible(self, K)=false` have `RecordOutOfRangeFirstSeen` recorded (if not already set) but are deleted only when `now - RecordOutOfRangeFirstSeen >= PRUNE_HYSTERESIS_DURATION`, every current close-group peer has mature repair proof for the current close-group snapshot, and every current close-group peer returns a positive nonce-bound audit proof for the key. Missing or failed proofs defer pruning and emit trust penalties after fresh responsibility confirmation; first-time bootstrap claims are penalized only after the bootstrap grace period, while repeated bootstrap claims are penalized immediately. Prune-confirmation work is bounded per pass. Keys that are in range have their `RecordOutOfRangeFirstSeen` cleared. Same hysteresis timing applies independently to `PaidForList` entries where `self ∉ PaidCloseGroup(K)` using `PaidOutOfRangeFirstSeen`, but paid-list pruning does not require remote storage confirmation.
+- When a full neighbor-sync round-robin cycle completes, node runs one prune pass using current `SelfInclusiveRT(self)` (`LocalRT(self) ∪ {self}`): stored keys outside the storage-retention width have `RecordOutOfRangeFirstSeen` recorded immediately (if not already set, and even while a retained commitment still contains the key). A key continuously out of range for `>= PRUNE_HYSTERESIS_DURATION` is a prune candidate unconditionally; it is deleted only when all but one of the current strict close group (6 of 7 at reference parameters) return valid positive nonce-bound possession proofs, with targets taken directly from the local routing table (never filtered through repair proofs). Missing or failed proofs defer deletion — preserving the candidate, its timestamp, and its retry on later passes — and emit trust penalties only after fresh responsibility confirmation; first-time bootstrap claims are penalized only after the bootstrap grace period, while repeated bootstrap claims are penalized immediately. Prune-confirmation work is bounded per pass. Keys that are in range have their `RecordOutOfRangeFirstSeen` cleared. Same hysteresis timing applies independently to `PaidForList` entries where `self ∉ PaidCloseGroup(K)` using `PaidOutOfRangeFirstSeen`, gated on paid-list confirmations from the current paid close group rather than storage confirmation.
 37. Non-`LocalRT` inbound sync behavior:
 - If a peer opens sync while not in receiver `LocalRT(self)`, receiver may still send hints to that peer, but receiver drops all inbound replica/paid hints from that peer.
 38. Neighbor-sync priority under peer join:
@@ -640,7 +658,7 @@ Each scenario should assert exact expected outcomes and state transitions.
 49. Bootstrap active claim cleared on normal response, with history retained:
 - Peer `P` previously claimed bootstrapping. `P` later responds normally to a sync or audit request. Node clears the active bootstrap claim but retains `BootstrapClaimFirstSeen(self, P)` history. If `P` later claims bootstrapping again, node emits `BootstrapClaimAbuse` immediately instead of granting a second grace period.
 50. Prune hysteresis prevents premature deletion:
-- Key `K` goes out of range at time `T`. `RecordOutOfRangeFirstSeen(self, K)` is set to `T`. Key is NOT deleted. At `T + 24h` (less than `PRUNE_HYSTERESIS_DURATION`), key is still retained. At `T + 3 days` (`>= PRUNE_HYSTERESIS_DURATION`), key is eligible for deletion on the next prune pass only if all current close-group peers have mature repair proof for the current close-group snapshot and return positive nonce-bound audit proofs.
+- Key `K` goes out of range at time `T`. `RecordOutOfRangeFirstSeen(self, K)` is set to `T`. Key is NOT deleted. At `T + 24h` (less than `PRUNE_HYSTERESIS_DURATION`), key is still retained. At `T + 3 days` (`>= PRUNE_HYSTERESIS_DURATION`), key becomes a prune candidate on the next prune pass and is deleted only when all but one of the current strict close group (6 of 7 at reference parameters) return valid positive nonce-bound possession proofs.
 51. Prune hysteresis timestamp reset on partition heal:
 - Key `K` goes out of range at time `T`. `RecordOutOfRangeFirstSeen(self, K)` is set to `T`. At `T + 4h`, partition heals, peers return, `K` is back in range. `RecordOutOfRangeFirstSeen` is cleared. Key is retained. If `K` later goes out of range again, the clock restarts from zero.
 52. Prune hysteresis applies to paid-list entries: