Skip to content

repair-proof contract between neighbor_sync and audit/pruning is implicit and unenforced #1

Description

@RexBytes

Summary

The replication repair-proof gate (anti-outsourcing: a node must prove it
sent replica-repair hints before it may audit / prune-gate its peers) is
implemented as a producer → shared-state → consumer pipeline whose
correctness depends on invariants the type system and call graph do not
enforce. Producer and consumers never reference each other; they communicate
only through the RepairProofs struct, so any semantic drift compiles cleanly
and silently weakens or disables a security mechanism.

Surfaced as hidden coupling (files that co-change in git but share no
structural edge) and confirmed by reading the code.

The pipeline

neighbor_sync.rs (PRODUCER)         types.rs (RepairProofs)        audit.rs + pruning.rs (CONSUMERS)
build_replica_hints_for_peer   →    record_replica_hint_sent(  →   has_mature_replica_hint(
  SentReplicaHint{ key,               peer, key,                      challenged_peer, key,
    close_peers }  ← snapshot         current_close_peers,            close_peers,   ← recomputed independently
                                      hinted_at_epoch }               current_sync_epoch, now )
       │ NeighborSyncOutcome                 ▲                        gate: proof.hinted_at_epoch < current_epoch
       └ mod.rs::record_sent_replica_hints ──┘                              + close-group match + time maturity
         (hinted_at_epoch = sync_cycle_epoch)
  • Producer: neighbor_sync.rs:145 builds SentReplicaHint { key, close_peers };
    recorded via mod.rs::record_sent_replica_hints
    RepairProofs::record_replica_hint_sent with hinted_at_epoch = *sync_cycle_epoch.
  • Two consumers, each independently recomputing the close group:
    • audit.rs:194mature_audit_keys_for_peerhas_mature_replica_hint (audit.rs:423)
    • pruning.rs:490has_mature_replica_hint

What's wrong (three unenforced invariants)

  1. The close-group snapshot must be computed identically by the producer and
    both consumers.
    The producer snapshots a "self-inclusive close group" into
    SentReplicaHint.close_peers; each consumer recomputes close_peers from its
    own lookup. If the computations diverge (different K, distance metric,
    routing-table vs DHT source, self-inclusion), proofs silently fail to match
    and audits / prune-gates silently never mature. There are three
    independent close-group computations (1 producer + 2 consumers) that must
    agree by developer discipline.

  2. The epoch maturity convention must agree. Producer stamps
    hinted_at_epoch; consumers require proof.hinted_at_epoch < current_epoch
    (types.rs:412), i.e. "≥ one full sync cycle old," carried only by a raw
    u64. Drift toward lenient → hints mature too early → peers audited /
    struck before they could repair (honest peers penalized). Drift toward
    strict → hints never mature → outsourcing / freeloading undetected.

  3. Self-inclusion is a convention, not a type. Both
    SentReplicaHint.close_peers and RepairProofEntry.close_peers are
    documented "self-inclusive"; the membership gates (types.rs:369,
    audit.rs:195) are off-by-one-member if any side computes it self-exclusive.

The compiler and call graph miss all three because every field is a
structurally valid HashSet / HashMap / u64 on every end.

Evidence this is real, not theoretical

Three separate changes have edited the two ends of this same contract in
lockstep — the classic "co-change with no code edge" signal:

  • gate audits by mature repair proofs (the feature itself)
  • batching neighbor-sync hint construction (producer side)
  • "tolerate one lagging peer in the prune proof gate" (consumer side)

What's worth implementing (ranked)

A. One shared close-group function (highest value, low risk). Extract a
single fn close_group_for(key) -> HashSet<PeerId> (one definition of K +
distance + self-inclusion) and call it from the hint producer and both
consumers (audit, pruning), instead of three independent computations.
Makes invariants #1 and WithAutonomi#3 structural — they can no longer drift.

C. A cross-module round-trip contract test. Isolated unit tests on
RepairProofs exist (audit.rs:1504+, types.rs:967) but none span producer →
proofs → consumer. Add a test that builds a hint against close group G, then
asserts (i) it matures & gates when the consumer still sees G, (ii) it does
not mature after the close group rotates, (iii) it does not mature
before one epoch elapses — for both the audit and pruning consumers.

B. (optional) SyncEpoch newtype wrapping the u64 with a single
is_mature_for_audit(hinted, current) -> bool, so the maturity rule lives in
one place and can't diverge across the producer and two consumers.

D. (floor) Document the rendezvous on RepairProofs: name its producer
(neighbor_sync::build_replica_hints_for_peer) and consumers
(audit / pruning::has_mature_replica_hint) and the three invariants.

Recommendation: A + C. A removes two of the three drift risks at the source
(now across three call sites, not two); C pins the epoch / maturity behaviour A
doesn't cover and gives a regression guard.

Severity

Hardening, not a known live bug — the current computations do appear to agree.
But the contract gates a security mechanism, breaks silently in either
direction, and now has two independent consumers, so making it structural is
worth more than a typical refactor.

Provenance

Surfaced via stitchgraph hidden-coupling analysis (git co-change, no structural
edge) and confirmed by reading neighbor_sync.rs / types.rs / audit.rs /
pruning.rs / mod.rs. Captured in-repo at docs/findings/repair-proof-coupling.md
(branch claude/loving-cori-l6rv6j).

The feature is intended design upstream (WithAutonomi/ant-node): it landed in
PR WithAutonomi#106 ("gate audits by mature repair proofs"), with later re-tunings in PR
WithAutonomi#150 (producer / neighbor-sync) and PR WithAutonomi#134 (prune-proof gate). None of those
introduced a shared close-group computation, an epoch newtype, or a
producer→consumer round-trip test.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions