fix(payment): price quotes from current_chunks() to match freshness gate#122
Merged
Merged
Conversation
The storage-delta freshness gate added in #120 / refined in 33db86a rejected essentially every payment on an actively-replicating network with: payment error: Quote from peer ... stale by 29 records (quoted 8 vs current 37, tolerance 5) Root cause: pricing and the freshness gate read two different record counts. - Pricing (`QuoteGenerator::create_quote`) used the in-memory `QuotingMetricsTracker`, whose `record_store()` is incremented ONLY on the client-paid PUT path (`storage/handler.rs`). Replication stores (`replication/mod.rs` fresh fan-out + repair fetch) write to LMDB without touching it. - The freshness gate (`validate_quote_freshness`) reads `LmdbStorage::current_chunks()` — the authoritative total, which DOES include replicated and repaired records. So `current` counted all records while `quoted` (price-derived) counted only direct paid PUTs. On any network with replication, current >> quoted, the delta blew past `tolerance = max(5, 5% of quoted)`, and every payment was rejected. 33db86a made the verifier's read authoritative but left pricing on the side counter, which is what introduced the divergence. STG-01 (~907 nodes, 30% NAT, 10 concurrent uploaders) hit ~100% upload failure on the first cycle; smaller runs only passed because per-node record counts stayed under the 5-record floor. Fix: price quotes from the same authoritative source the gate reads. `QuoteGenerator` gains an attached `Arc<LmdbStorage>` (mirroring `PaymentVerifier`) and prices from `current_chunks()`, falling back to the in-memory counter only when no store is attached (unit tests / misconfigured startup). `AntProtocol::new` attaches the store to the generator right beside the verifier, so the invariant holds for every construction path. `calculate_price` and `derive_records_stored_from_price` round-trip exactly, so feeding larger current_chunks() counts through pricing is lossless and the verifier recovers the exact count — leaving the freshness delta at ~genuine in-flight growth. current_chunks() is also a more accurate measure of node fullness for pricing than direct-paid-PUTs-only. Adds a regression test that writes records straight to the store (as replication would, without bumping the side counter) and asserts the quote prices off the store count — the divergence the prior unit tests structurally could not express. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2254961 to
bd86c51
Compare
jacderida
added a commit
to jacderida/ant-node
that referenced
this pull request
Jun 8, 2026
… delta PR WithAutonomi#122 made pricing and the freshness gate read the same record source, but uploads on a fresh, rapidly-filling testnet (STG-01, 0.11.6-rc.2) still failed ~100%. The residual cause is the freshness gate itself. The gate compared the price-derived quoted record count against the node's current count with a fixed `max(5, 5%)` tolerance, symmetrically (abs_diff). Two problems on an actively-replicating network: 1. Tolerance too tight for normal in-flight churn. Between a node quoting and verifying a payment (the client collects 7 quotes/chunk, pays on-chain via Arbitrum, then PUTs — several seconds) the node's record count drifts by a handful of records via replication. Sampled STG-01 rejects had a median delta of 8 — just over the floor of 5. At low/moderate fill the pricing curve is nearly flat, so that drift is a negligible price change, yet it was rejected. 2. Symmetric abs_diff rejected over-payments. ~36% of rejects had current < quoted: the node had FEWER records than when it quoted (prune/churn), so the client paid for a fuller, pricier node — an over-payment — and the node rejected it anyway. Rejecting money a node is owed is nonsensical. Because an upload needs every chunk stored, even a ~40% per-chunk reject rate drives upload success to ~0 (0.6^9 for a 9-chunk file; worse for large files). So per-chunk rejection has to approach zero, not merely shrink. Fix: make the gate price-based and one-directional. Compute the price the node would charge now for its current fullness and reject only if the quote paid less than that by more than QUOTE_PRICE_STALENESS_PCT_TOLERANCE (25%). - Over-payment (paid >= current price) is always accepted (fixes #2). - Comparing prices self-scales with the quadratic curve: benign churn on the flat part of the curve is ignored, while genuine staleness at high fill — where the curve is steep — is still caught (fixes #1). Replaces the two record-delta tolerance constants with a single price percentage. Reworks the freshness tests: a 10-record drift now passes (it was "stale by 10" before), over-payment passes (regression test for #2), and a quote underpricing by >25% is still rejected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On the STG-01 testnet (ant-node 0.11.6-rc.1, ~907 nodes, 30% NAT-simulated, 10 concurrent uploaders) ~100% of uploads failed on the first cycle, every chunk PUT rejected by the storing node:
Root cause
The storage-delta freshness gate (
validate_quote_freshness) compares two different record counts:quoted(price-derived)QuotingMetricsTrackerside countercurrent(freshness gate)LmdbStorage::current_chunks()record_store()(which bumps the side counter) fires only on the client-paid PUT path instorage/handler.rs. The replication store paths (replication/mod.rsfresh fan-out + repair fetch) write to LMDB without touching it. So on any replicating networkcurrent >> quoted, the delta blows pasttolerance = max(5, 5% of quoted), and every payment is rejected.33db86amade the verifier's read authoritative (current_chunks()) but left pricing on the side counter — that is what introduced the divergence. Smaller test runs only passed because per-node record counts stayed under the 5-record absolute floor; the bug was latent, not absent.Fix
Price quotes from the same authoritative source the gate reads.
QuoteGeneratorgains an attachedArc<LmdbStorage>(mirroringPaymentVerifier) and prices fromcurrent_chunks(), falling back to the in-memory counter only when no store is attached (unit tests / misconfigured startup).AntProtocol::newattaches the store to the generator right beside the verifier, so the invariant holds for every construction path.calculate_price↔derive_records_stored_from_priceround-trip exactly (the coefficient divides evenly), so feeding largercurrent_chunks()counts through pricing is lossless and the verifier recovers the exact count — leaving the freshness delta at ~genuine in-flight growth.current_chunks()is also a more accurate measure of node fullness for pricing than direct-paid-PUTs-only.Tests
test_pricing_tracks_attached_storage_not_side_counter: writes records straight to the store (as replication would, without bumping the side counter) and asserts the quote prices off the store count — the divergence the prior unit tests structurally could not express (they drove both sides with a single number viaset_records_stored_for_tests).cargo test -p ant-node --lib payment::→ 133 passed, 0 failed.cargo clippy -p ant-node --testsclean.Follow-up (not in this PR)
The side counter /
record_store()is now fallback-only and effectively dead in production. A later change could drop it entirely, symmetric with what33db86adid on the verifier side. Kept out here to keep the patch minimal and low-risk.🤖 Generated with Claude Code