Skip to content

fix(payment): price quotes from current_chunks() to match freshness gate#122

Merged
jacderida merged 1 commit into
rc-2026.5.5from
fix/quote-pricing-current-chunks
May 30, 2026
Merged

fix(payment): price quotes from current_chunks() to match freshness gate#122
jacderida merged 1 commit into
rc-2026.5.5from
fix/quote-pricing-current-chunks

Conversation

@jacderida

Copy link
Copy Markdown
Collaborator

Problem

On the STG-01 testnet (ant-node 0.11.6-rc.1, ~907 nodes, 30% NAT-simulated, 10 concurrent uploaders) ~100% of uploads failed on the first cycle, every chunk PUT rejected by the storing node:

payment error: Quote from peer ... stale by 29 records (quoted 8 vs current 37, tolerance 5)
→ Error: Upload failed: 0/N stored

Root cause

The storage-delta freshness gate (validate_quote_freshness) compares two different record counts:

Source Counts
quoted (price-derived) QuotingMetricsTracker side counter client-paid PUTs only
current (freshness gate) LmdbStorage::current_chunks() all records: client PUTs + replication + repair

record_store() (which bumps the side counter) fires only on the client-paid PUT path in storage/handler.rs. The replication store paths (replication/mod.rs fresh fan-out + repair fetch) write to LMDB without touching it. So on any replicating network current >> quoted, the delta blows past tolerance = max(5, 5% of quoted), and every payment is rejected.

33db86a made the verifier's read authoritative (current_chunks()) but left pricing on the side counter — that is what introduced the divergence. Smaller test runs only passed because per-node record counts stayed under the 5-record absolute floor; the bug was latent, not absent.

Fix

Price quotes from the same authoritative source the gate reads. QuoteGenerator gains an attached Arc<LmdbStorage> (mirroring PaymentVerifier) and prices from current_chunks(), falling back to the in-memory counter only when no store is attached (unit tests / misconfigured startup). AntProtocol::new attaches the store to the generator right beside the verifier, so the invariant holds for every construction path.

  • calculate_pricederive_records_stored_from_price round-trip exactly (the coefficient divides evenly), so feeding larger current_chunks() counts through pricing is lossless and the verifier recovers the exact count — leaving the freshness delta at ~genuine in-flight growth.
  • current_chunks() is also a more accurate measure of node fullness for pricing than direct-paid-PUTs-only.

Tests

  • New regression test test_pricing_tracks_attached_storage_not_side_counter: writes records straight to the store (as replication would, without bumping the side counter) and asserts the quote prices off the store count — the divergence the prior unit tests structurally could not express (they drove both sides with a single number via set_records_stored_for_tests).
  • cargo test -p ant-node --lib payment:: → 133 passed, 0 failed.
  • cargo clippy -p ant-node --tests clean.

Follow-up (not in this PR)

The side counter / record_store() is now fallback-only and effectively dead in production. A later change could drop it entirely, symmetric with what 33db86a did on the verifier side. Kept out here to keep the patch minimal and low-risk.

🤖 Generated with Claude Code

The storage-delta freshness gate added in #120 / refined in 33db86a rejected
essentially every payment on an actively-replicating network with:

  payment error: Quote from peer ... stale by 29 records
  (quoted 8 vs current 37, tolerance 5)

Root cause: pricing and the freshness gate read two different record counts.

- Pricing (`QuoteGenerator::create_quote`) used the in-memory
  `QuotingMetricsTracker`, whose `record_store()` is incremented ONLY on the
  client-paid PUT path (`storage/handler.rs`). Replication stores
  (`replication/mod.rs` fresh fan-out + repair fetch) write to LMDB without
  touching it.
- The freshness gate (`validate_quote_freshness`) reads
  `LmdbStorage::current_chunks()` — the authoritative total, which DOES include
  replicated and repaired records.

So `current` counted all records while `quoted` (price-derived) counted only
direct paid PUTs. On any network with replication, current >> quoted, the delta
blew past `tolerance = max(5, 5% of quoted)`, and every payment was rejected.
33db86a made the verifier's read authoritative but left pricing on the side
counter, which is what introduced the divergence. STG-01 (~907 nodes, 30% NAT,
10 concurrent uploaders) hit ~100% upload failure on the first cycle; smaller
runs only passed because per-node record counts stayed under the 5-record floor.

Fix: price quotes from the same authoritative source the gate reads.
`QuoteGenerator` gains an attached `Arc<LmdbStorage>` (mirroring
`PaymentVerifier`) and prices from `current_chunks()`, falling back to the
in-memory counter only when no store is attached (unit tests / misconfigured
startup). `AntProtocol::new` attaches the store to the generator right beside
the verifier, so the invariant holds for every construction path.

`calculate_price` and `derive_records_stored_from_price` round-trip exactly, so
feeding larger current_chunks() counts through pricing is lossless and the
verifier recovers the exact count — leaving the freshness delta at ~genuine
in-flight growth. current_chunks() is also a more accurate measure of node
fullness for pricing than direct-paid-PUTs-only.

Adds a regression test that writes records straight to the store (as
replication would, without bumping the side counter) and asserts the quote
prices off the store count — the divergence the prior unit tests structurally
could not express.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jacderida jacderida force-pushed the fix/quote-pricing-current-chunks branch from 2254961 to bd86c51 Compare May 30, 2026 15:56
@jacderida jacderida merged commit e89d873 into rc-2026.5.5 May 30, 2026
11 checks passed
@jacderida jacderida deleted the fix/quote-pricing-current-chunks branch May 30, 2026 16:44
jacderida added a commit to jacderida/ant-node that referenced this pull request Jun 8, 2026
… delta

PR WithAutonomi#122 made pricing and the freshness gate read the same record source, but
uploads on a fresh, rapidly-filling testnet (STG-01, 0.11.6-rc.2) still failed
~100%. The residual cause is the freshness gate itself.

The gate compared the price-derived quoted record count against the node's
current count with a fixed `max(5, 5%)` tolerance, symmetrically (abs_diff). Two
problems on an actively-replicating network:

1. Tolerance too tight for normal in-flight churn. Between a node quoting and
   verifying a payment (the client collects 7 quotes/chunk, pays on-chain via
   Arbitrum, then PUTs — several seconds) the node's record count drifts by a
   handful of records via replication. Sampled STG-01 rejects had a median
   delta of 8 — just over the floor of 5. At low/moderate fill the pricing
   curve is nearly flat, so that drift is a negligible price change, yet it was
   rejected.

2. Symmetric abs_diff rejected over-payments. ~36% of rejects had current <
   quoted: the node had FEWER records than when it quoted (prune/churn), so the
   client paid for a fuller, pricier node — an over-payment — and the node
   rejected it anyway. Rejecting money a node is owed is nonsensical.

Because an upload needs every chunk stored, even a ~40% per-chunk reject rate
drives upload success to ~0 (0.6^9 for a 9-chunk file; worse for large files).
So per-chunk rejection has to approach zero, not merely shrink.

Fix: make the gate price-based and one-directional. Compute the price the node
would charge now for its current fullness and reject only if the quote paid
less than that by more than QUOTE_PRICE_STALENESS_PCT_TOLERANCE (25%).

- Over-payment (paid >= current price) is always accepted (fixes #2).
- Comparing prices self-scales with the quadratic curve: benign churn on the
  flat part of the curve is ignored, while genuine staleness at high fill —
  where the curve is steep — is still caught (fixes #1).

Replaces the two record-delta tolerance constants with a single price
percentage. Reworks the freshness tests: a 10-record drift now passes (it was
"stale by 10" before), over-payment passes (regression test for #2), and a
quote underpricing by >25% is still rejected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant