Skip to content

[draft] Add cache hit stats to CCoinsViewCache#8

Draft
optout21 wants to merge 3 commits into
masterfrom
2605-coins-cache-stats
Draft

[draft] Add cache hit stats to CCoinsViewCache#8
optout21 wants to merge 3 commits into
masterfrom
2605-coins-cache-stats

Conversation

@optout21

@optout21 optout21 commented May 4, 2026

Copy link
Copy Markdown
Owner

The coins cache (CCoinsViewCache) caches part of the coins DB in memory for performance. It is crucial for validation performance during IBD, and (to a lesser extent) during steady state operation. The memory allocated to the cache is an important parameter, balanced between RAM usage and performance.

An important metric of the coins cache is the cache hit/miss ratio. This PR attempts to measure exactly that.

Some explanations/observations:

  • CCoinsViewCache acts as both a read cache and a write cache
  • The higher hit ratios are better, as there is less need to go down to the disk (LevelDB).
  • Larger cache size means higher hit ratios
  • When the cache grows over its maximum memory usage, it is flushed to disk, and cleared. The emptied cache means many misses.
  • There is a global cache (accessible through CoinsTip()). When a new block is validated, a temporary overlay cache (CoinsViewOverlay) is created on top of the global cache. Cache statistics are enabled for the global cache. Many accesses (for inputs) are caught by the overlay cache, so those don't shown up in the stats (only their first access).
  • Even with infinite RAM 100% hit ratio is not possible, because there are some checks for non-existent UTXOs.
  • To account for this better, misses due to "non-existent" checks are accounted separately
  • One source for non-existent checks is BIP30 checks: for each new TXO we need to check if it's already present. Fortunately on mainnet this affects only a relatively small set of early transactions (on Signet it's all the way).
  • Another source is in case of input checks on new block transactions, in the case they were produced in the same block. The overlay cache still doesn't have them, neither the base. Note: this is being addressed in validation: fetch block inputs on parallel threads bitcoin/bitcoin#31132.

The cache stats can be checked during raindex-chainstate:

bitcoind -reindex-chainstate -debug=coindb | grep -C1 coindb

Sample output (from Signet):

2026-05-04T11:39:45Z [coindb] UTXO cache stats: gets=201873004 miss=15321183 (7.59%) missNP=138044444 (68.38%) logreason=block period
2026-05-04T11:39:45Z UpdateTip: new best=0000000f66866455a6a64b8aed63f83c980c88b66ccc2c21abd9bb5fc93df6d2 height=302399 version=0x20000000 log2_work=43.698898 tx=29472770 date='2026-05-01T00:32:45Z' progress=0.999317 cache=85.0MiB(650339txo)

Including this in production code may be an overkill, there are several options:

  • include it in the production code, with occasional log lines (minimal non-zero overhead)
  • include it in the production code using Traces. Minimal overhead, can be turned on when needed.
  • include it in the production code with a build flag. No overhead, but can be turned on when needed.
  • don't include it, maintain as a patch/branch, use when needed. Has high maintenance cost and risk of diverging.

@optout21 optout21 changed the title Add cache hit stats to CCoinsViewCache [draft] Add cache hit stats to CCoinsViewCache May 4, 2026
@optout21
optout21 force-pushed the 2605-coins-cache-stats branch from d10e86e to e15908a Compare May 4, 2026 15:10
@optout21
optout21 force-pushed the 2605-coins-cache-stats branch from e15908a to 5073320 Compare May 4, 2026 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant