Skip to content

feat(blockstm): cache pre-state to optimize value-based validation#25909

Open
mmsqe wants to merge 14 commits intocosmos:mainfrom
mmsqe:cache_prestate
Open

feat(blockstm): cache pre-state to optimize value-based validation#25909
mmsqe wants to merge 14 commits intocosmos:mainfrom
mmsqe:cache_prestate

Conversation

@mmsqe
Copy link
Copy Markdown
Collaborator

@mmsqe mmsqe commented Feb 12, 2026

Description

  • cache pre-state for value-based validation to reduce expensive storage reads

Update: #25777

benchstat /private/tmp/stm_main.txt /private/tmp/stm_cache.txt
goos: darwin
goarch: arm64
pkg: github.com/cosmos/cosmos-sdk/internal/blockstm
cpu: Apple M4 Max
                                                │ /private/tmp/stm_main.txt │     /private/tmp/stm_cache.txt      │
                                                │          sec/op           │    sec/op     vs base               │
BlockSTM/random-10000/100-sequential-16                        831.7m ± ∞ ¹   769.4m ± ∞ ¹   -7.50% (p=0.029 n=4)
BlockSTM/random-10000/100-worker-1-16                          205.0m ± ∞ ¹   148.8m ± ∞ ¹  -27.40% (p=0.029 n=4)
BlockSTM/random-10000/100-worker-5-16                          14.36m ± ∞ ¹   14.36m ± ∞ ¹        ~ (p=0.686 n=4)
BlockSTM/random-10000/100-worker-10-16                         15.29m ± ∞ ¹   15.46m ± ∞ ¹        ~ (p=0.886 n=4)
BlockSTM/random-10000/100-worker-15-16                         18.82m ± ∞ ¹   19.07m ± ∞ ¹        ~ (p=0.057 n=4)
BlockSTM/random-10000/100-worker-20-16                         19.19m ± ∞ ¹   19.52m ± ∞ ¹        ~ (p=0.200 n=4)
BlockSTM/has-hit-10000/100-sequential-16                       190.1m ± ∞ ¹   190.8m ± ∞ ¹        ~ (p=0.343 n=4)
BlockSTM/has-hit-10000/100-worker-1-16                         430.2m ± ∞ ¹   281.1m ± ∞ ¹  -34.66% (p=0.029 n=4)
BlockSTM/has-hit-10000/100-worker-5-16                        117.35m ± ∞ ¹   68.04m ± ∞ ¹  -42.02% (p=0.029 n=4)
BlockSTM/has-hit-10000/100-worker-10-16                       119.82m ± ∞ ¹   64.88m ± ∞ ¹  -45.85% (p=0.029 n=4)
BlockSTM/has-hit-10000/100-worker-15-16                       151.88m ± ∞ ¹   62.09m ± ∞ ¹  -59.12% (p=0.029 n=4)
BlockSTM/has-hit-10000/100-worker-20-16                       156.22m ± ∞ ¹   55.68m ± ∞ ¹  -64.36% (p=0.029 n=4)
BlockSTM/has-miss-10000/100-sequential-16                      11.21m ± ∞ ¹   11.30m ± ∞ ¹        ~ (p=0.114 n=4)
BlockSTM/has-miss-10000/100-worker-1-16                        246.9m ± ∞ ¹   275.0m ± ∞ ¹  +11.41% (p=0.029 n=4)
BlockSTM/has-miss-10000/100-worker-5-16                        80.36m ± ∞ ¹   64.31m ± ∞ ¹  -19.98% (p=0.029 n=4)
BlockSTM/has-miss-10000/100-worker-10-16                      112.42m ± ∞ ¹   60.49m ± ∞ ¹  -46.19% (p=0.029 n=4)
BlockSTM/has-miss-10000/100-worker-15-16                      167.23m ± ∞ ¹   56.37m ± ∞ ¹  -66.29% (p=0.029 n=4)
BlockSTM/has-miss-10000/100-worker-20-16                      160.38m ± ∞ ¹   65.36m ± ∞ ¹  -59.25% (p=0.029 n=4)
BlockSTM/no-conflict-10000-sequential-16                       857.9m ± ∞ ¹   867.7m ± ∞ ¹   +1.15% (p=0.029 n=4)
BlockSTM/no-conflict-10000-worker-1-16                         348.3m ± ∞ ¹   353.8m ± ∞ ¹        ~ (p=0.114 n=4)
BlockSTM/no-conflict-10000-worker-5-16                         37.71m ± ∞ ¹   37.12m ± ∞ ¹        ~ (p=0.686 n=4)
BlockSTM/no-conflict-10000-worker-10-16                        34.87m ± ∞ ¹   37.43m ± ∞ ¹   +7.34% (p=0.029 n=4)
BlockSTM/no-conflict-10000-worker-15-16                        36.91m ± ∞ ¹   39.78m ± ∞ ¹   +7.78% (p=0.029 n=4)
BlockSTM/no-conflict-10000-worker-20-16                        39.82m ± ∞ ¹   41.74m ± ∞ ¹   +4.82% (p=0.029 n=4)
BlockSTM/worst-case-10000-sequential-16                        849.2m ± ∞ ¹   857.9m ± ∞ ¹   +1.02% (p=0.029 n=4)
BlockSTM/worst-case-10000-worker-1-16                          172.1m ± ∞ ¹   172.2m ± ∞ ¹        ~ (p=0.886 n=4)
BlockSTM/worst-case-10000-worker-5-16                          74.09m ± ∞ ¹   75.08m ± ∞ ¹        ~ (p=0.057 n=4)
BlockSTM/worst-case-10000-worker-10-16                         70.84m ± ∞ ¹   70.24m ± ∞ ¹   -0.84% (p=0.029 n=4)
BlockSTM/worst-case-10000-worker-15-16                         75.01m ± ∞ ¹   75.61m ± ∞ ¹        ~ (p=0.886 n=4)
BlockSTM/worst-case-10000-worker-20-16                         73.67m ± ∞ ¹   74.49m ± ∞ ¹   +1.11% (p=0.029 n=4)
BlockSTM/iterate-10000/100-sequential-16                       864.3m ± ∞ ¹   887.6m ± ∞ ¹   +2.71% (p=0.029 n=4)
BlockSTM/iterate-10000/100-worker-1-16                         506.8m ± ∞ ¹   512.1m ± ∞ ¹        ~ (p=0.200 n=4)
BlockSTM/iterate-10000/100-worker-5-16                         37.66m ± ∞ ¹   37.74m ± ∞ ¹        ~ (p=0.486 n=4)
BlockSTM/iterate-10000/100-worker-10-16                        43.21m ± ∞ ¹   43.35m ± ∞ ¹        ~ (p=0.200 n=4)
BlockSTM/iterate-10000/100-worker-15-16                        43.02m ± ∞ ¹   43.23m ± ∞ ¹        ~ (p=0.686 n=4)
BlockSTM/iterate-10000/100-worker-20-16                        45.17m ± ∞ ¹   47.23m ± ∞ ¹        ~ (p=0.114 n=4)
BlockSTM/iterate-10000/100-prepop-sequential-16                864.9m ± ∞ ¹   868.4m ± ∞ ¹        ~ (p=0.114 n=4)
BlockSTM/iterate-10000/100-prepop-worker-1-16                  507.7m ± ∞ ¹   509.2m ± ∞ ¹        ~ (p=0.200 n=4)
BlockSTM/iterate-10000/100-prepop-worker-5-16                  37.04m ± ∞ ¹   36.87m ± ∞ ¹        ~ (p=0.343 n=4)
BlockSTM/iterate-10000/100-prepop-worker-10-16                 43.04m ± ∞ ¹   42.91m ± ∞ ¹        ~ (p=0.343 n=4)
BlockSTM/iterate-10000/100-prepop-worker-15-16                 43.06m ± ∞ ¹   42.88m ± ∞ ¹        ~ (p=0.686 n=4)
BlockSTM/iterate-10000/100-prepop-worker-20-16                 44.38m ± ∞ ¹   45.52m ± ∞ ¹        ~ (p=0.343 n=4)
BlockSTM/iterate-newkeys-2000-sequential-16                    930.2µ ± ∞ ¹   945.4µ ± ∞ ¹        ~ (p=0.057 n=4)
BlockSTM/iterate-newkeys-2000-worker-1-16                      10.69m ± ∞ ¹   10.97m ± ∞ ¹        ~ (p=0.114 n=4)
BlockSTM/iterate-newkeys-2000-worker-5-16                      4.152m ± ∞ ¹   4.217m ± ∞ ¹        ~ (p=0.486 n=4)
BlockSTM/iterate-newkeys-2000-worker-10-16                     4.125m ± ∞ ¹   4.177m ± ∞ ¹        ~ (p=0.886 n=4)
BlockSTM/iterate-newkeys-2000-worker-15-16                     4.680m ± ∞ ¹   4.888m ± ∞ ¹   +4.46% (p=0.029 n=4)
BlockSTM/iterate-newkeys-2000-worker-20-16                     7.147m ± ∞ ¹   7.358m ± ∞ ¹        ~ (p=0.114 n=4)
geomean                                                        67.49m         59.46m        -11.90%
¹ need >= 6 samples for confidence interval at level 0.95

                                                │ /private/tmp/stm_main.txt │     /private/tmp/stm_cache.txt      │
                                                │         exec/txn          │  exec/txn    vs base                │
BlockSTM/random-10000/100-sequential-16                         1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/random-10000/100-worker-1-16                           1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/random-10000/100-worker-5-16                           1.069 ± ∞ ¹   1.067 ± ∞ ¹       ~ (p=0.971 n=4)
BlockSTM/random-10000/100-worker-10-16                          1.349 ± ∞ ¹   1.351 ± ∞ ¹       ~ (p=0.914 n=4)
BlockSTM/random-10000/100-worker-15-16                          1.958 ± ∞ ¹   1.961 ± ∞ ¹  +0.18% (p=0.029 n=4)
BlockSTM/random-10000/100-worker-20-16                          1.982 ± ∞ ¹   1.985 ± ∞ ¹       ~ (p=0.171 n=4)
BlockSTM/has-hit-10000/100-sequential-16                        1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-hit-10000/100-worker-1-16                          1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-hit-10000/100-worker-5-16                          1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-hit-10000/100-worker-10-16                         1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-hit-10000/100-worker-15-16                         1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-hit-10000/100-worker-20-16                         1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-miss-10000/100-sequential-16                       1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-miss-10000/100-worker-1-16                         1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-miss-10000/100-worker-5-16                         1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-miss-10000/100-worker-10-16                        1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-miss-10000/100-worker-15-16                        1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-miss-10000/100-worker-20-16                        1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/no-conflict-10000-sequential-16                        1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/no-conflict-10000-worker-1-16                          1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/no-conflict-10000-worker-5-16                          1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/no-conflict-10000-worker-10-16                         1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/no-conflict-10000-worker-15-16                         1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/no-conflict-10000-worker-20-16                         1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/worst-case-10000-sequential-16                         1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/worst-case-10000-worker-1-16                           1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/worst-case-10000-worker-5-16                           1.903 ± ∞ ¹   1.907 ± ∞ ¹  +0.18% (p=0.029 n=4)
BlockSTM/worst-case-10000-worker-10-16                          1.955 ± ∞ ¹   1.956 ± ∞ ¹       ~ (p=0.114 n=4)
BlockSTM/worst-case-10000-worker-15-16                          2.000 ± ∞ ¹   2.002 ± ∞ ¹  +0.12% (p=0.029 n=4)
BlockSTM/worst-case-10000-worker-20-16                          2.005 ± ∞ ¹   2.006 ± ∞ ¹       ~ (p=0.429 n=4)
BlockSTM/iterate-10000/100-sequential-16                        1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/iterate-10000/100-worker-1-16                          1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/iterate-10000/100-worker-5-16                          1.231 ± ∞ ¹   1.235 ± ∞ ¹       ~ (p=0.314 n=4)
BlockSTM/iterate-10000/100-worker-10-16                         1.801 ± ∞ ¹   1.802 ± ∞ ¹       ~ (p=0.486 n=4)
BlockSTM/iterate-10000/100-worker-15-16                         2.028 ± ∞ ¹   2.035 ± ∞ ¹       ~ (p=0.457 n=4)
BlockSTM/iterate-10000/100-worker-20-16                         2.191 ± ∞ ¹   2.205 ± ∞ ¹       ~ (p=0.171 n=4)
BlockSTM/iterate-10000/100-prepop-sequential-16                 1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/iterate-10000/100-prepop-worker-1-16                   1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/iterate-10000/100-prepop-worker-5-16                   1.232 ± ∞ ¹   1.230 ± ∞ ¹       ~ (p=0.114 n=4)
BlockSTM/iterate-10000/100-prepop-worker-10-16                  1.797 ± ∞ ¹   1.799 ± ∞ ¹       ~ (p=0.686 n=4)
BlockSTM/iterate-10000/100-prepop-worker-15-16                  2.027 ± ∞ ¹   2.028 ± ∞ ¹       ~ (p=0.686 n=4)
BlockSTM/iterate-10000/100-prepop-worker-20-16                  2.186 ± ∞ ¹   2.190 ± ∞ ¹       ~ (p=0.886 n=4)
BlockSTM/iterate-newkeys-2000-sequential-16                     1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/iterate-newkeys-2000-worker-1-16                       1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/iterate-newkeys-2000-worker-5-16                       1.010 ± ∞ ¹   1.009 ± ∞ ¹       ~ (p=1.000 n=4)
BlockSTM/iterate-newkeys-2000-worker-10-16                      1.029 ± ∞ ¹   1.028 ± ∞ ¹       ~ (p=0.743 n=4)
BlockSTM/iterate-newkeys-2000-worker-15-16                      1.126 ± ∞ ¹   1.126 ± ∞ ¹       ~ (p=0.886 n=4)
BlockSTM/iterate-newkeys-2000-worker-20-16                      1.793 ± ∞ ¹   1.812 ± ∞ ¹       ~ (p=0.486 n=4)
geomean                                                         1.225         1.226        +0.06%
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                                                │ /private/tmp/stm_main.txt │     /private/tmp/stm_cache.txt      │
                                                │          val/txn          │   val/txn    vs base                │
BlockSTM/random-10000/100-sequential-16                         0.000 ± ∞ ¹   0.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/random-10000/100-worker-1-16                           1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/random-10000/100-worker-5-16                           1.113 ± ∞ ¹   1.113 ± ∞ ¹       ~ (p=1.000 n=4)
BlockSTM/random-10000/100-worker-10-16                          1.568 ± ∞ ¹   1.572 ± ∞ ¹       ~ (p=0.886 n=4)
BlockSTM/random-10000/100-worker-15-16                          2.028 ± ∞ ¹   2.034 ± ∞ ¹       ~ (p=0.057 n=4)
BlockSTM/random-10000/100-worker-20-16                          2.050 ± ∞ ¹   2.051 ± ∞ ¹       ~ (p=0.400 n=4)
BlockSTM/has-hit-10000/100-sequential-16                        0.000 ± ∞ ¹   0.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-hit-10000/100-worker-1-16                          1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-hit-10000/100-worker-5-16                          1.001 ± ∞ ¹   1.002 ± ∞ ¹       ~ (p=0.086 n=4)
BlockSTM/has-hit-10000/100-worker-10-16                         1.001 ± ∞ ¹   1.004 ± ∞ ¹  +0.30% (p=0.029 n=4)
BlockSTM/has-hit-10000/100-worker-15-16                         1.001 ± ∞ ¹   1.001 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-hit-10000/100-worker-20-16                         1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-miss-10000/100-sequential-16                       0.000 ± ∞ ¹   0.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-miss-10000/100-worker-1-16                         1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-miss-10000/100-worker-5-16                         1.002 ± ∞ ¹   1.002 ± ∞ ¹       ~ (p=1.000 n=4)
BlockSTM/has-miss-10000/100-worker-10-16                        1.001 ± ∞ ¹   1.005 ± ∞ ¹  +0.40% (p=0.029 n=4)
BlockSTM/has-miss-10000/100-worker-15-16                        1.001 ± ∞ ¹   1.001 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-miss-10000/100-worker-20-16                        1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/no-conflict-10000-sequential-16                        0.000 ± ∞ ¹   0.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/no-conflict-10000-worker-1-16                          1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/no-conflict-10000-worker-5-16                          1.003 ± ∞ ¹   1.003 ± ∞ ¹       ~ (p=1.000 n=4)
BlockSTM/no-conflict-10000-worker-10-16                         1.004 ± ∞ ¹   1.004 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/no-conflict-10000-worker-15-16                         1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4)
BlockSTM/no-conflict-10000-worker-20-16                         1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/worst-case-10000-sequential-16                         0.000 ± ∞ ¹   0.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/worst-case-10000-worker-1-16                           1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/worst-case-10000-worker-5-16                           1.911 ± ∞ ¹   1.914 ± ∞ ¹  +0.16% (p=0.029 n=4)
BlockSTM/worst-case-10000-worker-10-16                          1.964 ± ∞ ¹   1.968 ± ∞ ¹       ~ (p=0.143 n=4)
BlockSTM/worst-case-10000-worker-15-16                          2.012 ± ∞ ¹   2.016 ± ∞ ¹  +0.22% (p=0.029 n=4)
BlockSTM/worst-case-10000-worker-20-16                          2.018 ± ∞ ¹   2.019 ± ∞ ¹       ~ (p=0.629 n=4)
BlockSTM/iterate-10000/100-sequential-16                        0.000 ± ∞ ¹   0.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/iterate-10000/100-worker-1-16                          1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/iterate-10000/100-worker-5-16                          1.352 ± ∞ ¹   1.357 ± ∞ ¹       ~ (p=0.143 n=4)
BlockSTM/iterate-10000/100-worker-10-16                         1.977 ± ∞ ¹   1.978 ± ∞ ¹       ~ (p=0.486 n=4)
BlockSTM/iterate-10000/100-worker-15-16                         2.215 ± ∞ ¹   2.227 ± ∞ ¹       ~ (p=0.429 n=4)
BlockSTM/iterate-10000/100-worker-20-16                         2.645 ± ∞ ¹   2.692 ± ∞ ¹       ~ (p=0.143 n=4)
BlockSTM/iterate-10000/100-prepop-sequential-16                 0.000 ± ∞ ¹   0.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/iterate-10000/100-prepop-worker-1-16                   1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/iterate-10000/100-prepop-worker-5-16                   1.356 ± ∞ ¹   1.351 ± ∞ ¹       ~ (p=0.200 n=4)
BlockSTM/iterate-10000/100-prepop-worker-10-16                  1.974 ± ∞ ¹   1.974 ± ∞ ¹       ~ (p=0.971 n=4)
BlockSTM/iterate-10000/100-prepop-worker-15-16                  2.219 ± ∞ ¹   2.220 ± ∞ ¹       ~ (p=0.686 n=4)
BlockSTM/iterate-10000/100-prepop-worker-20-16                  2.623 ± ∞ ¹   2.649 ± ∞ ¹       ~ (p=0.343 n=4)
BlockSTM/iterate-newkeys-2000-sequential-16                     0.000 ± ∞ ¹   0.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/iterate-newkeys-2000-worker-1-16                       1.000 ± ∞ ¹   1.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/iterate-newkeys-2000-worker-5-16                       1.014 ± ∞ ¹   1.014 ± ∞ ¹       ~ (p=1.000 n=4)
BlockSTM/iterate-newkeys-2000-worker-10-16                      1.037 ± ∞ ¹   1.036 ± ∞ ¹       ~ (p=0.571 n=4)
BlockSTM/iterate-newkeys-2000-worker-15-16                      1.147 ± ∞ ¹   1.152 ± ∞ ¹       ~ (p=0.343 n=4)
BlockSTM/iterate-newkeys-2000-worker-20-16                      1.989 ± ∞ ¹   2.010 ± ∞ ¹       ~ (p=0.486 n=4)
geomean                                                                   ³                +0.14%               ³
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal
³ summaries must be >0 to compute geomean

                                                │ /private/tmp/stm_main.txt │      /private/tmp/stm_cache.txt       │
                                                │           B/op            │     B/op       vs base                │
BlockSTM/random-10000/100-sequential-16                       9.395Mi ± ∞ ¹   9.395Mi ± ∞ ¹       ~ (p=0.429 n=4)
BlockSTM/random-10000/100-worker-1-16                         33.16Mi ± ∞ ¹   32.55Mi ± ∞ ¹  -1.84% (p=0.029 n=4)
BlockSTM/random-10000/100-worker-5-16                         34.45Mi ± ∞ ¹   34.41Mi ± ∞ ¹       ~ (p=1.000 n=4)
BlockSTM/random-10000/100-worker-10-16                        48.13Mi ± ∞ ¹   48.22Mi ± ∞ ¹       ~ (p=0.886 n=4)
BlockSTM/random-10000/100-worker-15-16                        76.47Mi ± ∞ ¹   76.70Mi ± ∞ ¹  +0.30% (p=0.029 n=4)
BlockSTM/random-10000/100-worker-20-16                        77.78Mi ± ∞ ¹   77.95Mi ± ∞ ¹       ~ (p=0.057 n=4)
BlockSTM/has-hit-10000/100-sequential-16                        0.000 ± ∞ ¹     0.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-hit-10000/100-worker-1-16                        277.0Mi ± ∞ ¹   278.6Mi ± ∞ ¹  +0.57% (p=0.029 n=4)
BlockSTM/has-hit-10000/100-worker-5-16                        277.1Mi ± ∞ ¹   278.9Mi ± ∞ ¹  +0.65% (p=0.029 n=4)
BlockSTM/has-hit-10000/100-worker-10-16                       277.1Mi ± ∞ ¹   279.2Mi ± ∞ ¹  +0.76% (p=0.029 n=4)
BlockSTM/has-hit-10000/100-worker-15-16                       277.1Mi ± ∞ ¹   279.1Mi ± ∞ ¹  +0.74% (p=0.029 n=4)
BlockSTM/has-hit-10000/100-worker-20-16                       277.0Mi ± ∞ ¹   279.0Mi ± ∞ ¹  +0.72% (p=0.029 n=4)
BlockSTM/has-miss-10000/100-sequential-16                       0.000 ± ∞ ¹     0.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-miss-10000/100-worker-1-16                       277.0Mi ± ∞ ¹   278.4Mi ± ∞ ¹  +0.49% (p=0.029 n=4)
BlockSTM/has-miss-10000/100-worker-5-16                       277.1Mi ± ∞ ¹   278.5Mi ± ∞ ¹  +0.50% (p=0.029 n=4)
BlockSTM/has-miss-10000/100-worker-10-16                      277.1Mi ± ∞ ¹   278.8Mi ± ∞ ¹  +0.63% (p=0.029 n=4)
BlockSTM/has-miss-10000/100-worker-15-16                      277.1Mi ± ∞ ¹   278.6Mi ± ∞ ¹  +0.56% (p=0.029 n=4)
BlockSTM/has-miss-10000/100-worker-20-16                      277.0Mi ± ∞ ¹   278.5Mi ± ∞ ¹  +0.55% (p=0.029 n=4)
BlockSTM/no-conflict-10000-sequential-16                      10.29Mi ± ∞ ¹   10.29Mi ± ∞ ¹       ~ (p=0.143 n=4)
BlockSTM/no-conflict-10000-worker-1-16                        76.86Mi ± ∞ ¹   80.04Mi ± ∞ ¹  +4.13% (p=0.029 n=4)
BlockSTM/no-conflict-10000-worker-5-16                        94.38Mi ± ∞ ¹   95.88Mi ± ∞ ¹  +1.59% (p=0.029 n=4)
BlockSTM/no-conflict-10000-worker-10-16                       147.0Mi ± ∞ ¹   147.0Mi ± ∞ ¹       ~ (p=0.686 n=4)
BlockSTM/no-conflict-10000-worker-15-16                       176.3Mi ± ∞ ¹   168.5Mi ± ∞ ¹  -4.44% (p=0.029 n=4)
BlockSTM/no-conflict-10000-worker-20-16                       163.7Mi ± ∞ ¹   167.0Mi ± ∞ ¹       ~ (p=0.114 n=4)
BlockSTM/worst-case-10000-sequential-16                       9.155Mi ± ∞ ¹   9.155Mi ± ∞ ¹       ~ (p=0.143 n=4)
BlockSTM/worst-case-10000-worker-1-16                         30.77Mi ± ∞ ¹   30.77Mi ± ∞ ¹       ~ (p=0.200 n=4)
BlockSTM/worst-case-10000-worker-5-16                         72.56Mi ± ∞ ¹   72.72Mi ± ∞ ¹  +0.22% (p=0.029 n=4)
BlockSTM/worst-case-10000-worker-10-16                        75.02Mi ± ∞ ¹   75.08Mi ± ∞ ¹       ~ (p=0.114 n=4)
BlockSTM/worst-case-10000-worker-15-16                        77.26Mi ± ∞ ¹   77.39Mi ± ∞ ¹  +0.17% (p=0.029 n=4)
BlockSTM/worst-case-10000-worker-20-16                        77.52Mi ± ∞ ¹   77.54Mi ± ∞ ¹       ~ (p=0.343 n=4)
BlockSTM/iterate-10000/100-sequential-16                      16.02Mi ± ∞ ¹   16.02Mi ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/iterate-10000/100-worker-1-16                        78.73Mi ± ∞ ¹   78.76Mi ± ∞ ¹  +0.04% (p=0.029 n=4)
BlockSTM/iterate-10000/100-worker-5-16                        94.61Mi ± ∞ ¹   94.95Mi ± ∞ ¹       ~ (p=0.200 n=4)
BlockSTM/iterate-10000/100-worker-10-16                       144.0Mi ± ∞ ¹   144.1Mi ± ∞ ¹       ~ (p=0.343 n=4)
BlockSTM/iterate-10000/100-worker-15-16                       162.1Mi ± ∞ ¹   162.7Mi ± ∞ ¹       ~ (p=0.486 n=4)
BlockSTM/iterate-10000/100-worker-20-16                       176.9Mi ± ∞ ¹   178.4Mi ± ∞ ¹       ~ (p=0.200 n=4)
BlockSTM/iterate-10000/100-prepop-sequential-16               16.01Mi ± ∞ ¹   16.01Mi ± ∞ ¹       ~ (p=1.000 n=4)
BlockSTM/iterate-10000/100-prepop-worker-1-16                 78.66Mi ± ∞ ¹   78.70Mi ± ∞ ¹  +0.04% (p=0.029 n=4)
BlockSTM/iterate-10000/100-prepop-worker-5-16                 94.72Mi ± ∞ ¹   94.49Mi ± ∞ ¹       ~ (p=0.114 n=4)
BlockSTM/iterate-10000/100-prepop-worker-10-16                143.7Mi ± ∞ ¹   143.8Mi ± ∞ ¹       ~ (p=0.486 n=4)
BlockSTM/iterate-10000/100-prepop-worker-15-16                162.2Mi ± ∞ ¹   162.1Mi ± ∞ ¹       ~ (p=0.886 n=4)
BlockSTM/iterate-10000/100-prepop-worker-20-16                176.0Mi ± ∞ ¹   176.7Mi ± ∞ ¹       ~ (p=0.686 n=4)
BlockSTM/iterate-newkeys-2000-sequential-16                   1.217Mi ± ∞ ¹   1.217Mi ± ∞ ¹       ~ (p=0.714 n=4)
BlockSTM/iterate-newkeys-2000-worker-1-16                     15.11Mi ± ∞ ¹   15.11Mi ± ∞ ¹  +0.01% (p=0.029 n=4)
BlockSTM/iterate-newkeys-2000-worker-5-16                     17.21Mi ± ∞ ¹   17.20Mi ± ∞ ¹       ~ (p=0.886 n=4)
BlockSTM/iterate-newkeys-2000-worker-10-16                    22.08Mi ± ∞ ¹   22.06Mi ± ∞ ¹       ~ (p=0.343 n=4)
BlockSTM/iterate-newkeys-2000-worker-15-16                    28.34Mi ± ∞ ¹   27.95Mi ± ∞ ¹  -1.36% (p=0.029 n=4)
BlockSTM/iterate-newkeys-2000-worker-20-16                    39.15Mi ± ∞ ¹   39.08Mi ± ∞ ¹       ~ (p=1.000 n=4)
geomean                                                                   ³                  +0.18%               ³
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal
³ summaries must be >0 to compute geomean

                                                │ /private/tmp/stm_main.txt │      /private/tmp/stm_cache.txt      │
                                                │         allocs/op         │  allocs/op    vs base                │
BlockSTM/random-10000/100-sequential-16                        220.0k ± ∞ ¹   220.0k ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/random-10000/100-worker-1-16                          660.0k ± ∞ ¹   650.6k ± ∞ ¹  -1.43% (p=0.029 n=4)
BlockSTM/random-10000/100-worker-5-16                          675.6k ± ∞ ¹   675.5k ± ∞ ¹       ~ (p=1.000 n=4)
BlockSTM/random-10000/100-worker-10-16                         874.8k ± ∞ ¹   876.6k ± ∞ ¹       ~ (p=0.886 n=4)
BlockSTM/random-10000/100-worker-15-16                         1.265M ± ∞ ¹   1.268M ± ∞ ¹  +0.28% (p=0.029 n=4)
BlockSTM/random-10000/100-worker-20-16                         1.285M ± ∞ ¹   1.288M ± ∞ ¹  +0.20% (p=0.029 n=4)
BlockSTM/has-hit-10000/100-sequential-16                        0.000 ± ∞ ¹    0.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-hit-10000/100-worker-1-16                         5.200M ± ∞ ¹   5.244M ± ∞ ¹  +0.84% (p=0.029 n=4)
BlockSTM/has-hit-10000/100-worker-5-16                         5.202M ± ∞ ¹   5.254M ± ∞ ¹  +0.99% (p=0.029 n=4)
BlockSTM/has-hit-10000/100-worker-10-16                        5.202M ± ∞ ¹   5.266M ± ∞ ¹  +1.23% (p=0.029 n=4)
BlockSTM/has-hit-10000/100-worker-15-16                        5.202M ± ∞ ¹   5.264M ± ∞ ¹  +1.20% (p=0.029 n=4)
BlockSTM/has-hit-10000/100-worker-20-16                        5.200M ± ∞ ¹   5.261M ± ∞ ¹  +1.17% (p=0.029 n=4)
BlockSTM/has-miss-10000/100-sequential-16                       0.000 ± ∞ ¹    0.000 ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/has-miss-10000/100-worker-1-16                        5.200M ± ∞ ¹   5.234M ± ∞ ¹  +0.65% (p=0.029 n=4)
BlockSTM/has-miss-10000/100-worker-5-16                        5.204M ± ∞ ¹   5.240M ± ∞ ¹  +0.68% (p=0.029 n=4)
BlockSTM/has-miss-10000/100-worker-10-16                       5.202M ± ∞ ¹   5.250M ± ∞ ¹  +0.91% (p=0.029 n=4)
BlockSTM/has-miss-10000/100-worker-15-16                       5.202M ± ∞ ¹   5.243M ± ∞ ¹  +0.79% (p=0.029 n=4)
BlockSTM/has-miss-10000/100-worker-20-16                       5.200M ± ∞ ¹   5.241M ± ∞ ¹  +0.78% (p=0.029 n=4)
BlockSTM/no-conflict-10000-sequential-16                       220.6k ± ∞ ¹   220.6k ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/no-conflict-10000-worker-1-16                         1.225M ± ∞ ¹   1.306M ± ∞ ¹  +6.62% (p=0.029 n=4)
BlockSTM/no-conflict-10000-worker-5-16                         1.414M ± ∞ ¹   1.481M ± ∞ ¹  +4.72% (p=0.029 n=4)
BlockSTM/no-conflict-10000-worker-10-16                        2.009M ± ∞ ¹   2.060M ± ∞ ¹  +2.52% (p=0.029 n=4)
BlockSTM/no-conflict-10000-worker-15-16                        2.372M ± ∞ ¹   2.326M ± ∞ ¹       ~ (p=0.114 n=4)
BlockSTM/no-conflict-10000-worker-20-16                        2.228M ± ∞ ¹   2.313M ± ∞ ¹  +3.85% (p=0.029 n=4)
BlockSTM/worst-case-10000-sequential-16                        220.0k ± ∞ ¹   220.0k ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/worst-case-10000-worker-1-16                          620.2k ± ∞ ¹   620.2k ± ∞ ¹       ~ (p=0.114 n=4)
BlockSTM/worst-case-10000-worker-5-16                          1.228M ± ∞ ¹   1.231M ± ∞ ¹  +0.19% (p=0.029 n=4)
BlockSTM/worst-case-10000-worker-10-16                         1.264M ± ∞ ¹   1.265M ± ∞ ¹       ~ (p=0.114 n=4)
BlockSTM/worst-case-10000-worker-15-16                         1.297M ± ∞ ¹   1.299M ± ∞ ¹  +0.15% (p=0.029 n=4)
BlockSTM/worst-case-10000-worker-20-16                         1.301M ± ∞ ¹   1.301M ± ∞ ¹       ~ (p=0.343 n=4)
BlockSTM/iterate-10000/100-sequential-16                       292.6k ± ∞ ¹   292.6k ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/iterate-10000/100-worker-1-16                         1.136M ± ∞ ¹   1.137M ± ∞ ¹  +0.07% (p=0.029 n=4)
BlockSTM/iterate-10000/100-worker-5-16                         1.310M ± ∞ ¹   1.315M ± ∞ ¹       ~ (p=0.200 n=4)
BlockSTM/iterate-10000/100-worker-10-16                        1.911M ± ∞ ¹   1.913M ± ∞ ¹       ~ (p=0.343 n=4)
BlockSTM/iterate-10000/100-worker-15-16                        2.139M ± ∞ ¹   2.148M ± ∞ ¹       ~ (p=0.486 n=4)
BlockSTM/iterate-10000/100-worker-20-16                        2.336M ± ∞ ¹   2.355M ± ∞ ¹       ~ (p=0.200 n=4)
BlockSTM/iterate-10000/100-prepop-sequential-16                292.5k ± ∞ ¹   292.5k ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/iterate-10000/100-prepop-worker-1-16                  1.136M ± ∞ ¹   1.137M ± ∞ ¹  +0.08% (p=0.029 n=4)
BlockSTM/iterate-10000/100-prepop-worker-5-16                  1.311M ± ∞ ¹   1.309M ± ∞ ¹       ~ (p=0.200 n=4)
BlockSTM/iterate-10000/100-prepop-worker-10-16                 1.908M ± ∞ ¹   1.910M ± ∞ ¹       ~ (p=0.486 n=4)
BlockSTM/iterate-10000/100-prepop-worker-15-16                 2.140M ± ∞ ¹   2.141M ± ∞ ¹       ~ (p=0.686 n=4)
BlockSTM/iterate-10000/100-prepop-worker-20-16                 2.325M ± ∞ ¹   2.335M ± ∞ ¹       ~ (p=0.486 n=4)
BlockSTM/iterate-newkeys-2000-sequential-16                    13.74k ± ∞ ¹   13.74k ± ∞ ¹       ~ (p=1.000 n=4) ²
BlockSTM/iterate-newkeys-2000-worker-1-16                      155.2k ± ∞ ¹   155.2k ± ∞ ¹  +0.02% (p=0.029 n=4)
BlockSTM/iterate-newkeys-2000-worker-5-16                      178.8k ± ∞ ¹   178.7k ± ∞ ¹       ~ (p=1.000 n=4)
BlockSTM/iterate-newkeys-2000-worker-10-16                     234.0k ± ∞ ¹   233.9k ± ∞ ¹       ~ (p=0.686 n=4)
BlockSTM/iterate-newkeys-2000-worker-15-16                     307.1k ± ∞ ¹   302.5k ± ∞ ¹  -1.49% (p=0.029 n=4)
BlockSTM/iterate-newkeys-2000-worker-20-16                     408.3k ± ∞ ¹   407.1k ± ∞ ¹       ~ (p=0.886 n=4)
geomean                                                                   ³                 +0.51%               ³
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal
³ summaries must be >0 to compute geomean

@codecov
Copy link
Copy Markdown

codecov bot commented Feb 12, 2026

Codecov Report

❌ Patch coverage is 82.08955% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.05%. Comparing base (d17338a) to head (f2385fb).

Files with missing lines Patch % Lines
internal/blockstm/storage.go 75.00% 8 Missing ⚠️
internal/blockstm/stm.go 89.28% 3 Missing ⚠️
internal/blockstm/mvview.go 75.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main   #25909      +/-   ##
==========================================
- Coverage   60.11%   60.05%   -0.06%     
==========================================
  Files         976      961      -15     
  Lines       64970    64044     -926     
==========================================
- Hits        39057    38463     -594     
+ Misses      25913    25581     -332     
Files with missing lines Coverage Δ
internal/blockstm/mvmemory.go 100.00% <100.00%> (ø)
internal/blockstm/mvview.go 92.45% <75.00%> (-0.08%) ⬇️
internal/blockstm/stm.go 88.33% <89.28%> (+4.61%) ⬆️
internal/blockstm/storage.go 75.00% <75.00%> (ø)

... and 19 files with indirect coverage changes

Impacted file tree graph

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@aljo242
Copy link
Copy Markdown
Contributor

aljo242 commented Feb 12, 2026

@mmsqe conflicts due to file movements sorry 😅

@mmsqe mmsqe force-pushed the cache_prestate branch 2 times, most recently from f9cdc06 to 9329e96 Compare February 13, 2026 12:33
@mmsqe mmsqe marked this pull request as draft February 27, 2026 05:09
@mmsqe mmsqe marked this pull request as ready for review March 4, 2026 10:32
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 4, 2026

Greptile Summary

This PR optimizes BlockSTM execution by caching pre-state storage reads, reducing expensive storage access during validation. The implementation adds a sync.Map-based cache per store that caches values when reading from storage (not from MVMemory). Key implementation details:

  • Cache scope: Per-block, per-store - created with MVMemory and garbage collected after block execution
  • Size limit: Byte slices larger than 4KB are not cached to prevent memory bloat
  • Thread safety: Uses sync.Map for concurrent access from parallel transactions
  • Integration: Only caches reads when !version.Valid() (pre-state reads), doesn't interfere with reads of values written by transactions

The refactoring in stm.go separates telemetry emission from execution logic and changes from IncrCounter to SetGauge for per-block metrics (more semantically correct for block-level stats).

Benchmark results show substantial improvements:

  • has-hit scenarios: 35-51% faster
  • has-miss scenarios: up to 60% faster
  • aba-samevalue scenarios benefit from reduced validation overhead
  • Overall geometric mean improvement: 8.73% faster execution

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk
  • The implementation is clean, well-designed, and correctly implements the caching optimization. Thread safety is properly handled with sync.Map, cache scope prevents memory leaks, and the cache doesn't affect correctness (only used for pre-state reads). The comprehensive benchmarks validate the performance improvements.
  • No files require special attention

Important Files Changed

Filename Overview
internal/blockstm/mvmemory.go Added preStateCache using sync.Map to cache pre-state storage reads. Clean implementation with proper initialization and thread-safe access patterns.
internal/blockstm/mvview.go Implemented getFromPreState method with intelligent caching (4KB size limit for byte slices). Correctly integrated into the read path for pre-state values only.
internal/blockstm/stm.go Refactored execution logic to return metrics and control telemetry emission. Changed telemetry from counters to gauges for per-block metrics. Clean separation of concerns.
internal/blockstm/bench_test.go Added comprehensive benchmark scenarios including has-hit/miss, aba-samevalue, and iterate-newkeys tests. Properly reports execution and validation metrics.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Transaction reads key] --> B{Check WriteSet}
    B -->|Found| C[Return from WriteSet]
    B -->|Not found| D[Read from MVMemory]
    D --> E{Version valid?}
    E -->|Yes| F[Return value from MVMemory]
    E -->|No - pre-state read| G{Check preState cache}
    G -->|Cache hit| H[Return cached value]
    G -->|Cache miss| I[Read from storage]
    I --> J{Value size check}
    J -->|Byte slice <= 4KB| K[Store in cache]
    J -->|Byte slice > 4KB or non-byte| L{Is byte slice?}
    L -->|No - any type| K
    L -->|Yes - too large| M[Skip caching]
    K --> N[Return value]
    M --> N
Loading

Last reviewed commit: 911a426

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants