Use aggregate descriptors for zoned stats#7938
Conversation
772d188 to
6e1ab90
Compare
c69b282 to
f9cf88c
Compare
Merging this PR will not alter performance
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | decompress_rd[f64, (10000, 0.01)] |
108.7 µs | 138.8 µs | -21.73% |
| ❌ | Simulation | decompress_rd[f64, (10000, 0.1)] |
109 µs | 139.2 µs | -21.71% |
| ❌ | Simulation | decompress_rd[f64, (10000, 0.0)] |
108.7 µs | 138.8 µs | -21.7% |
| ❌ | Simulation | decompress_rd[f32, (100000, 0.0)] |
496 µs | 583.5 µs | -15% |
| ❌ | Simulation | decompress_rd[f32, (10000, 0.1)] |
78.1 µs | 90.3 µs | -13.56% |
| ❌ | Simulation | decompress_rd[f32, (10000, 0.01)] |
78.1 µs | 90.3 µs | -13.56% |
| ❌ | Simulation | decompress_rd[f32, (10000, 0.0)] |
78.5 µs | 90.8 µs | -13.52% |
| ⚡ | Simulation | take_10k_first_chunk_only |
253.1 µs | 208.2 µs | +21.55% |
| ⚡ | Simulation | decompress_rd[f64, (100000, 0.1)] |
1,020.7 µs | 842.5 µs | +21.15% |
| ⚡ | Simulation | decompress_rd[f64, (100000, 0.01)] |
1,020.7 µs | 842.5 µs | +21.15% |
| ⚡ | Simulation | take_10k_dispersed |
266.6 µs | 221.8 µs | +20.18% |
| ⚡ | Simulation | decompress_rd[f32, (100000, 0.1)] |
582.9 µs | 495 µs | +17.75% |
| ⚡ | Simulation | decompress_rd[f32, (100000, 0.01)] |
582.9 µs | 495 µs | +17.75% |
| ⚡ | Simulation | bitwise_not_vortex_buffer_mut[128] |
244.4 ns | 215.3 ns | +13.55% |
| ⚡ | Simulation | patched_take_10k_adversarial |
260.9 µs | 230.5 µs | +13.16% |
| ⚡ | Simulation | patched_take_10k_first_chunk_only |
284.7 µs | 254.4 µs | +11.92% |
| ⚡ | Simulation | patched_take_10k_dispersed |
298.9 µs | 268.6 µs | +11.28% |
| ⚡ | Simulation | bitwise_not_vortex_buffer_mut[1024] |
304.7 ns | 275.6 ns | +10.58% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing ngates/stats-7707/aggregate-zoned-stats (02e7409) with develop (5d55fbc)
bc8f53c to
6ceb393
Compare
f9cf88c to
ed63b11
Compare
6ceb393 to
549e240
Compare
ed63b11 to
0d3003c
Compare
549e240 to
8b3da0d
Compare
0d3003c to
096128a
Compare
096128a to
68e19bb
Compare
8b3da0d to
6abd91b
Compare
68e19bb to
0473953
Compare
6abd91b to
c04ebf9
Compare
0473953 to
f14f6a1
Compare
c04ebf9 to
e3de60b
Compare
f14f6a1 to
fc97c20
Compare
c112569 to
95145a1
Compare
|
This PR has been marked as stale because it has been open for 14 days with no activity. Please comment or remove the stale label if you wish to keep it active, otherwise it will be closed in 7 days |
Signed-off-by: "Nicholas Gates" <nick@nickgates.com> Signed-off-by: Nicholas Gates <nick@nickgates.com>
eb8f266 to
1f13453
Compare
robert3005
left a comment
There was a problem hiding this comment.
I think we can delete some of the old write path? We no longer seem to be using old stats accumulators
Move the legacy Stat accumulator out of the zoned builder and into file stats, where it is still needed for footer statistics. Also derive the layout registry from VortexSession when loading layout flatbuffers. Signed-off-by: "Nicholas Gates" <nick@nickgates.com> Signed-off-by: Nicholas Gates <nick@nickgates.com>
Install LayoutSession in the unknown-layout test fixture now that layout deserialization resolves the registry through VortexSession. Signed-off-by: "Nicholas Gates" <nick@nickgates.com> Signed-off-by: Nicholas Gates <nick@nickgates.com>
Remove the aggregate descriptor helper and cached descriptor field from the zoned stats builder. Use AggregateFnRef Display directly for stats-table field names, and use const NonZeroUsize construction for bounded stat byte defaults. Signed-off-by: "Nicholas Gates" <nick@nickgates.com> Signed-off-by: Nicholas Gates <nick@nickgates.com>
Remove the separate in-memory AggregateSpec wrapper. Keep the prost aggregate spec as the raw metadata form, deserialize it into AggregateFnRef during layout build, and store aggregate functions on the runtime layout. Signed-off-by: "Nicholas Gates" <nick@nickgates.com> Signed-off-by: Nicholas Gates <nick@nickgates.com>
Replace encoding-id string matching with layout.is::<Zoned>() and layout.is::<LegacyStats>() in the zoned segment-ordering test. Signed-off-by: "Nicholas Gates" <nick@nickgates.com> Signed-off-by: Nicholas Gates <nick@nickgates.com>
This PR adds a new layout type: vortex.zoned
During deserialization, we read the old zone map layout
vortex.statsinto avortex.zonedin memory.The new zoned layout now stores aggregate descriptors instead of legacy
Statbits, enabling arbitrary aggregate functions to be used as zone statistics.FLUPs: