Remove the full analysis from benchmark comments#8233
Conversation
Signed-off-by: Adam Gutglick <adam@spiraldb.com>
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
Benchmarks: PolarSignals ProfilingVortex (geomean): 0.985x ➖ How to read Verdict and Engines
datafusion / vortex-file-compressed (0.985x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: FineWeb NVMeVerdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.913x ➖, 5↑ 0↓)
datafusion / vortex-compact (0.972x ➖, 1↑ 0↓)
datafusion / parquet (0.971x ➖, 1↑ 0↓)
duckdb / vortex-file-compressed (0.907x ➖, 2↑ 0↓)
duckdb / vortex-compact (0.914x ➖, 2↑ 0↓)
duckdb / parquet (0.950x ➖, 2↑ 0↓)
File Size Changes (1 files changed, -0.0% overall, 0↑ 1↓)
Totals:
|
Benchmarks: TPC-H SF=1 on NVMEVerdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.936x ➖, 2↑ 0↓)
datafusion / vortex-compact (0.958x ➖, 1↑ 0↓)
datafusion / parquet (0.964x ➖, 1↑ 2↓)
datafusion / arrow (0.918x ➖, 6↑ 0↓)
duckdb / vortex-file-compressed (0.953x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.953x ➖, 0↑ 0↓)
duckdb / parquet (0.951x ➖, 4↑ 0↓)
duckdb / duckdb (0.966x ➖, 0↑ 0↓)
File Size Changes (9 files changed, +0.0% overall, 5↑ 4↓)
Totals:
|
Merging this PR will improve performance by 20.94%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | bitwise_not_vortex_buffer_mut[128] |
246.1 ns | 275.3 ns | -10.6% |
| ⚡ | Simulation | chunked_bool_canonical_into[(1000, 10)] |
45.4 µs | 30.4 µs | +49.27% |
| ⚡ | Simulation | baseline_eq[4, 65536] |
237.8 µs | 185 µs | +28.55% |
| ⚡ | Simulation | baseline_lt[16, 65536] |
275.4 µs | 217.5 µs | +26.63% |
| ⚡ | Simulation | baseline_lt[4, 65536] |
253 µs | 200.8 µs | +26.01% |
| ⚡ | Simulation | chunked_varbinview_canonical_into[(1000, 10)] |
197.7 µs | 161.4 µs | +22.44% |
| ⚡ | Simulation | chunked_varbinview_into_canonical[(1000, 10)] |
212.3 µs | 176 µs | +20.64% |
| ⚡ | Simulation | baseline_eq[16, 65536] |
260.2 µs | 229.8 µs | +13.23% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing adamg/remove-full-details (b1d8275) with develop (7a53ad5)
Benchmarks: TPC-DS SF=1 on NVMEVerdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.954x ➖, 13↑ 0↓)
datafusion / vortex-compact (0.987x ➖, 4↑ 1↓)
datafusion / parquet (0.971x ➖, 17↑ 2↓)
duckdb / vortex-file-compressed (0.980x ➖, 3↑ 2↓)
duckdb / vortex-compact (0.979x ➖, 1↑ 0↓)
duckdb / parquet (0.994x ➖, 0↑ 0↓)
duckdb / duckdb (1.006x ➖, 2↑ 2↓)
File Size Changes (7 files changed, -0.0% overall, 4↑ 3↓)
Totals:
|
Benchmarks: FineWeb S3Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.134x ➖, 0↑ 1↓)
datafusion / vortex-compact (1.085x ➖, 0↑ 2↓)
datafusion / parquet (1.093x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (1.135x ➖, 0↑ 1↓)
duckdb / vortex-compact (1.080x ➖, 0↑ 1↓)
duckdb / parquet (1.068x ➖, 0↑ 0↓)
|
Benchmarks: Statistical and Population GeneticsVerdict: No clear signal (low confidence) How to read Verdict and Engines
duckdb / vortex-file-compressed (0.984x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.996x ➖, 0↑ 0↓)
duckdb / parquet (0.970x ➖, 0↑ 0↓)
File Size Changes (1 files changed, -0.0% overall, 0↑ 1↓)
Totals:
|
Benchmarks: TPC-H SF=10 on NVMEVerdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.953x ➖, 6↑ 0↓)
datafusion / vortex-compact (0.968x ➖, 0↑ 0↓)
datafusion / parquet (0.976x ➖, 0↑ 0↓)
datafusion / arrow (1.001x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (0.984x ➖, 1↑ 0↓)
duckdb / vortex-compact (0.983x ➖, 0↑ 0↓)
duckdb / parquet (0.973x ➖, 0↑ 0↓)
duckdb / duckdb (0.961x ➖, 0↑ 0↓)
File Size Changes (27 files changed, +0.0% overall, 14↑ 13↓)
Totals:
|
Benchmarks: Clickbench on NVMEVerdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.024x ➖, 1↑ 5↓)
datafusion / parquet (1.002x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (1.072x ➖, 1↑ 16↓)
duckdb / parquet (1.052x ➖, 0↑ 10↓)
duckdb / duckdb (0.959x ➖, 5↑ 0↓)
File Size Changes (106 files changed, -0.0% overall, 50↑ 56↓)
Totals:
|
Benchmarks: TPC-H SF=1 on S3Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.102x ➖, 0↑ 2↓)
datafusion / vortex-compact (1.237x ➖, 0↑ 7↓)
datafusion / parquet (0.999x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed (0.973x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.926x ➖, 0↑ 0↓)
duckdb / parquet (0.997x ➖, 0↑ 0↓)
|
Benchmarks: Appian on NVMEVerdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.953x ➖, 1↑ 0↓)
datafusion / parquet (0.955x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (1.000x ➖, 0↑ 0↓)
duckdb / parquet (0.999x ➖, 0↑ 0↓)
duckdb / duckdb (0.994x ➖, 0↑ 0↓)
File Size Changes (4 files changed, -0.0% overall, 2↑ 2↓)
Totals:
|
Benchmarks: TPC-H SF=10 on S3Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.034x ➖, 0↑ 2↓)
datafusion / vortex-compact (0.962x ➖, 0↑ 1↓)
datafusion / parquet (1.108x ➖, 0↑ 3↓)
duckdb / vortex-file-compressed (0.927x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.958x ➖, 0↑ 0↓)
duckdb / parquet (0.962x ➖, 0↑ 0↓)
|
Summary
Turns out we ran into the maximum github comment size, so time to make it smaller.