docs: update incremental benchmarks (3.4.0)#627
Conversation
Greptile SummaryThis is an automated benchmark update that appends 3.4.0 measurement data to Confidence Score: 5/5Safe to merge — purely a docs update with correct, internally consistent benchmark data. Single auto-generated docs file; no code changes. All numeric values cross-check correctly between the summary table, Latest results section, and embedded JSON block. The only known concern (1-file rebuild regression) was already acknowledged in the prior thread and is being tracked separately in #635. No files require special attention. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Release workflow publish.yml] -->|workflow_dispatch| B[Run benchmarks on own codebase]
B --> C{Engine}
C -->|Native Rust| D[Full build / No-op / 1-file timings]
C -->|WASM| E[Full build / No-op / 1-file timings]
D --> F[Compute import resolution stats]
E --> F
F --> G[Calculate % deltas vs previous version]
G --> H[Update summary table row]
G --> I[Update Latest results section]
G --> J[Append JSON entry to embedded data block]
H --> K[INCREMENTAL-BENCHMARKS.md]
I --> K
J --> K
K --> L[PR #627 opened by github-actions bot]
Reviews (2): Last reviewed commit: "Merge branch 'main' into benchmark/incre..." | Re-trigger Greptile |
| | 3.4.0 | native | 473 | 2.3s ↓4% | 12ms ↑9% | 448ms ↑29% | 4ms ↓58% | 12ms ↓54% | | ||
| | 3.4.0 | wasm | 473 | 5.0s ↑15% | 12ms ↑20% | 617ms ↑21% | 4ms ↓58% | 12ms ↓54% | |
There was a problem hiding this comment.
Disproportionate 1-file rebuild regression
The 1-file rebuild time increased +29% for native (346ms → 448ms) and +21% for WASM (508ms → 617ms), but the file count only grew by ~7% (442 → 473 files). For a single-file incremental rebuild the total file count should have minimal impact on latency, so this regression is disproportionate to the codebase growth.
The WASM full build also regressed +15% (4.4s → 5.0s), widening the native/WASM gap from ~1.85× to ~2.21×.
These regressions are faithfully recorded here and the numbers are correct — this is just a heads-up that the underlying performance delta in 3.4.0 may be worth investigating before the next release.
There was a problem hiding this comment.
Acknowledged — the 1-file rebuild regression (+29% native, +21% WASM) is disproportionate to the ~7% file count growth and worth investigating. Since this is an auto-generated benchmark recording and the numbers are correct, the fix belongs in the codebase (likely in the incremental rebuild path), not in this PR. Will track separately.
There was a problem hiding this comment.
Tracked in #635 for follow-up investigation. The regression is real and worth looking into, but as you noted, the benchmark data here is correct — the fix belongs in the incremental rebuild path, not in this docs PR.
Automated incremental benchmark update for 3.4.0 from workflow run #462.