Skip to content

docs: update incremental benchmarks (3.4.0)#627

Merged
carlos-alm merged 4 commits intomainfrom
benchmark/incremental-v3.4.0-20260326-073741
Mar 26, 2026
Merged

docs: update incremental benchmarks (3.4.0)#627
carlos-alm merged 4 commits intomainfrom
benchmark/incremental-v3.4.0-20260326-073741

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

Automated incremental benchmark update for 3.4.0 from workflow run #462.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 26, 2026

Greptile Summary

This is an automated benchmark update that appends 3.4.0 measurement data to generated/benchmarks/INCREMENTAL-BENCHMARKS.md, covering the summary table, the "Latest results" section, and the embedded JSON data block.\n\nI verified internal consistency across all three sections:\n\n- Native full build: fullBuildMs: 2276 → displayed as 2.3s ✓; ↓4% vs 3.3.1's 2369 ms ✓\n- Native 1-file rebuild: 448ms; ↑29% vs 3.3.1's 346 ms ✓\n- WASM full build: fullBuildMs: 50335.0s; ↑15% vs 3.3.1's 4363 ms ✓\n- WASM 1-file rebuild: 617ms; ↑21% vs 3.3.1's 508 ms ✓\n- Speedup ratio: jsFallbackMs: 11.8 / nativeBatchMs: 3.8 ≈ 3.1× ✓\n\nAll percentage deltas, rounded display values, and JSON raw values are consistent. The regression in 1-file rebuild latency and WASM full build that was flagged in a previous review thread has already been acknowledged and tracked in #635 — no further action is required here.

Confidence Score: 5/5

Safe to merge — purely a docs update with correct, internally consistent benchmark data.

Single auto-generated docs file; no code changes. All numeric values cross-check correctly between the summary table, Latest results section, and embedded JSON block. The only known concern (1-file rebuild regression) was already acknowledged in the prior thread and is being tracked separately in #635.

No files require special attention.

Important Files Changed

Filename Overview
generated/benchmarks/INCREMENTAL-BENCHMARKS.md Automated addition of 3.4.0 benchmark entries to the summary table, "Latest results" section, and embedded JSON data block; all numbers are internally consistent.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Release workflow publish.yml] -->|workflow_dispatch| B[Run benchmarks on own codebase]
    B --> C{Engine}
    C -->|Native Rust| D[Full build / No-op / 1-file timings]
    C -->|WASM| E[Full build / No-op / 1-file timings]
    D --> F[Compute import resolution stats]
    E --> F
    F --> G[Calculate % deltas vs previous version]
    G --> H[Update summary table row]
    G --> I[Update Latest results section]
    G --> J[Append JSON entry to embedded data block]
    H --> K[INCREMENTAL-BENCHMARKS.md]
    I --> K
    J --> K
    K --> L[PR #627 opened by github-actions bot]
Loading

Reviews (2): Last reviewed commit: "Merge branch 'main' into benchmark/incre..." | Re-trigger Greptile

Comment on lines +9 to +10
| 3.4.0 | native | 473 | 2.3s ↓4% | 12ms ↑9% | 448ms ↑29% | 4ms ↓58% | 12ms ↓54% |
| 3.4.0 | wasm | 473 | 5.0s ↑15% | 12ms ↑20% | 617ms ↑21% | 4ms ↓58% | 12ms ↓54% |
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Disproportionate 1-file rebuild regression

The 1-file rebuild time increased +29% for native (346ms → 448ms) and +21% for WASM (508ms → 617ms), but the file count only grew by ~7% (442 → 473 files). For a single-file incremental rebuild the total file count should have minimal impact on latency, so this regression is disproportionate to the codebase growth.

The WASM full build also regressed +15% (4.4s → 5.0s), widening the native/WASM gap from ~1.85× to ~2.21×.

These regressions are faithfully recorded here and the numbers are correct — this is just a heads-up that the underlying performance delta in 3.4.0 may be worth investigating before the next release.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acknowledged — the 1-file rebuild regression (+29% native, +21% WASM) is disproportionate to the ~7% file count growth and worth investigating. Since this is an auto-generated benchmark recording and the numbers are correct, the fix belongs in the codebase (likely in the incremental rebuild path), not in this PR. Will track separately.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracked in #635 for follow-up investigation. The regression is real and worth looking into, but as you noted, the benchmark data here is correct — the fix belongs in the incremental rebuild path, not in this docs PR.

@carlos-alm
Copy link
Copy Markdown
Contributor

@greptileai

@carlos-alm carlos-alm merged commit 1fae7e1 into main Mar 26, 2026
12 checks passed
@carlos-alm carlos-alm deleted the benchmark/incremental-v3.4.0-20260326-073741 branch March 26, 2026 22:41
@github-actions github-actions bot locked and limited conversation to collaborators Mar 26, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant