Skip to content

docs: update incremental benchmarks (3.6.0)#714

Open
github-actions[bot] wants to merge 2 commits intomainfrom
benchmark/incremental-v3.6.0-20260330-164447
Open

docs: update incremental benchmarks (3.6.0)#714
github-actions[bot] wants to merge 2 commits intomainfrom
benchmark/incremental-v3.6.0-20260330-164447

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

Automated incremental benchmark update for 3.6.0 from workflow run #538.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 30, 2026

Greptile Summary

This automated PR updates generated/benchmarks/INCREMENTAL-BENCHMARKS.md with 3.6.0 incremental benchmark data. The native engine entry is absent because the native worker crashed with SQLITE_CORRUPT (WAL race condition during detect-changes / purgeAndAddReverseDeps); a blockquote note explaining this was added in response to the previous review comment. Only WASM build metrics are recorded; import-resolution data for both engines is still present because that phase runs independently of the full build engine.

Key changes:

  • Adds a 3.6.0 | wasm row to the summary table (514 files, 6.1 s full build, 545 ms 1-file rebuild).
  • Updates the Latest results section from 3.5.0 → 3.6.0 and removes the now-stale <!-- NOTES_START/END --> block.
  • Adds a human-readable WASM build-tier breakdown and updates import-resolution figures (904 pairs, 3.0× speedup).
  • Prepends the 3.6.0 JSON entry to the embedded INCREMENTAL_BENCHMARK_DATA comment block.

All figures cross-check correctly: fullBuildMs 6128 → 6.1 s, nativeBatchMs 3.9 → 4 ms, jsFallbackMs 11.7 → 12 ms, speedup 11.7 / 3.9 = 3.0×. The absence of delta indicators (↑/↓) for the three build-tier columns in the summary table is expected because the previous WASM baseline (3.5.0) had null build data.

Confidence Score: 5/5

Safe to merge — documentation-only change with no code impact; all data is internally consistent and the previous reviewer concern has been addressed.

No P0 or P1 issues found. All numeric values in the summary table, Latest results section, and embedded JSON are mutually consistent. The blockquote note explaining the absent native row directly addresses the prior review thread. The PR is a routine automated docs update.

No files require special attention.

Important Files Changed

Filename Overview
generated/benchmarks/INCREMENTAL-BENCHMARKS.md Adds 3.6.0 wasm-only benchmark entry with blockquote explaining the missing native row (SQLITE_CORRUPT / WAL crash); all numeric values are internally consistent across table, Latest results section, and embedded JSON.

Sequence Diagram

sequenceDiagram
    participant WF as GitHub Actions Workflow #538
    participant BH as Benchmark Harness
    participant NE as Native Engine
    participant WE as WASM Engine
    participant RR as Resolve Runner
    participant MD as INCREMENTAL-BENCHMARKS.md

    WF->>BH: trigger benchmark run (v3.6.0)
    BH->>NE: run full / no-op / 1-file rebuild
    NE-->>BH: SQLITE_CORRUPT (WAL corruption)<br/>detect-changes → purgeAndAddReverseDeps
    BH->>WE: run full / no-op / 1-file rebuild
    WE-->>BH: fullBuildMs=6128, noopMs=13, 1fileMs=545
    BH->>RR: run import resolution (native + JS)
    RR-->>BH: nativeBatchMs=3.9, jsFallbackMs=11.7, imports=904
    BH->>MD: prepend 3.6.0 JSON entry (native: null)
    BH->>MD: update summary table row (wasm only)
    BH->>MD: update Latest results section + add WAL note
    WF->>WF: open PR #714
Loading

Reviews (2): Last reviewed commit: "docs(benchmarks): explain missing native..." | Re-trigger Greptile


| Version | Engine | Files | Full Build | No-op | 1-File | Resolve (native) | Resolve (JS) |
|---------|--------|------:|-----------:|------:|-------:|------------------:|-------------:|
| 3.6.0 | wasm | 514 | 6.1s | 13ms | 545ms | 4ms ↓3% | 12ms ↑9% |
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Missing native engine row with no explanation

3.6.0 has "native": null in the embedded JSON data, so only a wasm row appears in the summary table. For 3.5.0, the workflow generated a <!-- NOTES_START --> ... <!-- NOTES_END --> block explaining why both engines were null (napi-rs serialization bug, WAL race condition). That note has now been removed since 3.6.0 is the latest.

However, native was working in 3.4.1, and readers of this file have no indication of why the native engine is absent for 3.6.0. It may be worth adding a brief note in the "Latest results" section (or a comment in the JSON entry) explaining whether the native engine was intentionally skipped, failed during the benchmark run, or is being phased out.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed. Added a blockquote note in the "Latest results" section explaining that the native engine crashed with SQLITE_CORRUPT during the incremental rebuild tier (WAL corruption in detect-changes / purgeAndAddReverseDeps). See 839aa86.

Native worker crashed with SQLITE_CORRUPT during incremental rebuild
(WAL corruption in detect-changes). Add a note so readers understand
why only the WASM row appears.
@carlos-alm
Copy link
Copy Markdown
Contributor

@greptileai review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant