Skip to content

Add --base <run_id> to compare local run with another run#273

Merged
GuillaumeLagrange merged 3 commits intomainfrom
cod-1796-make-comparison-between-two-local-runs-easier
Mar 24, 2026
Merged

Add --base <run_id> to compare local run with another run#273
GuillaumeLagrange merged 3 commits intomainfrom
cod-1796-make-comparison-between-two-local-runs-easier

Conversation

@GuillaumeLagrange
Copy link
Copy Markdown
Contributor

@GuillaumeLagrange GuillaumeLagrange commented Mar 20, 2026

$ codspeed exec -m walltime --max-rounds 5 -- sh -c "sleep 0.5 && ls -al > /dev/null"

 Running the benchmarks

Place your finger on the fingerprint reader
  Sudo privileges are required to continue.
     Walltime - Running `sh -c sleep 0.5 && ls -al > /dev/null` with exec-harness
    ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
    │ Executing: sh -c sleep 0.5 && ls -al > /dev/null                                                                │
    │ Completed 2 warmup rounds                                                                                       │
    │ Warmup done, now performing 5 rounds                                                                            │
    ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

 Uploading results

  Performance data uploaded
  Linked repository: GuillaumeLagrange/local-runs
    Uploading results 4.3s

 Benchmark results

  sh -c sleep 0.5 && ls -al > /dev/null: best 508.05 ms (5 iterations, rel. stddev: 0.14%, total 2.54 s)

  View full report: http://localhost:3000/GuillaumeLagrange/local-runs/runs/69c28bfa6e3ac6b1b416512f

  To compare future runs against this one, use: --base 69c28bfa6e3ac6b1b416512f
$ codspeed exec -m walltime --max-rounds 5 --base 69c28bfa6e3ac6b1b416512f -- sh -c "sleep 0.5 && ls -al > /dev/null"

 Running the benchmarks

     Walltime - Running `sh -c sleep 0.5 && ls -al > /dev/null` with exec-harness
    ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
    │ Executing: sh -c sleep 0.5 && ls -al > /dev/null                                                                │
    │ Completed 2 warmup rounds                                                                                       │
    │ Warmup done, now performing 5 rounds                                                                            │
    ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

 Uploading results

  Performance data uploaded
  Linked repository: GuillaumeLagrange/local-runs
    Uploading results 1.4s

 Benchmark results

  ● Impact: 0.1%
  ╭────────────────────────────────────────────────────────────────────────────────────╮
  │                                      Walltime                                     │
  ├───────────────────────────────────────┬───────────┬───────────┬────────┬───────────┤
  │ Benchmark                             │      Base │      Head │ Change │    Status │
  ├───────────────────────────────────────┼───────────┼───────────┼────────┼───────────┤
  │ sh -c sleep 0.5 && ls -al > /dev/null │ 508.05 ms │ 507.53 ms │   0.1% │ No Change │
  ╰───────────────────────────────────────┴───────────┴───────────┴────────┴───────────╯

  View comparison report: http://localhost:3000/GuillaumeLagrange/local-runs/runs/compare/69c28bfa6e3ac6b1b416512f..69c28c5e6e3ac6b1b4165130

  To compare future runs against this one, use: --base 69c28c5e6e3ac6b1b4165130

@GuillaumeLagrange GuillaumeLagrange changed the title Cod 1796 make comparison between two local runs easier Add --base <run_id> to compare local run with another run Mar 20, 2026
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq bot commented Mar 20, 2026

Merging this PR will not alter performance

✅ 4 untouched benchmarks


Comparing cod-1796-make-comparison-between-two-local-runs-easier (aaad11b) with main (f9cdf47)

Open in CodSpeed

@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-1796-make-comparison-between-two-local-runs-easier branch 4 times, most recently from 0fffde9 to 2d85f8f Compare March 20, 2026 15:57
@GuillaumeLagrange GuillaumeLagrange marked this pull request as ready for review March 20, 2026 15:58
@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-1796-make-comparison-between-two-local-runs-easier branch from 2d85f8f to 6361678 Compare March 23, 2026 14:45
Add --base cli argument and simplify polling code
1. If a base run id is provided, we now use the `compareRunsPaginated`
   resolver and display results from there
2. If a base run id is not provided or if the base run id is not correct
   (missing, executor mismatch...), we display the single mode results

This overall simplifies the polling code and gets rid of the artificial
"for_run" and "for_exec" distinctions.
@GuillaumeLagrange GuillaumeLagrange force-pushed the cod-1796-make-comparison-between-two-local-runs-easier branch from 6361678 to aaad11b Compare March 24, 2026 13:04
@GuillaumeLagrange GuillaumeLagrange merged commit aaad11b into main Mar 24, 2026
18 checks passed
@GuillaumeLagrange GuillaumeLagrange deleted the cod-1796-make-comparison-between-two-local-runs-easier branch March 24, 2026 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants