Skip to content

ci: fix benchmark comments#28

Merged
mccutchen merged 2 commits intomainfrom
bench-comment-fix
Jan 22, 2025
Merged

ci: fix benchmark comments#28
mccutchen merged 2 commits intomainfrom
bench-comment-fix

Conversation

@mccutchen
Copy link
Owner

As discovered in #26 (comment), AFAICT changes to the benchmark workflow can't be tested till they're merged into master, due to restrictions on the workflow_dispatch event.

Here we make a couple of fixes and hope they work!

@github-actions
Copy link

github-actions bot commented Jan 22, 2025

🔥 Run benchmarks comparing ccf1295 against main:

gh workflow run bench.yaml -f pr_number=28

Note: this comment will update with each new commit.

@codecov
Copy link

codecov bot commented Jan 22, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.62%. Comparing base (9f05ba1) to head (ccf1295).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #28   +/-   ##
=======================================
  Coverage   79.62%   79.62%           
=======================================
  Files           3        3           
  Lines         432      432           
=======================================
  Hits          344      344           
  Misses         64       64           
  Partials       24       24           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions
Copy link

benchstats: 9f05ba1...ccf1295

View full benchmark output on the workflow summary.

goos: linux
goarch: amd64
pkg: github.com/mccutchen/websocket
cpu: AMD EPYC 7763 64-Core Processor                
                       │ ./baseline/bench-results.txt │      ./head/bench-results.txt       │
                       │            sec/op            │   sec/op     vs base                │
ReadFrame/1KiB-4                          1.026µ ± 3%   1.032µ ± 1%        ~ (p=0.698 n=10)
ReadFrame/1MiB-4                          915.3µ ± 1%   913.9µ ± 0%        ~ (p=0.684 n=10)
ReadFrame/8MiB-4                          7.060m ± 1%   7.052m ± 0%        ~ (p=0.353 n=10)
ReadFrame/16MiB-4                         14.18m ± 0%   14.26m ± 0%   +0.51% (p=0.007 n=10)
ReadMessage/1MiB/1-4                      1.075m ± 1%   1.083m ± 1%   +0.74% (p=0.011 n=10)
ReadMessage/8MiB/1-4                      7.869m ± 1%   7.807m ± 1%   -0.79% (p=0.015 n=10)
ReadMessage/16MiB/1-4                     15.66m ± 1%   15.66m ± 1%        ~ (p=0.912 n=10)
ReadMessage/1MiB/4-4                      1.555m ± 2%   1.450m ± 1%   -6.74% (p=0.000 n=10)
ReadMessage/8MiB/4-4                      10.66m ± 2%   10.95m ± 4%        ~ (p=0.123 n=10)
ReadMessage/16MiB/4-4                     19.74m ± 1%   19.77m ± 3%        ~ (p=0.739 n=10)
ReadMessage/1MiB/16-4                     2.085m ± 4%   1.986m ± 1%   -4.76% (p=0.001 n=10)
ReadMessage/8MiB/16-4                     17.17m ± 3%   15.62m ± 3%   -9.03% (p=0.000 n=10)
ReadMessage/16MiB/16-4                    22.66m ± 3%   26.43m ± 6%  +16.65% (p=0.000 n=10)
geomean                                   3.173m        3.167m        -0.18%

                       │ ./baseline/bench-results.txt │       ./head/bench-results.txt        │
                       │             B/op             │     B/op      vs base                 │
ReadFrame/1KiB-4                         1.055Ki ± 0%   1.055Ki ± 0%       ~ (p=1.000 n=10) ¹
ReadFrame/1MiB-4                         1.000Mi ± 0%   1.000Mi ± 0%       ~ (p=0.934 n=10)
ReadFrame/8MiB-4                         8.000Mi ± 0%   8.000Mi ± 0%       ~ (p=0.056 n=10)
ReadFrame/16MiB-4                        16.00Mi ± 0%   16.00Mi ± 0%  +0.00% (p=0.009 n=10)
ReadMessage/1MiB/1-4                     1.000Mi ± 0%   1.000Mi ± 0%       ~ (p=1.000 n=10)
ReadMessage/8MiB/1-4                     8.000Mi ± 0%   8.000Mi ± 0%       ~ (p=0.356 n=10)
ReadMessage/16MiB/1-4                    16.00Mi ± 0%   16.00Mi ± 0%       ~ (p=0.565 n=10)
ReadMessage/1MiB/4-4                     3.602Mi ± 0%   3.602Mi ± 0%  +0.00% (p=0.000 n=10)
ReadMessage/8MiB/4-4                     28.57Mi ± 0%   28.57Mi ± 0%       ~ (p=0.158 n=10)
ReadMessage/16MiB/4-4                    57.09Mi ± 0%   57.09Mi ± 0%       ~ (p=0.836 n=10)
ReadMessage/1MiB/16-4                    5.407Mi ± 0%   5.407Mi ± 0%       ~ (p=0.730 n=10)
ReadMessage/8MiB/16-4                    47.22Mi ± 0%   47.22Mi ± 0%       ~ (p=0.986 n=10)
ReadMessage/16MiB/16-4                   93.81Mi ± 0%   93.81Mi ± 0%       ~ (p=0.137 n=10)
geomean                                  5.263Mi        5.263Mi       +0.00%
¹ all samples are equal

                       │ ./baseline/bench-results.txt │      ./head/bench-results.txt       │
                       │          allocs/op           │ allocs/op   vs base                 │
ReadFrame/1KiB-4                           5.000 ± 0%   5.000 ± 0%       ~ (p=1.000 n=10) ¹
ReadFrame/1MiB-4                           5.000 ± 0%   5.000 ± 0%       ~ (p=1.000 n=10) ¹
ReadFrame/8MiB-4                           5.000 ± 0%   5.000 ± 0%       ~ (p=1.000 n=10) ¹
ReadFrame/16MiB-4                          5.000 ± 0%   5.000 ± 0%       ~ (p=1.000 n=10) ¹
ReadMessage/1MiB/1-4                       6.000 ± 0%   6.000 ± 0%       ~ (p=1.000 n=10) ¹
ReadMessage/8MiB/1-4                       6.000 ± 0%   6.000 ± 0%       ~ (p=1.000 n=10) ¹
ReadMessage/16MiB/1-4                      6.000 ± 0%   6.000 ± 0%       ~ (p=1.000 n=10) ¹
ReadMessage/1MiB/4-4                       24.00 ± 0%   24.00 ± 0%       ~ (p=1.000 n=10) ¹
ReadMessage/8MiB/4-4                       24.00 ± 0%   24.00 ± 0%       ~ (p=1.000 n=10) ¹
ReadMessage/16MiB/4-4                      24.00 ± 0%   24.00 ± 0%       ~ (p=1.000 n=10) ¹
ReadMessage/1MiB/16-4                      90.00 ± 0%   90.00 ± 0%       ~ (p=1.000 n=10) ¹
ReadMessage/8MiB/16-4                      90.00 ± 0%   90.00 ± 0%       ~ (p=1.000 n=10) ¹
ReadMessage/16MiB/16-4                     90.00 ± 0%   90.00 ± 0%       ~ (p=1.000 n=10) ¹
geomean                                    14.59        14.59       +0.00%
¹ all samples are equal

@mccutchen mccutchen merged commit 784b11d into main Jan 22, 2025
8 checks passed
@mccutchen mccutchen deleted the bench-comment-fix branch January 22, 2025 06:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant