Skip to content

perf: add benchmark for WriteFrame#51

Merged
mccutchen merged 4 commits intomainfrom
bench/writeframe
Apr 26, 2025
Merged

perf: add benchmark for WriteFrame#51
mccutchen merged 4 commits intomainfrom
bench/writeframe

Conversation

@mccutchen
Copy link
Owner

@mccutchen mccutchen commented Apr 24, 2025

Also, while we're here:

  • Drop benchmark for ReadMessage, for now, till we circle back to focus on the higher level API
  • Reduce the number of frame sizes we benchmark against
  • Shuffle benchmark code location
  • Quick rename of a var in ReadFrame

@github-actions
Copy link

github-actions bot commented Apr 24, 2025

🔥 Run benchmarks comparing 44ed96a against main:

gh workflow run bench.yaml -f pr_number=51

Note: this comment will update with each new commit.

@codecov
Copy link

codecov bot commented Apr 24, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.76%. Comparing base (eb4da17) to head (44ed96a).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #51   +/-   ##
=======================================
  Coverage   92.76%   92.76%           
=======================================
  Files           2        2           
  Lines         484      484           
=======================================
  Hits          449      449           
  Misses         28       28           
  Partials        7        7           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link

benchstats: eb4da17...0ae7d4b

View full benchmark output on the workflow summary.

goos: linux
goarch: amd64
pkg: github.com/mccutchen/websocket
cpu: AMD EPYC 7763 64-Core Processor                
                         │ ./baseline/bench-results.txt │       ./head/bench-results.txt       │
                         │            sec/op            │   sec/op     vs base                 │
ReadFrame/1KiB-4                           1043.0n ± 7%   962.3n ± 0%  -7.74% (p=0.000 n=10)
ReadFrame/1MiB-4                            701.2µ ± 2%   634.9µ ± 1%  -9.46% (p=0.000 n=10)
ReadFrame/8MiB-4                            5.331m ± 1%
ReadFrame/16MiB-4                           10.88m ± 1%
ReadMessage/1KiB/1-4                        1.504µ ± 1%
ReadMessage/8MiB/1-4                        6.229m ± 1%
ReadMessage/16MiB/1-4                       13.01m ± 2%
ReadMessage/1KiB/4-4                        2.636µ ± 1%
ReadMessage/8MiB/4-4                        11.64m ± 6%
ReadMessage/16MiB/4-4                       22.25m ± 7%
ReadMessage/1KiB/16-4                       5.096µ ± 1%
ReadMessage/8MiB/16-4                       18.93m ± 9%
ReadMessage/16MiB/16-4                      29.39m ± 4%
ReadFrame/256B-4                                          479.7n ± 1%
ReadFrame/4MiB-4                                          2.394m ± 0%
WriteFrame/Masked/256B-4                                  627.8n ± 1%
WriteFrame/Masked/1KiB-4                                  1.386µ ± 0%
WriteFrame/Masked/1MiB-4                                  1.017m ± 5%
WriteFrame/Masked/4MiB-4                                  3.886m ± 1%
geomean                                     701.7µ        35.30µ       -8.60%                ¹
¹ benchmark set differs from baseline; geomeans may not be comparable

                         │ ./baseline/bench-results.txt │        ./head/bench-results.txt         │
                         │             B/s              │      B/s       vs base                  │
ReadFrame/1KiB-4                          943.7Mi ±  6%   1022.8Mi ± 0%   +8.37% (p=0.000 n=10)
ReadFrame/1MiB-4                          1.393Gi ±  2%    1.538Gi ± 1%  +10.45% (p=0.000 n=10)
ReadFrame/8MiB-4                          1.465Gi ±  1%
ReadFrame/16MiB-4                         1.436Gi ±  1%
ReadMessage/1KiB/1-4                      654.5Mi ±  1%
ReadMessage/8MiB/1-4                      1.254Gi ±  1%
ReadMessage/16MiB/1-4                     1.201Gi ±  2%
ReadMessage/1KiB/4-4                      382.1Mi ±  1%
ReadMessage/8MiB/4-4                      687.1Mi ±  6%
ReadMessage/16MiB/4-4                     719.1Mi ±  7%
ReadMessage/1KiB/16-4                     209.6Mi ±  1%
ReadMessage/8MiB/16-4                     422.6Mi ± 10%
ReadMessage/16MiB/16-4                    544.5Mi ±  4%
ReadFrame/256B-4                                           524.8Mi ± 1%
ReadFrame/4MiB-4                                           1.632Gi ± 0%
WriteFrame/Masked/256B-4                                   401.0Mi ± 1%
WriteFrame/Masked/1KiB-4                                   710.2Mi ± 0%
WriteFrame/Masked/1MiB-4                                   982.9Mi ± 5%
WriteFrame/Masked/4MiB-4                                   1.005Gi ± 1%
geomean                                   759.5Mi          893.7Mi        +9.41%                ¹
¹ benchmark set differs from baseline; geomeans may not be comparable

                         │ ./baseline/bench-results.txt │       ./head/bench-results.txt        │
                         │             B/op             │     B/op      vs base                 │
ReadFrame/1KiB-4                           1.039Ki ± 0%   1.039Ki ± 0%       ~ (p=1.000 n=10) ¹
ReadFrame/1MiB-4                           1.000Mi ± 0%   1.000Mi ± 0%  +0.00% (p=0.038 n=10)
ReadFrame/8MiB-4                           8.000Mi ± 0%
ReadFrame/16MiB-4                          16.00Mi ± 0%
ReadMessage/1KiB/1-4                       1.070Ki ± 0%
ReadMessage/8MiB/1-4                       8.000Mi ± 0%
ReadMessage/16MiB/1-4                      16.00Mi ± 0%
ReadMessage/1KiB/4-4                       3.938Ki ± 0%
ReadMessage/8MiB/4-4                       28.57Mi ± 0%
ReadMessage/16MiB/4-4                      57.09Mi ± 0%
ReadMessage/1KiB/16-4                      4.781Ki ± 0%
ReadMessage/8MiB/16-4                      47.22Mi ± 0%
ReadMessage/16MiB/16-4                     93.81Mi ± 0%
ReadFrame/256B-4                                            296.0 ± 0%
ReadFrame/4MiB-4                                          4.000Mi ± 0%
WriteFrame/Masked/256B-4                                    288.0 ± 0%
WriteFrame/Masked/1KiB-4                                  1.125Ki ± 0%
WriteFrame/Masked/1MiB-4                                  1.008Mi ± 0%
WriteFrame/Masked/4MiB-4                                  4.008Mi ± 0%
geomean                                    1.065Mi        33.76Ki       +0.00%                ²
¹ all samples are equal
² benchmark set differs from baseline; geomeans may not be comparable

                         │ ./baseline/bench-results.txt │      ./head/bench-results.txt       │
                         │          allocs/op           │ allocs/op   vs base                 │
ReadFrame/1KiB-4                             5.000 ± 0%   5.000 ± 0%       ~ (p=1.000 n=10) ¹
ReadFrame/1MiB-4                             5.000 ± 0%   5.000 ± 0%       ~ (p=1.000 n=10) ¹
ReadFrame/8MiB-4                             5.000 ± 0%
ReadFrame/16MiB-4                            5.000 ± 0%
ReadMessage/1KiB/1-4                         6.000 ± 0%
ReadMessage/8MiB/1-4                         7.000 ± 0%
ReadMessage/16MiB/1-4                        7.000 ± 0%
ReadMessage/1KiB/4-4                         24.00 ± 0%
ReadMessage/8MiB/4-4                         26.00 ± 0%
ReadMessage/16MiB/4-4                        26.00 ± 0%
ReadMessage/1KiB/16-4                        70.00 ± 0%
ReadMessage/8MiB/16-4                        94.00 ± 1%
ReadMessage/16MiB/16-4                       94.00 ± 0%
ReadFrame/256B-4                                          5.000 ± 0%
ReadFrame/4MiB-4                                          5.000 ± 0%
WriteFrame/Masked/256B-4                                  1.000 ± 0%
WriteFrame/Masked/1KiB-4                                  1.000 ± 0%
WriteFrame/Masked/1MiB-4                                  1.000 ± 0%
WriteFrame/Masked/4MiB-4                                  1.000 ± 0%
geomean                                      14.94        2.236       +0.00%                ²
¹ all samples are equal
² benchmark set differs from baseline; geomeans may not be comparable

@mccutchen mccutchen merged commit 527a9ec into main Apr 26, 2025
12 checks passed
@mccutchen mccutchen deleted the bench/writeframe branch April 26, 2025 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant