Skip to content

Revert "[roda] Return database results with symbols instead of strings - #996

Merged
MDA2AV merged 2 commits into
MDA2AV:mainfrom
p8:roda/revert-c503a98d07166a
Jul 17, 2026
Merged

Revert "[roda] Return database results with symbols instead of strings#996
MDA2AV merged 2 commits into
MDA2AV:mainfrom
p8:roda/revert-c503a98d07166a

Conversation

@p8

@p8 p8 commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Commit 76acc5c, which allowed Symbol#to_s to return a frozen string, showed a big performance regression on the async-db test.

Maybe this is related to the change to return symbols instead of strings.

This reverts commit c503a98.

@p8

p8 commented Jul 13, 2026

Copy link
Copy Markdown
Contributor Author

/benchmark -f roda

@github-actions

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: roda | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 102,236 4758.2% 9.1GiB -0.1% ~0%
baseline 4096 59,408 4138.4% 8.6GiB +4.5% ~0%
pipelined 512 737,646 6648.7% 1.7GiB +1.1% ~0%
pipelined 4096 721,371 6643.1% 2.1GiB +2.8% +5.0%
limited-conn 512 28,717 3749.2% 5.5GiB +0.9% ~0%
limited-conn 4096 28,530 3513.0% 6.4GiB -1.0% +1.6%
json 4096 250,084 6340.0% 4.3GiB +1.9% +2.4%
json-comp 512 129,788 6422.7% 3.5GiB -1.1% ~0%
json-comp 4096 141,464 6434.8% 5.0GiB +1.4% +6.4%
json-comp 16384 140,488 6248.8% 5.6GiB -0.9% +1.8%
json-tls 4096 221,383 6496.1% 6.2GiB -0.2% -1.6%
upload 32 954 3140.1% 8.8GiB +0.4% -1.1%
upload 256 902 6507.9% 7.4GiB -0.2% +1.4%
api-4 256 13,758 400.8% 476MiB -2.7% -0.4%
api-16 1024 48,227 1690.9% 1.7GiB -2.9% +6.2%
static 1024 144,014 6543.2% 5.6GiB +3.7% +3.7%
static 4096 142,417 6542.1% 5.9GiB +3.2% -4.8%
static 6800 136,177 6520.6% 5.7GiB -1.4% -3.4%
async-db 1024 100,537 6095.8% 2.5GiB +118.3% +13.6%
crud 4096 59,611 2775.4% 3.4GiB -2.8% -2.9%
Full log
  Reconnects: 1304
  Per-template: 9398,9476,9339,9422,9441
  Per-template-ok: 5447,5550,5356,5536,5645

  WARNING: 19542/47076 responses (41.5%) had unexpected status (expected 2xx)
[info] CPU 706.2% | Mem 1.7GiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   9.45ms   4.32ms   9.49ms   115.90ms   151.40ms

  1012205 requests in 10.00s, 1012206 responses
  Throughput: 101.20K req/s
  Bandwidth:  404.30MB/s
  Status codes: 2xx=1000198, 3xx=0, 4xx=0, 5xx=12008
  Latency samples: 1012205 / 1012206 responses (100.0%)
  Reconnects: 40188
  Per-template: 201822,202629,203044,202856,201854
  Per-template-ok: 199582,200123,200551,200505,199436

  WARNING: 12008/1012206 responses (1.2%) had unexpected status (expected 2xx)
[info] CPU 5925.1% | Mem 2.5GiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   9.41ms   4.16ms   9.51ms   115.50ms   147.80ms

  1017393 requests in 10.00s, 1017395 responses
  Throughput: 101.72K req/s
  Bandwidth:  406.64MB/s
  Status codes: 2xx=1005371, 3xx=0, 4xx=0, 5xx=12024
  Latency samples: 1017392 / 1017395 responses (100.0%)
  Reconnects: 40429
  Per-template: 202971,203313,204027,203814,203267
  Per-template-ok: 200681,200761,201586,201410,200930

  WARNING: 12024/1017395 responses (1.2%) had unexpected status (expected 2xx)
[info] CPU 6095.8% | Mem 2.5GiB

=== Best: 100537 req/s (CPU: 6095.8%, Mem: 2.5GiB) ===
[info] input BW: 6.71MB/s (avg template: 70 bytes)
[info] saved results/async-db/1024/roda.json
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / crud / 4096c (tool=gcannon) ===
==============================================
[info] resetting postgres for a clean per-profile baseline
[info] starting postgres sidecar
httparena-postgres
[info] postgres ready (seeded)
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  200
  Templates: 20
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   68.77ms   69.60ms   78.20ms   92.30ms   280.70ms

  857004 requests in 15.00s, 854453 responses
  Throughput: 56.95K req/s
  Bandwidth:  18.32MB/s
  Status codes: 2xx=854453, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 854453 / 854453 responses (100.0%)
  Reconnects: 2675
  Per-template: 42547,43003,42107,42351,42591,42756,42556,42797,42685,42249,42411,42527,42504,43141,42581,42541,42669,44750,42998,42689
  Per-template-ok: 42547,43003,42107,42351,42591,42756,42556,42797,42685,42249,42411,42527,42504,43141,42581,42541,42669,44750,42998,42689
[info] CPU 2762.3% | Mem 3.1GiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  200
  Templates: 20
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   66.33ms   68.20ms   75.90ms   87.60ms   261.10ms

  894305 requests in 15.00s, 892110 responses
  Throughput: 59.46K req/s
  Bandwidth:  19.00MB/s
  Status codes: 2xx=892110, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 892109 / 892110 responses (100.0%)
  Reconnects: 3186
  Per-template: 44721,44860,44339,44425,44292,44470,44464,44534,44418,44666,44624,44362,44406,44452,44372,44387,43698,47157,44440,45022
  Per-template-ok: 44721,44860,44339,44425,44292,44470,44464,44534,44418,44666,44624,44362,44406,44452,44372,44387,43698,47157,44440,45022
[info] CPU 2705.6% | Mem 3.4GiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  200
  Templates: 20
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   66.85ms   67.70ms   75.20ms   92.40ms   113.60ms

  898039 requests in 15.00s, 894169 responses
  Throughput: 59.60K req/s
  Bandwidth:  19.06MB/s
  Status codes: 2xx=894169, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 894169 / 894169 responses (100.0%)
  Reconnects: 3609
  Per-template: 44931,44786,44400,44439,44614,44679,44403,44348,44375,44817,44524,44725,44529,44831,44650,44454,44035,46997,44905,44727
  Per-template-ok: 44931,44786,44400,44439,44614,44679,44403,44348,44375,44817,44524,44725,44529,44831,44650,44454,44035,46997,44905,44727
[info] CPU 2775.4% | Mem 3.4GiB

=== Best: 59611 req/s (CPU: 2775.4%, Mem: 3.4GiB) ===
[info] input BW: 5.12MB/s (avg template: 90 bytes)
[info] saved results/crud/4096/roda.json
httparena-bench-roda
httparena-bench-roda
[info] skip: roda does not subscribe to fortunes
[info] skip: roda does not subscribe to baseline-h2
[info] skip: roda does not subscribe to static-h2
[info] skip: roda does not subscribe to baseline-h2c
[info] skip: roda does not subscribe to json-h2c
[info] skip: roda does not subscribe to baseline-h3
[info] skip: roda does not subscribe to static-h3
[info] skip: roda does not subscribe to gateway-64
[info] skip: roda does not subscribe to gateway-h3
[info] skip: roda does not subscribe to production-stack
[info] skip: roda does not subscribe to unary-grpc
[info] skip: roda does not subscribe to unary-grpc-tls
[info] skip: roda does not subscribe to stream-grpc
[info] skip: roda does not subscribe to stream-grpc-tls
[info] skip: roda does not subscribe to echo-ws
[info] skip: roda does not subscribe to echo-ws-pipeline
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/crud-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536

MDA2AV#782)"

Commit 76acc5c, which allowed
Symbol#to_s to return a frozen string, showed a big performance
regression on the async-db test.

Maybe this is related to the change to return symbols instead of strings.

This reverts commit c503a98.
@p8
p8 force-pushed the roda/revert-c503a98d07166a branch from fba4756 to 98c935c Compare July 14, 2026 07:43
@p8

p8 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

/benchmark -f roda --save

@github-actions

Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: roda | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 100,648 4772.5% 9.0GiB -1.7% -1.1%
baseline 4096 49,154 4002.9% 8.2GiB -13.5% -4.7%
pipelined 512 718,707 6642.8% 1.7GiB -1.5% ~0%
pipelined 4096 697,544 6668.9% 2.0GiB -0.6% ~0%
limited-conn 512 28,599 3725.9% 5.7GiB +0.5% +3.6%
limited-conn 4096 28,814 3432.2% 6.4GiB ~0% +1.6%
json 4096 249,577 6396.1% 4.3GiB +1.7% +2.4%
json-comp 512 130,040 6456.3% 3.5GiB -0.9% ~0%
json-comp 4096 141,482 6418.5% 4.9GiB +1.5% +4.3%
json-comp 16384 141,281 6509.0% 5.3GiB -0.4% -3.6%
json-tls 4096 220,720 6587.9% 5.8GiB -0.5% -7.9%
upload 32 949 3010.2% 8.9GiB -0.1% ~0%
upload 256 900 6366.2% 8.1GiB -0.4% +11.0%
api-4 256 13,645 400.6% 475MiB -3.5% -0.6%
api-16 1024 48,671 1697.8% 1.7GiB -2.0% +6.2%
static 1024 141,669 6539.8% 5.6GiB +2.1% +3.7%
static 4096 139,115 6523.0% 5.8GiB +0.8% -6.5%
static 6800 138,957 6547.6% 5.9GiB +0.6% ~0%
async-db 1024 84,584 5696.2% 2.4GiB +83.7% +9.1%
crud 4096 61,151 2588.3% 3.4GiB -0.3% -2.9%
Full log
  Reconnects: 1923
  Per-template: 12012,12007,12477,12418,12223
  Per-template-ok: 8254,8074,8555,8507,8174

  WARNING: 19573/61137 responses (32.0%) had unexpected status (expected 2xx)
[info] CPU 840.9% | Mem 1.7GiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   11.46ms   2.29ms   8.51ms   301.40ms   620.00ms

  827084 requests in 10.00s, 827084 responses
  Throughput: 82.69K req/s
  Bandwidth:  328.81MB/s
  Status codes: 2xx=812819, 3xx=0, 4xx=0, 5xx=14265
  Latency samples: 827081 / 827084 responses (100.0%)
  Reconnects: 32789
  Per-template: 165139,165340,165744,165578,165280
  Per-template-ok: 162163,162352,163003,162801,162497

  WARNING: 14265/827084 responses (1.7%) had unexpected status (expected 2xx)
[info] CPU 5450.3% | Mem 2.4GiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   11.18ms   2.56ms   8.36ms   273.60ms   452.50ms

  859768 requests in 10.00s, 859769 responses
  Throughput: 85.96K req/s
  Bandwidth:  342.10MB/s
  Status codes: 2xx=845842, 3xx=0, 4xx=0, 5xx=13927
  Latency samples: 859765 / 859769 responses (100.0%)
  Reconnects: 34130
  Per-template: 171632,171898,172333,172357,171545
  Per-template-ok: 168691,169067,169707,169351,169022

  WARNING: 13927/859769 responses (1.6%) had unexpected status (expected 2xx)
[info] CPU 5696.2% | Mem 2.4GiB

=== Best: 84584 req/s (CPU: 5696.2%, Mem: 2.4GiB) ===
[info] input BW: 5.65MB/s (avg template: 70 bytes)
[info] saved results/async-db/1024/roda.json
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / crud / 4096c (tool=gcannon) ===
==============================================
[info] resetting postgres for a clean per-profile baseline
[info] starting postgres sidecar
httparena-postgres
[info] postgres ready (seeded)
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  200
  Templates: 20
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   66.99ms   66.30ms   77.10ms   110.30ms   283.40ms

  882788 requests in 15.00s, 880178 responses
  Throughput: 58.67K req/s
  Bandwidth:  18.87MB/s
  Status codes: 2xx=880178, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 880178 / 880178 responses (100.0%)
  Reconnects: 3142
  Per-template: 44207,44530,43910,43889,43616,44028,43943,43770,43756,44035,43678,43538,43447,43821,44122,43738,44012,45984,44012,44142
  Per-template-ok: 44207,44530,43910,43889,43616,44028,43943,43770,43756,44035,43678,43538,43447,43821,44122,43738,44012,45984,44012,44142
[info] CPU 2889.8% | Mem 3.1GiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  200
  Templates: 20
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   63.78ms   65.80ms   72.80ms   83.40ms   239.10ms

  919815 requests in 15.00s, 917267 responses
  Throughput: 61.14K req/s
  Bandwidth:  19.55MB/s
  Status codes: 2xx=917267, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 917267 / 917267 responses (100.0%)
  Reconnects: 3609
  Per-template: 46052,46142,45613,45470,45606,45900,45857,45559,45529,45772,45585,45575,45544,45995,45746,45470,45184,48488,46186,45994
  Per-template-ok: 46052,46142,45613,45470,45606,45900,45857,45559,45529,45772,45585,45575,45544,45995,45746,45470,45184,48488,46186,45994
[info] CPU 2588.3% | Mem 3.4GiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  200
  Templates: 20
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   65.98ms   67.70ms   74.60ms   86.30ms   102.80ms

  901217 requests in 15.00s, 899081 responses
  Throughput: 59.93K req/s
  Bandwidth:  19.19MB/s
  Status codes: 2xx=899081, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 899080 / 899081 responses (100.0%)
  Reconnects: 3513
  Per-template: 45228,45238,44745,44629,44629,44930,44892,44829,44555,44670,44440,44795,44853,45043,44700,44529,44529,47233,45309,45304
  Per-template-ok: 45228,45238,44745,44629,44629,44930,44892,44829,44555,44670,44440,44795,44853,45043,44700,44529,44529,47233,45309,45304
[info] CPU 2588.8% | Mem 3.4GiB

=== Best: 61151 req/s (CPU: 2588.3%, Mem: 3.4GiB) ===
[info] input BW: 5.25MB/s (avg template: 90 bytes)
[info] saved results/crud/4096/roda.json
httparena-bench-roda
httparena-bench-roda
[info] skip: roda does not subscribe to fortunes
[info] skip: roda does not subscribe to baseline-h2
[info] skip: roda does not subscribe to static-h2
[info] skip: roda does not subscribe to baseline-h2c
[info] skip: roda does not subscribe to json-h2c
[info] skip: roda does not subscribe to baseline-h3
[info] skip: roda does not subscribe to static-h3
[info] skip: roda does not subscribe to gateway-64
[info] skip: roda does not subscribe to gateway-h3
[info] skip: roda does not subscribe to production-stack
[info] skip: roda does not subscribe to unary-grpc
[info] skip: roda does not subscribe to unary-grpc-tls
[info] skip: roda does not subscribe to stream-grpc
[info] skip: roda does not subscribe to stream-grpc-tls
[info] skip: roda does not subscribe to echo-ws
[info] skip: roda does not subscribe to echo-ws-pipeline
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/crud-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536

@MDA2AV
MDA2AV merged commit 9e80099 into MDA2AV:main Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants