Aggregate data rates at vhost level for all protocols

[kickster97]

WHAT is this pull request doing?

Track data rates at vhost level across all protocols

Previously, byte rates in the overview were aggregated from per-connection stats, which meant HTTP API traffic was invisible in the data rates graph since HTTP has no persistent connections. This adds recv_oct and send_oct counters to the vhost, fed by all three protocols: AMQP and MQTT clients bubble up their per-connection byte counts to the vhost alongside existing connection-level tracking, and HTTP reports bytes directly to the vhost. The overview endpoint now reads byte rates from the vhost instead of summing connections, giving a complete picture of all traffic.

Based off discussion in #1682

HOW can this pull request be tested?

specs to come

[carlhoerberg]

Did it have any performance impact?

[carlhoerberg]

HTTP revc/send doesn't include the whole request/response size, but i guess that's ok.

[jage]

Can we add specs? Ideally specs that break if we add additional protocols and miss this.

[claude]

PR Review: Vhost-level byte rate tracking

1. Test logic flaw in "all protocols should aggregate byte counts to vhost level" (spec/vhost_stats_spec.cr)

The send_and_receive_over_protocol method for AMQP explicitly waits for connections to be removed:

wait_for { vhost.connections.none?(LavinMQ::AMQP::Client) }

And the MQTT path's with_client_io blocks close connections when they exit. By the time the test reaches:

conn_recv_sum = vhost.connections.sum(&.recv_oct_count)
conn_send_sum = vhost.connections.sum(&.send_oct_count)

vhost.recv_oct_count.should be >= conn_recv_sum
vhost.send_oct_count.should be >= conn_send_sum

vhost.connections is empty (or nearly so), making both sums 0. The >= assertions pass trivially and do not actually verify that vhost-level byte counts aggregate per-connection counts. The wait_for condition reinforces this since it succeeds immediately when vhost.connections.empty? due to the ||:

wait_for { vhost.connections.empty? || vhost.connections.all? { |c| c.recv_oct_count > 0 } }

The final > 0 assertions are the only ones that verify anything meaningful. Consider either keeping connections alive during the comparison, or removing the misleading >= conn_sum assertions.

Everything else looks correct

• The atomic add calls with :relaxed ordering in add_recv_bytes/add_send_bytes are consistent with existing per-connection tracking and appropriate for the hot path (single atomic add instruction, no new allocations).
• The rate_stats macro correctly generates @recv_oct_count/@send_oct_count atomics and the corresponding rate computation in update_rates, which is already called periodically via Server#update_stats_rates.
• The overview endpoint change from summing per-connection rates to using vhost-level rates is correct since the vhost now tracks bytes directly.
• HTTP API publish/get byte tracking (body-only, no protocol framing) is a reasonable design choice for non-AMQP paths.
• All AMQP client send paths (single frame, deliver method+header+body frames) and the receive path (process_frame) are covered.