fix(relay): clean up in-flight websocket requests - #4998
Conversation
Signed-off-by: Stuart Douglas <sdouglas@block.xyz> Amp-Thread-ID: https://ampcode.com/threads/T-019fd4c7-3d09-71af-b091-05920b3b0957 Co-authored-by: Amp <amp@ampcode.com>
wesbillman
left a comment
There was a problem hiding this comment.
Carl reviewing on behalf of Wes. The lifecycle barrier is directionally sound, but the NIP-50 path still permits output from a cancelled generation, so I don't think this head fully establishes the advertised CLOSE/replacement guarantee. Please also add a deterministic lifecycle regression that stages a blocked REQ, processes CLOSE/disconnect/replacement, resumes it, and asserts the local subscription map, fan-out registry, and Redis desired-topic refcount—not just token bookkeeping. Focused cargo test -p buzz-relay connection::tests:: passes at this exact head (10 tests), but those tests do not exercise the state race.
| trace_state.as_ref(), | ||
| ) | ||
| .await; | ||
| tokio::select! { |
There was a problem hiding this comment.
P1 — CLOSE/replacement can still be followed by stale search output. This outer select! only observes cancellation when handle_search_req yields. After its page hydration await, handle_search_req synchronously walks the hydrated page and calls conn.send for every accepted event, then sends EOSE (req.rs:719-765), with no cancellation check or yield. On a multi-thread runtime, the recv task can process CLOSE (or a replacement REQ) and enqueue CLOSED/new-generation output while this task continues enqueueing old-generation EVENTs and EOSE. Please pass the generation token into the search helper and check it before each emission and before EOSE (and make awaited phases cancellation-aware), with a deterministic regression proving no old-generation output is emitted after CLOSE/replacement wins.
There was a problem hiding this comment.
Fixed in cd62a82. NIP-50 search now observes cancellation around search/hydration awaits and before every EVENT/EOSE emission. Each emission also holds the active request-generation guard through the enqueue, so CLOSE removal or same-ID replacement is serialized with old-generation output; once either operation wins, the old generation cannot enqueue another frame. Added deterministic CLOSE/replacement output coverage plus blocked-registration lifecycle regressions for CLOSE, disconnect, and replacement. Those tests poll the real cleanup/replacement path to Pending behind the connection subscription mutex, then resume registration and assert the connection-local map, fan-out registry, and PubSub topic refcount. Validation: connection tests (14), REQ tests (47), clippy -D warnings, and pre-push hooks all pass.
Signed-off-by: Stuart Douglas <sdouglas@block.xyz> Amp-Thread-ID: https://ampcode.com/threads/T-019fd4c7-3d09-71af-b091-05920b3b0957 Co-authored-by: Amp <amp@ampcode.com>
Summary
REQhandlers by subscription ID and generationCLOSE, replace a subscription ID, or disconnectRoot cause
REQhandlers run in detached tasks because historical reads can take time.CLOSEand disconnect cleanup run independently.A history timeout or other early cleanup could therefore produce this ordering:
REQCLOSE(or disconnects)On a long-lived WebSocket, that state was effectively leaked. Disconnect had the same race because its registry cleanup was one-shot and did not synchronize with REQ registration.
The fix uses cooperative cancellation plus the existing per-connection subscription mutex as the registration/cleanup barrier. Once registration begins, the registry and pub/sub transition runs to completion; teardown then removes it. If teardown wins the mutex, the REQ observes cancellation and does not register.
Testing
cargo test -p buzz-relay connection::tests:: --libcargo clippy -p buzz-relay --lib -- -D warningsA full
cargo test -p buzz-relay --librun completed 852 tests successfully; nine unrelated database-backed admin/media tests failed because the local Postgres pool timed out.