[ISSUE #10609] Detach telemetry observer when gRPC stream closes#10610
Open
123123213weqw wants to merge 1 commit into
Open
[ISSUE #10609] Detach telemetry observer when gRPC stream closes#10610123123213weqw wants to merge 1 commit into
123123213weqw wants to merge 1 commit into
Conversation
When ClientActivity#telemetry receives the client SETTINGS command it stores the gRPC response observer in GrpcClientChannel via setClientObserver. isActive(), isOpen() and isWritable() all derive from that observer reference. Neither the onCompleted() nor onError() callback of the telemetry stream cleared the observer, so a closed stream stayed referenced by the channel and kept being reported as active/writable until a later writeTelemetryCommand failed. Capture the GrpcClientChannel bound to the stream while processing SETTINGS and detach its observer (clearClientObserver) in both onCompleted() and onError(). clearClientObserver relies on compareAndSet, so a delayed callback from an old stream cannot detach an observer that belongs to a newer, reconnected stream. processAndWriteClientSettings now returns the GrpcClientChannel it registered so the stream observer can clear the right channel. GrpcClientChannel#clearClientObserver is widened from protected to public so the cross-package ClientActivity can invoke it. Producer/Consumer unregistration is intentionally left to the existing termination and heartbeat timeout mechanisms. Add unit tests covering observer detach on completion/error and the reconnect safety where an old stream completion does not clear a newer observer. Signed-off-by: 王越 <1939455790@qq.com>
RockteMQ-AI
reviewed
Jul 12, 2026
RockteMQ-AI
left a comment
Contributor
There was a problem hiding this comment.
Review by github-manager-bot
Summary
Fixes #10609 — detaches the telemetry response observer from GrpcClientChannel when the gRPC stream closes (onCompleted / onError), so that channel liveness state (isActive / isOpen / isWritable) reflects stream closure immediately instead of lingering until a later write failure.
Findings
- [Info]
ClientActivity.java— The newgrpcClientChannelfield on the anonymousContextStreamObserveris notvolatile. gRPC serialises callbacks per stream, so this is safe in practice, but addingvolatilewould make the intent explicit and guard against future refactors that might introduce cross-thread access. - [Info]
ClientActivity.java:processAndWriteClientSettings— Return type changed fromvoidtoGrpcClientChannel. This is a breaking change for any subclass that overrides this method (compile error). If there are known external subclasses, consider deprecating the old signature first or adding an overload. - [Info]
GrpcClientChannel.java:clearClientObserver— Visibility widened fromprotectedtopublic. Backward-compatible, but consider whether package-private + a dedicated accessor on a shared interface would be a tighter encapsulation.
Suggestions
- Consider
volatileongrpcClientChannel— Low cost, documents the cross-callback visibility contract:private volatile GrpcClientChannel grpcClientChannel = null;
- The
compareAndSetinclearClientObservercorrectly prevents a stale stream's delayed callback from clearing a newer observer — good use of CAS for reconnection safety.
Tests
Test coverage is thorough:
- ✅ Basic set/clear cycle (
testSetAndClearClientObserver) - ✅ CAS prevents old observer from clearing newer one (
testClearClientObserverDoesNotClearNewerObserver) - ✅
onCompletedclears channel state (testTelemetryOnCompletedClearsClientObserver) - ✅
onErrorclears channel state (testTelemetryOnErrorClearsClientObserver) - ✅ Reconnect safety — old stream completion doesn't detach newer observer (
testOldStreamCompletionDoesNotClearNewerObserver)
Cross-repo Note
This change is Proxy-side only and does not affect the gRPC wire protocol or client SDKs. No coordinated change needed in apache/rocketmq-clients.
Automated review by github-manager-bot
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
Fix #10609.
Proxy stored the response observer of a gRPC telemetry stream in
GrpcClientChannelwhen the clientSETTINGScommand arrived, but the stream'sonCompleted()andonError()callbacks never detached it. As a result aclosed telemetry stream remained referenced by the channel, and
isActive()/isOpen()/isWritable()kept returningtrueuntil a laterwriteTelemetryCommandfailed (or another cleanup path removed the channel).This makes the channel liveness state reflect stream closure immediately, while
ensuring a delayed callback from an old stream cannot affect a reconnected
client's newer observer.
Brief changelog
proxy/.../grpc/v2/client/ClientActivity.javatelemetrystream observer now captures theGrpcClientChannelboundto this stream while processing
SETTINGS.onCompleted()andonError()detach the stream observer from thatchannel via
clearClientObserver(responseObserver)(null-safe).processAndWriteClientSettingsnow returns the registeredGrpcClientChannelso the stream observer can clear the correct channel.proxy/.../grpc/v2/channel/GrpcClientChannel.javaclearClientObserveris widened fromprotectedtopublicso thecross-package
ClientActivitycan invoke it. It still usescompareAndSet(future, null), so only the observer that actually belongsto the finishing stream is removed.
Producer/Consumer unregistration is intentionally untouched; it continues to use
the existing termination and heartbeat timeout mechanisms, as noted in the issue.
How was this patch verified
Added/extended unit tests:
ClientActivityTesttestTelemetryOnCompletedClearsClientObserver: channel becomes inactiveafter
onCompleted().testTelemetryOnErrorClearsClientObserver: channel becomes inactive afteronError().testOldStreamCompletionDoesNotClearNewerObserver: after a reconnect, adelayed
onCompleted()of the old stream does not detach the newerobserver; only the active stream's completion detaches it.
GrpcClientChannelTesttestSetAndClearClientObserver: set/clear drivesisActive/isOpen/isWritable.testClearClientObserverDoesNotClearNewerObserver:compareAndSetpreventsan old observer from clearing a newer one.
Ran the focused tests and checkstyle under JDK 17: