Skip to content

[fix][client] Don't pin broker-assigned redirect URL across reconnect retries - #26009

Merged
lhotari merged 1 commit into
apache:masterfrom
merlimat:mmerli/fix-connection-handler-redirect-pinning
Jun 12, 2026
Merged

[fix][client] Don't pin broker-assigned redirect URL across reconnect retries#26009
lhotari merged 1 commit into
apache:masterfrom
merlimat:mmerli/fix-connection-handler-redirect-pinning

Conversation

@merlimat

Copy link
Copy Markdown
Contributor

Fixes #25997

Motivation

#25929 added an explicitHostURI pin to ConnectionHandler so the v5 transaction coordinator's metadata-store discovery re-dials the elected leader on retry instead of falling back to the service URL. The same change also persisted the broker-assigned redirect URL from CommandCloseProducer/CommandCloseConsumer unload notifications (PIP-307) into that pin: a producer/consumer that was redirected once stayed pinned to that address on every subsequent retry, never going through topic lookup again, and nothing ever cleared the field.

A redirect to a wrong or stale broker — e.g. a stepping-down ExtensibleLoadManagerImpl leader whose closeInternalTopics() redirects internal-topic clients to itself via the stale channel-table assignment — now wedged the client permanently with ServiceUnitNotReadyException instead of recovering via lookup on the next retry. This is the cause of the ExtensibleLoadManagerImplTest.testRoleChange flakiness on master. Before #25929 the redirect was honored for the immediate reconnect attempt only, so one failed attempt was followed by a lookup that found the new owner.

Modifications

  • ConnectionHandler.connectionClosed() updates the explicitHostURI pin only for handlers already in explicit-host mode, i.e. the TC metadata-store discovery, which entered it via grabCnx(URI, boolean). The pin update is still needed there: TransactionMetaStoreHandler.retargetLeader() switches leaders by closing the channel and passing the new leader URI through connectionClosed(), and a failed first dial to the new leader must not retry against the old one.
  • Lookup-based handlers (producers/consumers) never enter explicit-host mode, so the broker-assigned redirect is honored for the immediate reconnect attempt only; if that attempt fails, retries fall back to topic lookup, restoring the pre-[improve][broker] PIP-473 P5.3: metadata-store TC leader election + assignment watch #25929 behavior.
  • Added a regression test that delivers a disconnect+redirect with an unreachable assignedBrokerServiceUrl to a live producer and consumer (exactly as ClientCnx#handleCloseProducer/#handleCloseConsumer do) and asserts both recover via lookup and can exchange a message.

Verifying this change

  • Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

… retries

Fixes apache#25997

apache#25929 added an explicitHostURI pin to ConnectionHandler so the v5
transaction coordinator's metadata-store discovery re-dials the elected
leader on retry. The same change also persisted the broker-assigned
redirect URL from CommandCloseProducer/CommandCloseConsumer unload
notifications into that pin: a producer/consumer redirected once stayed
pinned to that address on every subsequent retry, never falling back to
topic lookup. A redirect to a wrong or stale broker (e.g. a stepping-down
ExtensibleLoadManagerImpl leader redirecting internal-topic clients to
itself) wedged the client permanently with ServiceUnitNotReadyException,
causing the testRoleChange flakiness on master.

Update the pin in connectionClosed() only for handlers already in
explicit-host mode, i.e. the TC metadata-store discovery, which entered it
via grabCnx(URI, boolean). The pin update is still needed there:
retargetLeader() switches leaders by closing the channel and passing the
new leader through connectionClosed(), and a failed first dial to the new
leader must not retry against the old one. Lookup-based handlers
(producers/consumers) never enter explicit-host mode, so the redirect is
honored for the immediate reconnect attempt only, restoring the
pre-apache#25929 lookup fallback on failure.
@lhotari
lhotari merged commit 85bdf67 into apache:master Jun 12, 2026
120 of 124 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug][client] Regression in #25929: broker-assigned redirect URL is permanently pinned by ConnectionHandler, breaking lookup fallback on reconnect

2 participants