Skip to content

fix the default retry letter and dead letter topic name - #10129

Merged
codelipenghui merged 4 commits into
apache:masterfrom
wangjialing218:branch-fix-retry-letter
May 15, 2021
Merged

fix the default retry letter and dead letter topic name #10129
codelipenghui merged 4 commits into
apache:masterfrom
wangjialing218:branch-fix-retry-letter

Conversation

@wangjialing218

Copy link
Copy Markdown
Contributor

Motivation

Fixes #9327

Modifications

Correct the default retry letter and dead letter topic name depend on full topic name

@merlimat

merlimat commented Apr 3, 2021

Copy link
Copy Markdown
Contributor

The problem with this change is that by renaming the default, a user who upgrades the client will suddenly have the DLQ messages routed to a different topic, and this could potentially lead to loose these messages.

@wangjialing218

Copy link
Copy Markdown
Contributor Author

Currently, the default name of dead letter topic is {TopicName}-{Subscription}-DLQ while enableRetry(true) is not speicified, but when enableRetry(true) is speicified, the default name of dead letter topic is {NamespaceName}-{Subscription}-DLQ, which will be confused.

@wangjialing218

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

@codelipenghui

Copy link
Copy Markdown
Contributor

@wangjialing218 We need to handle the compatibility issues when users upgrade to the new Pulsar Client. As @merlimat mentioned, this will lead to data loss when upgrading to the new Pulsar Client.

@wangjialing218

wangjialing218 commented Apr 7, 2021

Copy link
Copy Markdown
Contributor Author

@merlimat @codelipenghui Currently a cosumer with retry message enable could receive another topic's retry message due to the same retry letter topic name.
Consider a consumer subscribe to a topic with retry topic enable, and another consumer subscribe to another topic under same namespace with same subscription name, both of the two consumers will subscribe to the retry topic {NamespaceName}-{Subscription}-RETRY, and they could receive the retry messages from each other.
Changing default name of retry letter topic will not cause compatibility issues when users upgrade pulsar client, because consumer will automatic subscribe the retry letter topic at builder time.
For dead letter topic, since user need to specify the topic name, we need to consider how to handle the compatibility issue. such as only change the default name of retry letter name?

@codelipenghui

Copy link
Copy Markdown
Contributor

@wangjialing218 This will cause compatible issues because users might topic1 and topic2 as the retry topic name and DLQ name, but after upgrade to the new client version, the client will subscribe to a new retry topic topic3 and the message which need to send to the DLQ will send to a new topic topic4, this is why we need to consider the compatible issue.

@wangjialing218
wangjialing218 force-pushed the branch-fix-retry-letter branch from 46c4bd2 to 25afc89 Compare April 9, 2021 09:07
@wangjialing218

Copy link
Copy Markdown
Contributor Author

@codelipenghui @merlimat Add compatible check. If old name of DLQ and retry topic exists, using old topics.

@wangjialing218

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

@wangjialing218

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

@sijie sijie added this to the 2.8.0 milestone Apr 13, 2021
@wangjialing218
wangjialing218 force-pushed the branch-fix-retry-letter branch from 0a403bb to fba5f8f Compare April 16, 2021 08:14
@sijie

sijie commented May 4, 2021

Copy link
Copy Markdown
Member

@eolivelli please review this PR again

@eolivelli eolivelli left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

I would like to point out that with this change it is possibile to upgrade to the new version safely but in case of downgrade the user will probably have surprise.

Nice work. Thanks for your contribution

@eolivelli

Copy link
Copy Markdown
Contributor

@codelipenghui PTAL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Retry and Dead letter topics are generated with wrong names when retry is enabled

5 participants