Fix create consumer on partitioned topic while disable topic auto creation. by codelipenghui · Pull Request #5572 · apache/pulsar

codelipenghui · 2019-11-06T09:56:30Z

Motivation

Currently, disable the topic auto creation will cause consumer create failed on a partitioned topic. Since the partitioned topic is already created, so we should handle the topic partition create when disable the topic auto creation.

Modifications

By default, create partitioned topics also try to create all partitions, and if create partitions failed, users can use create-missed-partitions to repair.

If users already have a partitioned topic without created partitions, can also use create-missed-partitions to repair.

Verifying this change

Add new unit tests for this change

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): (no)
The public API: (no)
The schema: (no)
The default values of configurations: (no)
The wire protocol: (no)
The rest endpoints: (no)
The admin cli options: (no)
Anything that affects deployment: (no)

Documentation

Does this pull request introduce a new feature? (no)

…ation.

codelipenghui · 2019-11-06T09:57:13Z

@wolfstudy if this PR can complete before cut 2.4.2, please considering include it, thanks.

sijie

I don't think that is the right fix for this problem. The purpose of disabling topic auto-creation is to prevent consumers and producers creating topics.

I think the correct fix should be improving the pulsar-admin tool to create and delete partitions when creating or updating a partitioned topic.

codelipenghui · 2019-11-12T07:50:56Z

run integration tests

codelipenghui · 2019-11-12T08:24:47Z

run java8 tests

wolfstudy · 2019-11-13T03:16:47Z

@codelipenghui l will change the Milestone to 2.4.3. So we can cut 2.4.2 and if needed
2.4.3 in a few weeks.

codelipenghui · 2019-11-13T03:18:07Z

@wolfstudy Thanks

codelipenghui · 2019-11-13T06:44:30Z

run java8 tests

codelipenghui · 2019-11-16T00:42:30Z

run java8 tests

codelipenghui · 2019-11-19T00:42:34Z

run java8 tests

codelipenghui · 2019-11-19T01:54:54Z

run java8 tests

codelipenghui · 2019-11-19T03:42:01Z

run java8 tests

codelipenghui · 2019-11-19T06:44:53Z

run java8 tests

codelipenghui · 2019-11-19T12:10:02Z

@sijie Please help review this PR again

sijie · 2019-11-25T20:57:51Z

@codelipenghui are you going to include this fix in 2.5.0 or shall we move it to 2.5.1 or 2.6.0?

codelipenghui · 2019-11-27T02:04:00Z

it can move to 2.5.1 if 2.5.0 branch will start cut soon. Sijie Guo <notifications@github.com>于2019年11月26日周二04:57写道：

…

@codelipenghui <https://github.com/codelipenghui> are you going to include this fix in 2.5.0 or shall we move it to 2.5.1 or 2.6.0? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5572?email_source=notifications&email_token=ADACIBNEFVHYYSH7LBLXYSLQVQ35BA5CNFSM4JJSNE2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFDYW6Q#issuecomment-558336890>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADACIBNF3KJK4F5B2MUDI3DQVQ35BANCNFSM4JJSNE2A> .

codelipenghui · 2019-12-24T03:53:30Z

@sijie I have addressed your comments, PTAL.

codelipenghui · 2019-12-24T06:07:19Z

run java8 tests

codelipenghui · 2019-12-24T09:13:45Z

run java8 tests
run integration tests

codelipenghui · 2019-12-24T11:02:15Z

run integration tests

codelipenghui · 2019-12-24T12:29:20Z

run integration tests

…ation. (#5572) ### Motivation Currently, disable the topic auto creation will cause consumer create failed on a partitioned topic. Since the partitioned topic is already created, so we should handle the topic partition create when disable the topic auto creation. ### Modifications By default, create partitioned topics also try to create all partitions, and if create partitions failed, users can use `create-missed-partitions` to repair. If users already have a partitioned topic without created partitions, can also use `create-missed-partitions` to repair. (cherry picked from commit 602f1c2)

jiazhai · 2020-02-17T07:01:14Z

done the cherry-pick to branch-2.5

…ation. (#5572) ### Motivation Currently, disable the topic auto creation will cause consumer create failed on a partitioned topic. Since the partitioned topic is already created, so we should handle the topic partition create when disable the topic auto creation. ### Modifications By default, create partitioned topics also try to create all partitions, and if create partitions failed, users can use `create-missed-partitions` to repair. If users already have a partitioned topic without created partitions, can also use `create-missed-partitions` to repair. (cherry picked from commit 602f1c2)

…ation. (apache#5572) ### Motivation Currently, disable the topic auto creation will cause consumer create failed on a partitioned topic. Since the partitioned topic is already created, so we should handle the topic partition create when disable the topic auto creation. ### Modifications By default, create partitioned topics also try to create all partitions, and if create partitions failed, users can use `create-missed-partitions` to repair. If users already have a partitioned topic without created partitions, can also use `create-missed-partitions` to repair. (cherry picked from commit 602f1c2)

…ation. (apache#5572) Currently, disable the topic auto creation will cause consumer create failed on a partitioned topic. Since the partitioned topic is already created, so we should handle the topic partition create when disable the topic auto creation. By default, create partitioned topics also try to create all partitions, and if create partitions failed, users can use `create-missed-partitions` to repair. If users already have a partitioned topic without created partitions, can also use `create-missed-partitions` to repair.

…ation. (apache#5572) ### Motivation Currently, disable the topic auto creation will cause consumer create failed on a partitioned topic. Since the partitioned topic is already created, so we should handle the topic partition create when disable the topic auto creation. ### Modifications By default, create partitioned topics also try to create all partitions, and if create partitions failed, users can use `create-missed-partitions` to repair. If users already have a partitioned topic without created partitions, can also use `create-missed-partitions` to repair.

Fix create consumer on partitioned topic while disable topic auto cre…

b88939a

…ation.

codelipenghui requested review from jiazhai, merlimat, sijie and wolfstudy November 6, 2019 09:56

fix checkstyle

dd281b8

wolfstudy added this to the 2.4.2 milestone Nov 6, 2019

sijie requested changes Nov 6, 2019

View reviewed changes

wolfstudy assigned codelipenghui Nov 6, 2019

wolfstudy added the type/bug The PR fixed a bug or issue reported a bug label Nov 6, 2019

Try to create partitions when create partitioned topic.

d308dd9

codelipenghui requested a review from sijie November 12, 2019 07:54

codelipenghui mentioned this pull request Nov 12, 2019

Enable or disable topic auto creation for non-persistent topic. #5631

Open

wolfstudy modified the milestones: 2.4.2, 2.4.3 Nov 13, 2019

Fix unit tests

ff5379d

Fix unit tests

6ae3c5c

sijie requested changes Nov 20, 2019

View reviewed changes

codelipenghui modified the milestones: 2.4.3, 2.5.0 Nov 25, 2019

sijie modified the milestones: 2.5.0, 2.5.1 Nov 27, 2019

Fix comments

06bb67f

sijie approved these changes Dec 24, 2019

View reviewed changes

codelipenghui merged commit 602f1c2 into apache:master Jan 9, 2020

sijie mentioned this pull request Jan 22, 2020

[BUG] User report getOffset not return right value. streamnative/kop#79

Open

sijie added the release/2.5.1 label Jan 22, 2020

sijie modified the milestones: 2.5.1, 2.6.0 Jan 22, 2020

codelipenghui mentioned this pull request Apr 21, 2020

Update apache pulsar 2.5.1 release notes #6774

Merged

codelipenghui deleted the issue-5565 branch October 27, 2020 10:18

Uh oh!

Conversation

codelipenghui commented Nov 6, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

Verifying this change

Does this pull request potentially affect one of the following parts:

Documentation

Uh oh!

codelipenghui commented Nov 6, 2019

Uh oh!

sijie left a comment

Choose a reason for hiding this comment

Uh oh!

codelipenghui commented Nov 12, 2019

Uh oh!

codelipenghui commented Nov 12, 2019

Uh oh!

wolfstudy commented Nov 13, 2019

Uh oh!

codelipenghui commented Nov 13, 2019

Uh oh!

codelipenghui commented Nov 13, 2019

Uh oh!

codelipenghui commented Nov 16, 2019

Uh oh!

codelipenghui commented Nov 19, 2019

Uh oh!

codelipenghui commented Nov 19, 2019

Uh oh!

codelipenghui commented Nov 19, 2019

Uh oh!

codelipenghui commented Nov 19, 2019

Uh oh!

codelipenghui commented Nov 19, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sijie commented Nov 25, 2019

Uh oh!

codelipenghui commented Nov 27, 2019 via email

Uh oh!

codelipenghui commented Dec 24, 2019

Uh oh!

codelipenghui commented Dec 24, 2019

Uh oh!

codelipenghui commented Dec 24, 2019

Uh oh!

codelipenghui commented Dec 24, 2019

Uh oh!

codelipenghui commented Dec 24, 2019

Uh oh!

jiazhai commented Feb 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codelipenghui commented Nov 6, 2019 •

edited

Loading

jiazhai commented Feb 17, 2020 •

edited

Loading