[improve][broker] Create non-partitioned system topics in auto topic creation - #20397
[improve][broker] Create non-partitioned system topics in auto topic creation#20397lhotari wants to merge 1 commit into
Conversation
lhotari
added
type/bug
|
/pulsarbot rerun-failure-checks |
| // system topics should be non-partitioned by default regardless of the broker config | ||
| // allowAutoTopicCreationType setting | ||
| if (SystemTopicNames.isSystemTopic(topicName) | ||
| || NamespaceService.isSystemServiceNamespace(topicName.getNamespace())) { | ||
| return false; | ||
| } |
There was a problem hiding this comment.
Why must it be non-partitioned?
Should it be configurable?
Should this be discussed in the mail list?
There was a problem hiding this comment.
Why must it be non-partitioned?
It's not "must", it's more about it being a bad default to make system topics such as the __change_events a partitioned topic. The amount of changes going into these topics isn't something that would justify using partitioned topics by default.
Should it be configurable?
If someone wants to use a specific amount of partitions, it's possible to create the system topics manually. Similarly as for transactions.
Should this be discussed in the mail list?
yes. I guess @michaeljmarshall had plans to open a discussion around this.
There was a problem hiding this comment.
I am pretty sure we want most, if not all, of these to be a single topic to ensure a specific ordering of events. It's worth discussing on the mailing list though. I can start something later today.
michaeljmarshall
left a comment
michaeljmarshall
left a comment
There was a problem hiding this comment.
I support the change, but I think we'll need to discuss on the ML first
| // system topics should be non-partitioned by default regardless of the broker config | ||
| // allowAutoTopicCreationType setting | ||
| if (SystemTopicNames.isSystemTopic(topicName) | ||
| || NamespaceService.isSystemServiceNamespace(topicName.getNamespace())) { | ||
| return false; | ||
| } |
There was a problem hiding this comment.
I am pretty sure we want most, if not all, of these to be a single topic to ensure a specific ordering of events. It's worth discussing on the mailing list though. I can start something later today.
|
The pr had no activity for 30 days, mark with Stale label. |
|
For anyone interested, I did raise this one the mailing list here: https://lists.apache.org/thread/1lndgf1hx821fc12t0pc6j6zdrhkntht. There wasn't much engagement. I am going to leave this open for now. |
|
The pr had no activity for 30 days, mark with Stale label. |
lhotari
added
release/3.0.9
release/4.0.2
and removed
release/3.0.8
release/4.0.1
labels
lhotari
added
release/3.0.10
and removed
release/3.0.9
release/2.10.5
release/2.11.2
release/4.0.2
release/3.0.10
labels
7 participants
Motivation
Modifications
Documentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: lhotari#147