[Broker] Always let system topic TRANSACTION_BUFFER_SNAPSHOT be auto created

[michaeljmarshall]

Fixes #10871

Motivation

From the linked issue:

While testing the 2.8.0rc2 I found that if you start a Pulsar broker with transactionCoordinatorEnabled=true and allowAutoTopicCreation=false the broker does not work.

The TRANSACTION_BUFFER_SNAPSHOT topic is a system topic, and it should be auto created as needed.

Modifications

1. Updated the logic in the isAllowAutoTopicCreation method to ensure that TRANSACTION_BUFFER_SNAPSHOT is considered a system topic.
2. Updated the logic in checkTopicIsEventsNames to take a TopicName and then check for equality in stead of check endsWith. That could have technically led to unexpected behavior if the topic name was something like my__change_events or my__transaction_buffer_snapshot.
3. Check for isSystemTopicEnabled first to prevent potentially unnecessary string equality checks.

Verifying this change

This is a simple fix that has limit impact. We'll want to verify that the logic is correct regarding allowing TRANSACTION_BUFFER_SNAPSHOT to be auto created. Otherwise, there should be tests covering the uses of this checkTopicIsEventsNames method.

Note that this change will impact the PersistentTopic and the PersistentSubscription classes. Based on reading through the usage of the checkTopicIsEventsNames method, I think this is the right change for those classes. Here are the affected usages:

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java

Lines 317 to 330 in 85a60b0

	TopicName topicName = TopicName.get(topic);
	if (brokerService.getPulsar().getConfiguration().isTransactionCoordinatorEnabled()
	&& !checkTopicIsEventsNames(topic)
	&& !topicName.getEncodedLocalName().startsWith(TopicName.TRANSACTION_COORDINATOR_ASSIGN.getLocalName())
	&& !topicName.getEncodedLocalName().startsWith(MLTransactionLogImpl.TRANSACTION_LOG_PREFIX)
	&& !topicName.getEncodedLocalName().endsWith(MLPendingAckStore.PENDING_ACK_STORE_SUFFIX)) {
	this.transactionBuffer = brokerService.getPulsar()
	.getTransactionBufferProvider().newTransactionBuffer(this, transactionCompletableFuture);
	} else {
	this.transactionCompletableFuture.complete(null);
	this.transactionBuffer = new TransactionBufferDisable();
	}
	transactionBuffer.syncMaxReadPositionForNormalPublish((PositionImpl) ledger.getLastConfirmedEntry());
	}

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentSubscription.java

Lines 143 to 151 in 85a60b0

	if (topic.getBrokerService().getPulsar().getConfig().isTransactionCoordinatorEnabled()
	&& !checkTopicIsEventsNames(topicName)
	&& !topicName.startsWith(TopicName.TRANSACTION_COORDINATOR_ASSIGN.getLocalName())
	&& !topicName.startsWith(MLTransactionLogImpl.TRANSACTION_LOG_PREFIX)
	&& !topicName.endsWith(MLPendingAckStore.PENDING_ACK_STORE_SUFFIX)) {
	this.pendingAckHandle = new PendingAckHandleImpl(this);
	} else {
	this.pendingAckHandle = new PendingAckHandleDisabled();
	}

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

• Dependencies (does it add or upgrade a dependency): no
• The public API: no
• The schema: no
• The default values of configurations: no
• The wire protocol: no
• The rest endpoints: no
• The admin cli options: no
• Anything that affects deployment: no

Documentation

I don't think we need new documentation for this, but if someone more familiar with transactions wants docs, please let me know where I can add docs.

[michaeljmarshall]

@congbobo184 @codelipenghui @sijie @merlimat - PTAL. I am not familiar with the intricacies of the transaction code base within the brokers, but I think this change should fix the issue that @eolivelli discovered in #10871.

[michaeljmarshall]

/pulsarbot run-failure-checks

[eolivelli]

looks good

please add an unit test

[michaeljmarshall]

@eolivelli - done. I didn't see a good way to test the other usages of the checkTopicIsEventsNames method. If someone more familiar with transactions can provide a good way to test this method's usage in those cases, please let me know.

[congbobo184]

LGTM

The unit test was added. Merging now to include in 2.8 rc3

[eolivelli]

LGTM

it would be great to have a new 2.8.0rc3