[fix][broker][branch-2.9] Fix system topic schema not compatible bug

[dragonls]

Fixes #17979

Motivation

While upgrading broker version from 2.8.2 to 2.9.3, broker log shows lots of Unable to read schema error, which should not happen.

#12598 has tried to fix the same bug but not completely. schemaCompatibilityStrategy in org.apache.pulsar.broker.service.AbstractTopic is SchemaCompatibilityStrategy.FULL defaultly and only be updated in org.apache.pulsar.broker.service.AbstractTopic#setSchemaCompatibilityStrategy.

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractTopic.java

Lines 552 to 567 in 4abd4d2

	protected void setSchemaCompatibilityStrategy(Policies policies) {
	if (isSystemTopic()) {
	schemaCompatibilityStrategy =
	brokerService.pulsar().getConfig().getSystemTopicSchemaCompatibilityStrategy();
	return;
	}
	schemaCompatibilityStrategy = policies.schema_compatibility_strategy;
	if (SchemaCompatibilityStrategy.isUndefined(schemaCompatibilityStrategy)) {
	schemaCompatibilityStrategy = SchemaCompatibilityStrategy.fromAutoUpdatePolicy(
	policies.schema_auto_update_compatibility_strategy);
	if (SchemaCompatibilityStrategy.isUndefined(schemaCompatibilityStrategy)) {
	schemaCompatibilityStrategy = brokerService.pulsar().getConfig().getSchemaCompatibilityStrategy();
	}
	}
	}

Once org.apache.pulsar.broker.service.AbstractTopic#setSchemaCompatibilityStrategy not called, the schemaCompatibilityStrategy of system topics will always be SchemaCompatibilityStrategy.FULL, which should be brokerService.pulsar().getConfig().getSystemTopicSchemaCompatibilityStrategy() (default SchemaCompatibilityStrategy.ALWAYS_COMPATIBLE) instead.

This bug only exists in branch 2.9 after #13557 meged. The master code is fine.

Modifications

Add schemaCompatibilityStrategy update logic if the topic is system topic, as #12598 did, in constructor of AbstractTopic.

Verifying this change

• Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

• org.apache.pulsar.broker.systopic.NamespaceEventsSystemTopicServiceTest#testSystemTopicSchemaCompatibility

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The binary protocol
• The REST endpoints
• The admin CLI options
• Anything that affects deployment

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository: dragonls#1

[codelipenghui]

Nice catch!

The change looks good.

I try to understand why the setSchemaCompatibilityStrategy is not been called and why it is related to #13557

It looks like we will go to here https://github.com/apache/pulsar/blob/branch-2.9/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L339-L343 which cause missed the setSchemaCompatibilityStrategy?

[dragonls]

Nice catch!

The change looks good.

I try to understand why the setSchemaCompatibilityStrategy is not been called and why it is related to #13557

It looks like we will go to here https://github.com/apache/pulsar/blob/branch-2.9/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L339-L343 which cause missed the setSchemaCompatibilityStrategy?

As for why the setSchemaCompatibilityStrategy is not been called, I think you are right and I am seeing this log:

15:35:09.949 [bookkeeper-ml-scheduler-OrderedScheduler-4-0] WARN  org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://pulsar/mmdcshpulsarbi/30.160.109.200:8080/__change_events-partition-0] Error getting policies KeeperErrorCode = NoNode and isEncryptionRequired will be set to false

After #13557 merged, the system topic schema has been changed, see:

And before this bugfix, schemaCompatibilityStrategy is SchemaCompatibilityStrategy.FULL if setSchemaCompatibilityStrategy is not been called, which cause the schema not compatible error.