[fix] PIP-457: Remove support for V1 topic names and V1 Admin API (implementation) - #25304
Merged
Conversation
merlimat
added
type/cleanup
github-actions
Bot
added
the
doc-required
…plementation)
This commit implements PIP-457 by removing all V1 topic name support
and V1 Admin API endpoints from Apache Pulsar.
Topic naming changes:
- Remove V1 topic name format (persistent://tenant/cluster/namespace/topic)
Only V2 format is now supported (persistent://tenant/namespace/topic)
- Remove V1 namespace name format (tenant/cluster/namespace)
Only V2 format is now supported (tenant/namespace)
- TopicName and NamespaceName constructors now reject V1 format names
- fromPersistenceNamingEncoding() still handles 5-part managed ledger
names for backward compatibility with existing stored data
V1 Admin API removal:
- Delete entire org.apache.pulsar.broker.admin.v1 package
(Namespaces, PersistentTopics, NonPersistentTopics, Properties,
Brokers, BrokerStats, Clusters, Functions, ResourceQuotas,
SchemasResource)
- Delete V1 topic lookup endpoint (broker.lookup.v1.TopicLookup)
- Delete V1 WebSocket stats endpoint (websocket.admin.v1)
- Remove V1 servlet registrations from PulsarWebResource,
ProxyServiceStarter, WebSocketServiceStarter
- Remove /admin/v1/ path prefix handling from all client admin impls
(BrokerStatsImpl, BrokersImpl, LookupImpl, NamespacesImpl,
NonPersistentTopicsImpl, ResourceQuotasImpl, SchemasImpl,
TopicPoliciesImpl, TopicsImpl)
- Remove V1 cluster-aware namespace handling from NamespacesBase
Deprecated Properties API removal:
- Delete Properties.java interface (V1 term for Tenants)
- Remove Properties compat bridge methods from TenantsImpl
- Remove properties() accessor from PulsarAdmin/PulsarAdminImpl
TopicVersion enum removal:
- Delete TopicVersion.java (V1/V2 enum)
- Remove TopicVersion parameter from healthcheck API
(Brokers interface, BrokersImpl, BrokersBase, CmdBrokers)
- Remove topicVersion HTTP query parameter from healthcheck requests
- Simplify PulsarService.runHealthCheck() and BrokerService
V1 heartbeat and SLA namespace cleanup:
- Remove V1 heartbeat namespace format (pulsar/cluster/brokerId)
Rename V2 format methods to be the standard (pulsar/brokerId)
- Update SLA namespace to V2 format (sla-monitor/brokerId)
- Simplify HealthChecker to use single heartbeat topic
- Remove V1 heartbeat registration from ServiceUnitStateTableViewBase
Property to tenant terminology rename:
- Rename parameter/variable names from "property" to "tenant"
in AdminResource, TransactionsBase, and 7 V2 Namespaces endpoints
- Fix @PathParam("property") to @PathParam("tenant") in V2 Namespaces
- Rename in test utilities and test variables
Configuration cleanup:
- Remove allowAutoTopicCreationType config (V1 only)
- Remove V1 auto-topic-creation settings from broker.conf/standalone.conf
Test cleanup:
- Delete V1-specific test suites (V1AdminApiTest, V1AdminApi2Test,
V1ProducerConsumerTest, V1ProducerConsumerBase,
V1ProxyAuthenticationTest)
- Remove V1 test cases from TopicNameTest, NamespaceNameTest
- Update all remaining tests to use V2 format exclusively
- Remove TopicVersion test parameters from AdminApiHealthCheckTest
- Update SLAMonitoringTest for 2-part SLA namespace format
- Fix import order in AdminResource (HashSet before Map) - Remove unused ArrayList and List imports from PulsarWebResource
Migrate all test files from V1 topic name format (persistent://tenant/cluster/namespace/topic) to V2 format (persistent://tenant/namespace/topic). Key changes: - Replace V1 topic names with V2 in 82 test files across broker, proxy, websocket, client, and compaction tests - Update MockBrokerService HTTP handler to match V2 admin URLs (/admin/v2/persistent and /lookup/v2/topic/persistent) - Fix test setup cluster references from "use" to "test" to match the broker's configured cluster name - Add my-property/my-ns tenant+namespace creation in BrokerTestBase for tests that previously relied on V1 implicit namespace resolution - Add namespace creation in ManagedCursorMetricsTest and StrategicCompactionTest setups - Fix ResendRequestTest and PerMessageUnAcknowledgedRedeliveryTest to call baseSetup() instead of internalSetup() - Remove V1-specific test methods (namespacesCreateV1, namespacesCreateV1WithBundlesAndClusters) - Remove V1-specific test assertions (non-local cluster topic rejection in ServerCnxTest, V1 managed ledger path listener in TopicResourcesTest) - Update PulsarAdminToolTest createNamespace verifications to match V2 API call signature
…ckstyle - Convert 4-part V1 topic names to 3-part V2 format in proxy tests, BrokerServiceLookupTest, and ConnectionPoolTest - Fix V1 3-part namespace creation in ModularLoadManagerImplTest and AdminApiSchemaTest - Remove V1-specific test methods from SchemaUpdateStrategyTest - Remove duplicate tenant creation in ResendRequestTest - Update V1 format references in comments and CLI parameter descriptions - Fix checkstyle import order in StrategicCompactionTest and ManagedCursorMetricsTest
…pace Use public/default namespace (created by setupDefaultTenantAndNamespace) instead of sample/local which was never created by these tests.
- NamespaceIsolationPolicyImplTest: Change V1 3-part namespaces (pulsar/use/testns-1) to V2 format (pulsar/testns-1) in namespace regex patterns and NamespaceName.get() calls - NamespaceIsolationPoliciesTest: Fix testBrokerAssignment to use a namespace that doesn't match the policy for the shared broker test, and fix case-sensitivity issue in testGetNamespaceIsolationPolicyByNamespace
Change /ws/consumer/ and /ws/producer/ (V1 4-part format with cluster) to /ws/v2/consumer/ and /ws/v2/producer/ (V2 3-part format).
…Test
Replace getNamespaces("prop-xyz", "test") (V1 2-arg cluster-scoped API)
with getNamespaces("prop-xyz") (V2 tenant-scoped API).
- mockBundleName: change 3-part V1 format to 2-part V2 format - testBrokerAffinity: remove cluster from namespace path - testLoadSheddingWithNamespaceIsolationPolicies: remove cluster from namespace string - testNamespaceIsolationPoliciesForPrimaryAndSecondaryBrokers: fix misaligned String.format args after template change from 3-part to 2-part namespace format
Remove V1 3-part namespace formats (tenant/cluster/namespace) and V1 lookup URLs (/lookup/v2/destination/) from tests. Add missing tenant/namespace creation and replication_clusters configuration where V2 topic operations now require them. Files fixed: AdminTest, NamespacesV2Test, PersistentTopicsTest, TopicPoliciesTest, AntiAffinityNamespaceGroupExtensionTest, HttpTopicLookupv2Test, PeerReplicatorTest, ReplicatorGlobalNSTest, WebServiceTest.
merlimat
marked this pull request as draft
March 10, 2026 05:50
The broker's internal PulsarClient (created lazily by SystemTopicBasedTopicPoliciesService during namespace bundle ownership) adds extra binary connections during test setup. Use >= assertions for cumulative connection counts that include setup connections, while keeping exact assertions for failure count and zero-value checks.
- Remove "global" from mocked clusters set (V1-specific) - Remove test block for V1-specific "global not allowed in replication clusters" behavior - Add peer cluster configuration for "usc" since NamespaceName.isGlobal() now always returns true, causing all namespaces to go through the peer-cluster redirect path in checkLocalOrGetPeerReplicationCluster - Update deleteNamespace redirect URL expectation to include ?authoritative=false query param added by validateNamespacePoliciesAsync
After V1 removal, managed ledger names use the format tenant/namespace/domain/topic. The parseNamespaceFromLedgerName method captured the first 3 path components (tenant/namespace/domain) as the namespace, which was correct for V1 format (tenant/cluster/namespace/domain/topic) but incorrect for V2. Fix by extracting only groups 2 and 3 (tenant/namespace) from the regex, excluding the domain component.
merlimat
marked this pull request as ready for review
March 10, 2026 19:55
After V1 removal, NamespaceName.isGlobal() always returns true, causing checkLocalOrGetPeerReplicationCluster to run for all namespaces. The guard at line 875 excluded heartbeat namespaces but not SLA namespaces, so SLA namespace lookups failed with "Namespace not found" when no policies existed in the metadata store. Fix: use isSLAOrHeartbeatNamespace() to skip peer cluster validation for both system namespace types. Also fix DispatcherBlockConsumerTest.testBlockDispatcherStats which referenced namespace prop/ns-abc that is not created by ProducerConsumerBase setup.
# Conflicts: # pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v1/Namespaces.java # pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApi2Test.java # pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/NamespacesTest.java # pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/v1/V1AdminApi2Test.java # pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerTestBase.java # pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ReplicatorTestBase.java # pulsar-broker/src/test/java/org/apache/pulsar/client/api/NonPersistentTopicTest.java # pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/internal/NamespacesImpl.java # pulsar-client-tools-test/src/test/java/org/apache/pulsar/admin/cli/PulsarAdminToolTest.java # pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdNamespaces.java # tests/pulsar-client-admin-shade-test/src/test/java/org/apache/pulsar/tests/integration/SimpleProducerConsumerTest.java # tests/pulsar-client-all-shade-test/src/test/java/org/apache/pulsar/tests/integration/SimpleProducerConsumerTest.java # tests/pulsar-client-shade-test/src/test/java/org/apache/pulsar/tests/integration/SimpleProducerConsumerTest.java
Changed TopicName constructor to use limit(4) split and reject 4-part paths (tenant/cluster/namespace/topic) instead of silently misinterpreting them as V2 names. Also updated toFullTopicName() with the same check. Removed support for slashes in V2 topic local names since V2 format does not allow them. Updated websocket handler to no longer join slash-separated URI path segments into topic names.
Restored the check that prevents removing clusters from a tenant's allowed list when namespaces still reference those clusters. The implementation is adapted for V2: instead of checking the V1 cluster path hierarchy, it now lists all namespaces under the tenant and checks their replication_clusters policy field.
Keep the existing namespace dimension format (tenant/namespace/domain) to avoid breaking existing dashboards and alerts. The cleanup can be done separately.
- AbstractWebSocketHandlerTest: remove slashes from test topic names - BrokerBkEnsemblesTest: remove slashes from wildcard char topic name - ResourceGroupUsageAggregationTest: fix topic name that was accidentally constructing a V1 4-part path (pulsar-test/test/test/topic) - NamespacesTest: remove usw from namespace replication clusters before removing it from tenant allowed clusters, matching canUpdateCluster validation semantics
- AbstractWebSocketHandlerTest: remove slashes from test topic names - BrokerBkEnsemblesTest: remove slashes from wildcard char topic name - ResourceGroupUsageAggregationTest: fix topic name that was accidentally constructing a V1 4-part path (pulsar-test/test/test/topic) - NamespacesTest: remove usw from namespace replication clusters before removing it from tenant allowed clusters, matching canUpdateCluster validation semantics
shibd
reviewed
Mar 12, 2026
The V1 cluster-scoped namespace listing is no longer relevant. Removed the method from Namespaces interface, NamespacesImpl, the list-cluster CLI command, and the validatePropertyCluster helper.
Remove r4 from pulsar/system namespace replication clusters before shrinking the pulsar tenant's allowed clusters from r1-r4 to r1-r3. The canUpdateCluster validation now correctly prevents removing a cluster from a tenant when namespaces still reference it.
Remove r4 from pulsar/system namespace replication clusters before shrinking the pulsar tenant's allowed clusters from r1-r4 to r1-r3. The canUpdateCluster validation now correctly prevents removing a cluster from a tenant when namespaces still reference it.
Add null check for topic policies in Awaitility assertion to handle the case where policies haven't propagated yet.
Add null check for topic policies in Awaitility assertion to handle the case where policies haven't propagated yet.
4 tasks
11 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit implements PIP-457 by removing all V1 topic name support and V1 Admin API endpoints from Apache Pulsar.
Topic naming changes:
V1 Admin API removal:
Deprecated Properties API removal:
TopicVersion enum removal:
V1 heartbeat and SLA namespace cleanup:
Property to tenant terminology rename:
Configuration cleanup:
Test cleanup:
Fixes #xyz
Main Issue: #xyz
PIP: #xyz
Motivation
Modifications
Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Documentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: