Skip to content

[improve][admin] Unset namespace policy to improve deleting namespace. - #17033

Merged
Technoboy- merged 4 commits into
apache:masterfrom
Technoboy-:fix-de-ns-for
Aug 14, 2022
Merged

[improve][admin] Unset namespace policy to improve deleting namespace.#17033
Technoboy- merged 4 commits into
apache:masterfrom
Technoboy-:fix-de-ns-for

Conversation

@Technoboy-

Copy link
Copy Markdown
Contributor

Motivation

When there are many partitioned topics under a namespace, deleting the namespace forcibly will fail, because many requests update ns policies simultaneously and cause the metadata store throws BadVersionException.

See internalRemovePartitionsAuthenticationPoliciesAsync

protected void internalDeletePartitionedTopic(AsyncResponse asyncResponse,
boolean authoritative,
boolean force) {
validateTopicOwnershipAsync(topicName, authoritative)
.thenCompose(__ -> validateNamespaceOperationAsync(topicName.getNamespaceObject(),
NamespaceOperation.DELETE_TOPIC))
.thenCompose(__ -> pulsar().getBrokerService()
.fetchPartitionedTopicMetadataAsync(topicName)
.thenCompose(partitionedMeta -> {
final int numPartitions = partitionedMeta.partitions;
if (numPartitions < 1){
return CompletableFuture.completedFuture(null);
}
return internalRemovePartitionsAuthenticationPoliciesAsync(numPartitions)
.thenCompose(unused -> internalRemovePartitionsTopicAsync(numPartitions, force));
})
// Only tries to delete the znode for partitioned topic when all its partitions are successfully deleted
).thenCompose(__ -> namespaceResources()

private CompletableFuture<Void> internalRemovePartitionsAuthenticationPoliciesAsync(int numPartitions) {
CompletableFuture<Void> future = new CompletableFuture<>();
pulsar().getPulsarResources().getNamespaceResources()
.setPoliciesAsync(topicName.getNamespaceObject(), p -> {
IntStream.range(0, numPartitions)
.forEach(i -> p.auth_policies.getTopicAuthentication()
.remove(topicName.getPartition(i).toString()));
p.auth_policies.getTopicAuthentication().remove(topicName.toString());
return p;
})
.whenComplete((r, ex) -> {

When deleting the namespace, there is no need to update namespace policies, we can delete the namespace zk node in the end.

protected CompletableFuture<Void> internalClearZkSources() {
// clear resource of `/namespace/{namespaceName}` for zk-node
return namespaceResources().deleteNamespaceAsync(namespaceName)
.thenCompose(ignore -> namespaceResources().getPartitionedTopicResources()
.clearPartitionedTopicMetadataAsync(namespaceName))
// clear resource for manager-ledger z-node
.thenCompose(ignore -> pulsar().getPulsarResources().getTopicResources()
.clearDomainPersistence(namespaceName))
.thenCompose(ignore -> pulsar().getPulsarResources().getTopicResources()
.clearNamespacePersistence(namespaceName))
// we have successfully removed all the ownership for the namespace, the policies
// z-node can be deleted now
.thenCompose(ignore -> namespaceResources().deletePoliciesAsync(namespaceName))
// clear z-node of local policies
.thenCompose(ignore -> getLocalPolicies().deleteLocalPoliciesAsync(namespaceName))
// clear /loadbalance/bundle-data
.thenCompose(ignore -> namespaceResources().deleteBundleDataAsync(namespaceName));
}

Documentation

  • doc-not-needed
    (Please explain why)

@Technoboy- Technoboy- modified the milestones: 2.12.0, 2.11.0 Aug 10, 2022
@Technoboy- Technoboy- added type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages area/admin doc-not-needed Your PR changes do not impact docs labels Aug 10, 2022
@Technoboy- Technoboy- self-assigned this Aug 10, 2022

@gaoran10 gaoran10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense, LGTM, left a trivial comment.

@Technoboy-
Technoboy- merged commit 70020f1 into apache:master Aug 14, 2022
Technoboy- added a commit to Technoboy-/pulsar that referenced this pull request Aug 14, 2022
liangyepianzhou pushed a commit that referenced this pull request Mar 20, 2023
liangyepianzhou pushed a commit that referenced this pull request Mar 20, 2023
@liangyepianzhou liangyepianzhou added the cherry-picked/branch-2.9 Archived: 2.9 is end of life label Mar 21, 2023
liangyepianzhou added a commit that referenced this pull request Mar 21, 2023
…ing namespace (#17033) (#19865)

Co-authored-by: Jiwei Guo <technoboy@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/admin cherry-picked/branch-2.9 Archived: 2.9 is end of life cherry-picked/branch-2.10 cherry-picked/branch-2.11 doc-not-needed Your PR changes do not impact docs release/2.9.5 release/2.10.5 release/2.11.1 type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants