Skip to content

[Issue 2689] Support set backlog quota on topic level. - #7646

Merged
codelipenghui merged 9 commits into
apache:masterfrom
jianyun8023:topic-policy-backlog-quota
Jul 31, 2020
Merged

[Issue 2689] Support set backlog quota on topic level.#7646
codelipenghui merged 9 commits into
apache:masterfrom
jianyun8023:topic-policy-backlog-quota

Conversation

@jianyun8023

@jianyun8023 jianyun8023 commented Jul 23, 2020

Copy link
Copy Markdown
Contributor

Motivation

Support set backlog quota on topic level.
Based on the system topic function, refer to @codelipenghui topic-level backlog quota based on zk implementation

###Modifications
Support get-backlog-quotas on topic level.
Support set-backlog-quota on topic level.
Support remove-backlog-quota on topic level.

Verifying this change

This change added tests and can be verified as follows:
TopicBacklogQuotaTest.java

  • test set topic backlog quota
  • test remove topic backlog quota
  • test disabled topic backlog quota
  • test topic backlog quota parameters are valid

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API: (no)
  • The schema: (don't know)
  • The default values of configurations: (yes / no)
  • The wire protocol: (yes / no)
  • The rest endpoints: (yes / no)
  • The admin cli options: (yes)
  • Anything that affects deployment: (no)

Documentation

  • Does this pull request introduce a new feature? (yes)
  • If yes, how is the feature documented? (docs / JavaDocs)

codelipenghui
codelipenghui reviewed Jul 23, 2020
View reviewed changes

@codelipenghui codelipenghui left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, just left some minor comments.

Comment thread pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java Outdated
Comment thread pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java Outdated
Comment thread pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java Outdated
Comment on lines +2081 to +2094
private boolean checkQuotas(BacklogQuota quota, RetentionPolicies retention) {
if (retention==null||retention.getRetentionSizeInMB() == 0 ||
retention.getRetentionSizeInMB() == -1) {
return true;
}
if (quota == null) {
quota = pulsar().getBrokerService().getBacklogQuotaManager().getDefaultQuota();
}
if (quota.getLimit() >= ( retention.getRetentionSizeInMB() * 1024 * 1024)) {
return false;
}
return true;
}

@codelipenghui codelipenghui Jul 23, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should keep consistent with checkQuotas in the NamespacesBase, And you can move this method to AdminResource so that you can use it both in the NamespacesBase and PersistentTopicsBase

@jianyun8023 jianyun8023 Jul 24, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I extracted it.

Comment thread pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/PersistentTopics.java
@codelipenghui codelipenghui added this to the 2.7.0 milestone Jul 23, 2020
@codelipenghui

codelipenghui commented Jul 24, 2020

Copy link
Copy Markdown
Contributor

/pulsarbot run-failure-checks

3 similar comments
@jianyun8023

jianyun8023 commented Jul 24, 2020

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

@jianyun8023

jianyun8023 commented Jul 26, 2020

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

@codelipenghui

codelipenghui commented Jul 27, 2020

Copy link
Copy Markdown
Contributor

/pulsarbot run-failure-checks

@codelipenghui codelipenghui added doc-required Your PR changes impact docs and you will update later. component/topic-policy labels Jul 27, 2020
@jianyun8023
jianyun8023 force-pushed the topic-policy-backlog-quota branch from c705b50 to 3dc2dab Compare July 28, 2020 01:37
@jianyun8023

jianyun8023 commented Jul 28, 2020

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

@jianyun8023

jianyun8023 commented Jul 30, 2020

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

1 similar comment
@jianyun8023

jianyun8023 commented Jul 30, 2020

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

@codelipenghui
codelipenghui merged commit 26c49a8 into apache:master Jul 31, 2020
@315157973 315157973 mentioned this pull request Aug 3, 2020
Closed
16 tasks
@codelipenghui codelipenghui mentioned this pull request Aug 3, 2020
Closed
This was referenced Aug 3, 2020
Closed
Closed
@sijie sijie mentioned this pull request Aug 10, 2020
Closed
13 tasks
@Huanli-Meng Huanli-Meng mentioned this pull request Aug 19, 2020
Merged
huangdx0726 pushed a commit to huangdx0726/pulsar that referenced this pull request Aug 24, 2020
@dd-ray @huangdx0726
[Issue 2689] Support set backlog quota on topic level. (apache#7646)
f6edf46 
### Motivation
Support set backlog quota on topic level.
Based on the system topic function, refer to @codelipenghui  topic-level backlog quota based on zk implementation

###Modifications
Support get-backlog-quotas on topic level.
Support set-backlog-quota on topic level.
Support remove-backlog-quota on topic level.
lbenc135 pushed a commit to lbenc135/pulsar that referenced this pull request Sep 5, 2020
@dd-ray Livio
[Issue 2689] Support set backlog quota on topic level. (apache#7646)
29aaad5 
### Motivation
Support set backlog quota on topic level.
Based on the system topic function, refer to @codelipenghui  topic-level backlog quota based on zk implementation

###Modifications
Support get-backlog-quotas on topic level.
Support set-backlog-quota on topic level.
Support remove-backlog-quota on topic level.
lbenc135 pushed a commit to lbenc135/pulsar that referenced this pull request Sep 5, 2020
@dd-ray Livio
[Issue 2689] Support set backlog quota on topic level. (apache#7646)
79414b4 
### Motivation
Support set backlog quota on topic level.
Based on the system topic function, refer to @codelipenghui  topic-level backlog quota based on zk implementation

###Modifications
Support get-backlog-quotas on topic level.
Support set-backlog-quota on topic level.
Support remove-backlog-quota on topic level.
jiazhai pushed a commit that referenced this pull request Sep 15, 2020
@Huanli-Meng
[Doc]--Update Pulsar Admin CLI doc to support to set/get/remove persi…
fd2319f 
…stence polices, message TTL, and backlog quota on topic level (#7852)

Motivation
In PRs, #7738, #7646, #7817, persistence polices, message TTL, and backlog quota policies are supported on topic level. The PR for code modification is updated and merged. but the doc is not updated accordingly.

This PR is used to support to set/get/remove persistence polices, message TTL, and backlog quota on topic level.

Modifications
Update the Pulsar Admin CLI doc to support to set/get/remove persistence polices, message TTL, and backlog quota on topic level.

* Update backlog-quota policies on topic level

* update contents

* update contents
lbenc135 pushed a commit to lbenc135/pulsar that referenced this pull request Oct 3, 2020
@Huanli-Meng livio
[Doc]--Update Pulsar Admin CLI doc to support to set/get/remove persi…
db965b5 
…stence polices, message TTL, and backlog quota on topic level (apache#7852)

Motivation
In PRs, apache#7738, apache#7646, apache#7817, persistence polices, message TTL, and backlog quota policies are supported on topic level. The PR for code modification is updated and merged. but the doc is not updated accordingly.

This PR is used to support to set/get/remove persistence polices, message TTL, and backlog quota on topic level.

Modifications
Update the Pulsar Admin CLI doc to support to set/get/remove persistence polices, message TTL, and backlog quota on topic level.

* Update backlog-quota policies on topic level

* update contents

* update contents
@wolfstudy

wolfstudy commented Oct 28, 2020

Copy link
Copy Markdown
Member

Move this change to 2.6.2, because the #8199 depends on it.

wolfstudy pushed a commit that referenced this pull request Oct 30, 2020
@dd-ray @wolfstudy
[Issue 2689] Support set backlog quota on topic level. (#7646)
ff0f7b0 
### Motivation
Support set backlog quota on topic level.
Based on the system topic function, refer to @codelipenghui  topic-level backlog quota based on zk implementation

###Modifications
Support get-backlog-quotas on topic level.
Support set-backlog-quota on topic level.
Support remove-backlog-quota on topic level.

(cherry picked from commit 26c49a8)
@Anonymitaet Anonymitaet added doc Your PR contains doc changes, no matter whether the changes are in markdown or code files. and removed doc-required Your PR changes impact docs and you will update later. labels Feb 21, 2022
@jianyun8023
jianyun8023 deleted the topic-policy-backlog-quota branch April 15, 2022 01:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@codelipenghui codelipenghui codelipenghui approved these changes
@sijie sijie Awaiting requested review from sijie
@merlimat merlimat Awaiting requested review from merlimat
@jiazhai jiazhai Awaiting requested review from jiazhai

Assignees

@jianyun8023 jianyun8023

Labels

doc Your PR contains doc changes, no matter whether the changes are in markdown or code files. release/2.6.2

Projects

None yet

Milestone

2.7.0

Development

Successfully merging this pull request may close these issues.

4 participants

@jianyun8023 @codelipenghui @wolfstudy @Anonymitaet