[feat][doc] Add docs for message dispatch throttling by momo-jun · Pull Request #386 · apache/pulsar-site

momo-jun · 2023-01-18T08:55:39Z

Fixes apache/pulsar#18506

Modifications

Add docs for message dispatch throttling.
Install plugins and configure docusaurus to support math equations.

Thanks @poorbarcode for providing the draft and technical details, and thanks @heesung-sn for the initial review.

More comments on the initial draft review can be found in the the Google doc.

//cc @Anonymitaet @DaveDuggins
FYI - this PR also enables math equations, and you can take the examples for reference.

Preview screenshots:

doc
doc-required
doc-not-needed
doc-complete

momo-jun · 2023-01-20T07:53:51Z

Ping @tisonkun and @urfreespace to take a look first, because this PR tends to make changes to the Docusaurus configurations to enable the math equation.

tisonkun

Please avoid checking in package-lock.json file. The rest should be fine as long as precommit build passed and your preview locally is correct :)

github-actions · 2023-02-03T09:29:22Z

@momo-jun Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

momo-jun · 2023-02-08T02:25:07Z

Ping @poorbarcode @heesung-sn @codelipenghui @gaoran10 for review.

tisonkun

Generally looks good. Comments inline.

tisonkun · 2023-02-08T11:24:21Z

+
+- **Limit the allocation of client’s hardware resources at topic/subscription levels**
+
+  When there is a large backlog of messages to consume, clients may receive a large amount of data in a short period of time, which monopolizes their computing resources. Since the client has no mechanisms to proactively limit the consumption rate, using the message dispatch throttling feature can also regulate the allocation of the client’s hardware resources.


Since the client has no mechanisms to proactively limit the consumption rate

Why? Consumer#receive receives one message per call. Client app can define their consume logic.

@poorbarcode can you pls take a look at this question?

Hi @tisonkun

Since the client has no mechanisms to proactively limit the consumption rate

Why? Consumer#receive receives one message per call. Client app can define their consume logic.

The code that users consume messages often looks like this:

while ((msg = consumer.receive()) != null){ doService(msg) }

Method doService costs more cpu, memory, and other resources than receiving messages. If there is a lot backlog in this topic( perhaps more than one topic ), then these threads of consumers will keep working busy, which puts other services( such as web service ) on the same machine at a disadvantage.

In a standard service, messages tend to be processed asynchronously (not high priority), so there are scenarios where customers need to limit the amount of system resources consumed by consuming messages to keep other services working

heesung-sohn · 2023-02-14T17:16:39Z

+
+   **Workaround**
+
+   Configuring `preciseDispatcherFlowControl` or `dispatchThrottlingOnBatchMessageEnabled` can mitigate the over-delivery issue. For example, turning on `preciseDispatcherFlowControl` can mitigate the limitation by pre-decrementing the quota using the approximated average message count per entry. See [Throttling configurations](#throttling-configurations) for more details.


If this mitigation also helps the second limitation, please move this to the bottom of this section and mention these configs can help both limitations.

@poorbarcode can this workaround resolve the limitation caused by the second case? I thought the over-delivery limitation can be caused by two cases, and this workaround only serves the first one. Can you please clarify this?

Hi @momo-jun

can this workaround resolve the limitation caused by the second case? I thought the over-delivery limitation can be caused by two cases

Yes, you are right. Thanks for @heesung-sn

heesung-sohn · 2023-02-14T17:22:09Z

+:--------|:------------|:----------|:----------------
+Set [broker configurations](#throttling-configurations) or [dynamic broker configurations](admin-api-brokers.md#dynamic-broker-configuration) | <li>`dispatchThrottlingRateInMsg`</li><li>`dispatchThrottlingRateInByte`</li> | <li>`dispatchThrottlingRatePerTopicInMsg`</li><li>`dispatchThrottlingRatePerTopicInByte`</li><br />It applies to all topics in the cluster. | <li>`dispatchThrottlingRatePerSubscriptionInMsg`</li><li>`dispatchThrottlingRatePerSubscriptionInByte`</li><br />It applies to all subscriptions in the cluster.
+Set namespace policies | N/A | Refer to [Configure dispatch throttling for topics](admin-api-namespaces.md#configure-dispatch-throttling-for-topics). | Refer to [Configure dispatch throttling for subscriptions](admin-api-namespaces.md#configure-dispatch-throttling-for-subscription).
+Set topic policies | N/A | Refer to [Set topic-level dispatch rate](https://pulsar.apache.org/admin-rest-api/#operation/persistent_setDispatchRate). | Refer to [Set subscription-level dispatch rate](https://pulsar.apache.org/admin-rest-api/#operation/getSubscriptionLevelDispatchRate).<br />It applies to all subscriptions in a topic.


Please confirm if these links are in the correct MD format.

This link validity has a dependency on apache/pulsar#19193. I'll double-check it after apache/pulsar#19193 is merged.

Hi @momo-jun

Sorry, I think this PR will be terminated because I have found a better solution, and I will submit a new PR next week or later.

@heesung-sn

Thanks.

@poorbarcode Thanks for letting us know. Keep us posted when the link is ready.

Are you going to develop a new feature? Keep us posted. I'll close this PR when the new one is ready.

No. I will just rewrite the PR apache/pulsar#19193.

Hi @momo-jun @heesung-sn

I have updated the changes in apache/pulsar#19193 to solve the admin API link conflict. Please help review it when you have time.

Hi @momo-jun

apache/pulsar#19193 has been merged.

momo-jun · 2023-03-10T02:06:03Z

@tisonkun @heesung-sn @codelipenghui do you have any other questions before we merge this PR?

gaoran10 · 2023-03-10T07:32:12Z

+
+     When you set the dispatch rate limit in bytes/throttling-period (`dispatchThrottlingRateInByte`/`ratePeriodInSecond`), the broker calculates $$the \ number \ of \ entries \ to \ read \ from \ bookies$$ in one throttling period through the following equation:
+
+     $$


I found there are many symbols $$ and \, it seems that they are not valid markdown symbols.

@gaoran10 Thanks for noticing it. I should have attached all the preview screenshots to make this clear:)

This is a new style to highlight the unknown of the math equation. The preview looks good as follows.

Oh, got it, thanks.

tisonkun

LGTM. One inline comment.

…rottling

momo-jun · 2023-03-13T07:47:19Z

Confirmed with @poorbarcode, this feature has been available since 2.8.x. I will add the docs to those earlier versions in the next commit.

heesung-sohn

lgtm

momo-jun added 2 commits January 18, 2023 16:52

Add draft content and image

8048a07

Install and config for math equation

3fafdba

momo-jun self-assigned this Jan 18, 2023

momo-jun requested review from tisonkun and urfreespace and removed request for tisonkun January 20, 2023 07:30

address review comments

d5b7047

tisonkun reviewed Jan 20, 2023

View reviewed changes

momo-jun added 3 commits January 20, 2023 17:49

remove unexpected check-in package-lock.json

7258ffa

Update concepts-throttling.md

0ab20d7

address review comments

596f712

momo-jun marked this pull request as ready for review February 3, 2023 09:24

github-actions Bot added the doc-label-missing label Feb 3, 2023

github-actions Bot added doc Improvements or additions to documentation and removed doc-label-missing labels Feb 3, 2023

Update concepts-throttling.md

611f610

urfreespace approved these changes Feb 8, 2023

View reviewed changes

tisonkun self-requested a review February 8, 2023 02:39

tisonkun reviewed Feb 8, 2023

View reviewed changes

momo-jun added 4 commits February 9, 2023 11:20

correct the single quote character

9fa8d28

Update concepts-throttling.md

b1f16ec

Update concepts-throttling.md

e31feec

Update concepts-throttling.md

82a46ea

poorbarcode approved these changes Feb 13, 2023

View reviewed changes

heesung-sohn reviewed Feb 14, 2023

View reviewed changes

gaoran10 reviewed Mar 10, 2023

View reviewed changes

gaoran10 approved these changes Mar 12, 2023

View reviewed changes

tisonkun approved these changes Mar 12, 2023

View reviewed changes

Comment thread docs/concepts-throttling.md Outdated

momo-jun added 2 commits March 13, 2023 09:23

Merge remote-tracking branch 'upstream/main' into message-dispatch-th…

7c98754

…rottling

update REST API links and image meta description

6853527

Anonymitaet approved these changes Mar 13, 2023

View reviewed changes

Comment thread docs/concepts-throttling.md Outdated

update REST API links

c2dd888

apply doc updates to versions since 2.8.x

50041fb

momo-jun merged commit 42648bf into apache:main Mar 13, 2023

heesung-sohn reviewed Mar 13, 2023

View reviewed changes

Anonymitaet added this to the 3.0.0 milestone Apr 28, 2023


		- Limit the allocation of client’s hardware resources at topic/subscription levels

		When there is a large backlog of messages to consume, clients may receive a large amount of data in a short period of time, which monopolizes their computing resources. Since the client has no mechanisms to proactively limit the consumption rate, using the message dispatch throttling feature can also regulate the allocation of the client’s hardware resources.


		Workaround

		Configuring `preciseDispatcherFlowControl` or `dispatchThrottlingOnBatchMessageEnabled` can mitigate the over-delivery issue. For example, turning on `preciseDispatcherFlowControl` can mitigate the limitation by pre-decrementing the quota using the approximated average message count per entry. See [Throttling configurations](#throttling-configurations) for more details.


		When you set the dispatch rate limit in bytes/throttling-period (`dispatchThrottlingRateInByte`/`ratePeriodInSecond`), the broker calculates $$the \ number \ of \ entries \ to \ read \ from \ bookies$$ in one throttling period through the following equation:

		$$

Uh oh!

Conversation

momo-jun commented Jan 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Modifications

Uh oh!

momo-jun commented Jan 20, 2023

Uh oh!

tisonkun left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Feb 3, 2023

Uh oh!

momo-jun commented Feb 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tisonkun left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

poorbarcode Feb 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

poorbarcode Feb 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

momo-jun Feb 26, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

momo-jun commented Mar 10, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

momo-jun Mar 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tisonkun left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

momo-jun commented Mar 13, 2023

Uh oh!

heesung-sohn left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

momo-jun commented Jan 18, 2023 •

edited

Loading

momo-jun commented Feb 8, 2023 •

edited

Loading

poorbarcode Feb 24, 2023 •

edited

Loading

poorbarcode Feb 24, 2023 •

edited

Loading

momo-jun Feb 26, 2023 •

edited

Loading

momo-jun Mar 11, 2023 •

edited

Loading