[improve][java-client] Only trigger the batch receive timeout when having pending batch receives requests - #16160
Merged
codelipenghui merged 4 commits intoJun 23, 2022
Conversation
…ving pending batch receives requests ### Motivation The consumer will apply the default batch receive policy even if the user will not use the batch receive API. https://github.com/apache/pulsar/blob/6704f12104219611164aa2bb5bbdfc929613f1bf/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/BatchReceivePolicy.java#L60-L61 This will consume lots of CPU if the client have many consumers (100k consumers) The Pulsar perf tool can also reproduce the problem if run the test with many consumers ### Modification If there is no pending batch receive operation for a consumer, no need to trigger the batch timeout task periodically. We can only start the timeout check after adding batch receive request to pending request queue. Remove the lock in MultiTopicsConsumerImpl as apache#10352 does ### Verification Added new test to verify the batch receive timeout task will not start if no batch receive request
codelipenghui
added
type/enhancement
|
@codelipenghui Please provide a correct documentation label for your PR. |
codelipenghui
added
doc-not-needed
codelipenghui
requested review from
315157973,
Jason918,
congbobo184,
hangc0276,
merlimat and
michaeljmarshall
June 21, 2022 15:13
merlimat
approved these changes
Jun 21, 2022
michaeljmarshall
requested changes
Jun 21, 2022
michaeljmarshall
left a comment
michaeljmarshall
left a comment
Member
There was a problem hiding this comment.
@codelipenghui - this is a great improvement. I have one concern about concurrent timer tasks, but otherwise, this looks good.
Jason918
reviewed
Jun 22, 2022
Jason918
approved these changes
Jun 22, 2022
codelipenghui
added a commit
that referenced
this pull request
Jun 28, 2022
…ving pending batch receives requests (#16160) The consumer will apply the default batch receive policy even if the user will not use the batch receive API. https://github.com/apache/pulsar/blob/6704f12104219611164aa2bb5bbdfc929613f1bf/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/BatchReceivePolicy.java#L60-L61 This will consume lots of CPU if the client have many consumers (100k consumers) The Pulsar perf tool can also reproduce the problem if run the test with many consumers If there is no pending batch receive operation for a consumer, no need to trigger the batch timeout task periodically. We can only start the timeout check after adding batch receive request to pending request queue. Remove the lock in MultiTopicsConsumerImpl as #10352 does Added new test to verify the batch receive timeout task will not start if no batch receive request (cherry picked from commit a0ccdc9)
nicoloboschi
pushed a commit
to datastax/pulsar
that referenced
this pull request
Jul 4, 2022
…ving pending batch receives requests (apache#16160) The consumer will apply the default batch receive policy even if the user will not use the batch receive API. https://github.com/apache/pulsar/blob/6704f12104219611164aa2bb5bbdfc929613f1bf/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/BatchReceivePolicy.java#L60-L61 This will consume lots of CPU if the client have many consumers (100k consumers) The Pulsar perf tool can also reproduce the problem if run the test with many consumers If there is no pending batch receive operation for a consumer, no need to trigger the batch timeout task periodically. We can only start the timeout check after adding batch receive request to pending request queue. Remove the lock in MultiTopicsConsumerImpl as apache#10352 does Added new test to verify the batch receive timeout task will not start if no batch receive request (cherry picked from commit a0ccdc9) (cherry picked from commit 6ed4ed0)
congbobo184
pushed a commit
that referenced
this pull request
Nov 10, 2022
…ving pending batch receives requests (#16160) The consumer will apply the default batch receive policy even if the user will not use the batch receive API. https://github.com/apache/pulsar/blob/6704f12104219611164aa2bb5bbdfc929613f1bf/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/BatchReceivePolicy.java#L60-L61 This will consume lots of CPU if the client have many consumers (100k consumers) The Pulsar perf tool can also reproduce the problem if run the test with many consumers If there is no pending batch receive operation for a consumer, no need to trigger the batch timeout task periodically. We can only start the timeout check after adding batch receive request to pending request queue. Remove the lock in MultiTopicsConsumerImpl as #10352 does Added new test to verify the batch receive timeout task will not start if no batch receive request (cherry picked from commit a0ccdc9)
congbobo184
reviewed
Nov 17, 2022
Comment on lines
699
to
692
| public void redeliverUnacknowledgedMessages() { | ||
| lock.writeLock().lock(); | ||
| try { | ||
| internalPinnedExecutor.execute(() -> { | ||
| CONSUMER_EPOCH.incrementAndGet(this); |
Contributor
There was a problem hiding this comment.
use internalPinnedExecutor then CONSUMER_EPOCH.incrementAndGet(this); the epoch will not work
4 tasks
congbobo184
pushed a commit
that referenced
this pull request
Nov 26, 2022
…ving pending batch receives requests (#16160) The consumer will apply the default batch receive policy even if the user will not use the batch receive API. https://github.com/apache/pulsar/blob/6704f12104219611164aa2bb5bbdfc929613f1bf/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/BatchReceivePolicy.java#L60-L61 This will consume lots of CPU if the client have many consumers (100k consumers) The Pulsar perf tool can also reproduce the problem if run the test with many consumers If there is no pending batch receive operation for a consumer, no need to trigger the batch timeout task periodically. We can only start the timeout check after adding batch receive request to pending request queue. Remove the lock in MultiTopicsConsumerImpl as #10352 does Added new test to verify the batch receive timeout task will not start if no batch receive request (cherry picked from commit a0ccdc9)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The consumer will apply the default batch receive policy even if the user will not use the batch receive API.
pulsar/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/BatchReceivePolicy.java
Lines 60 to 61 in 6704f12
This will consume lots of CPU if the client has many consumers (100k consumers)
consumer-cpu-threads.html.txt
The Pulsar perf tool can also reproduce the problem if run the test with many consumers
Modification
If there is no pending batch receive operation for a consumer, no need to trigger the
batch timeout task periodically. We can only start the timeout check after adding batch
receive request to pending request queue.
Remove the lock in MultiTopicsConsumerImpl as #10352 does
Verification
Added new test to verify the batch receive timeout task will not start if no batch
receive request
Documentation
Check the box below or label this PR directly.
Need to update docs?
doc-required(Your PR needs to update docs and you will update later)
doc-not-needed(Please explain why)
doc(Your PR contains doc changes)
doc-complete(Docs have been already added)