Skip to content

[fix][client] The partitionedProducer maxPendingMessages always is 0 - #23593

Merged
shibd merged 1 commit into
apache:masterfrom
shibd:fix_max_pending
Nov 13, 2024
Merged

[fix][client] The partitionedProducer maxPendingMessages always is 0#23593
shibd merged 1 commit into
apache:masterfrom
shibd:fix_max_pending

Conversation

@shibd

@shibd shibd commented Nov 12, 2024 •
edited
Loading

Copy link
Copy Markdown
Member

Motivation

After #13344, the default value of maxPendingMessages and maxPendingMessagesAcrossPartitions change to 0.

When creating a producer with the partitioned topic, setting one of the above configurations doesn't take effect; the actual value is always 0.

Because here have a minimum judgment.

pulsar/pulsar-client/src/main/java/org/apache/pulsar/client/impl/PartitionedProducerImpl.java

Lines 86 to 89 in 92448d5

// MaxPendingMessagesAcrossPartitions doesn't support partial partition such as SinglePartition correctly
int maxPendingMessages = Math.min(conf.getMaxPendingMessages(),
conf.getMaxPendingMessagesAcrossPartitions() / numPartitions);
conf.setMaxPendingMessages(maxPendingMessages);

Modifications

  • When setting one of the values, use it directly. Only perform the min check when both values are not zero.

Verifying this change

  • Add PartitionedProducerImplTest.testMaxPendingQueueSize to cover this logic.
  • Add testProducerQueueFullBlockingWithPartitionedTopic test to cover blockQueueFullWithPartitiond case.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@shibd
shibd requested review from HQebupt and poorbarcode November 12, 2024 14:10
@shibd shibd self-assigned this Nov 12, 2024
@github-actions github-actions Bot added the doc-not-needed Your PR changes do not impact docs label Nov 12, 2024
@shibd
shibd requested a review from Technoboy- November 13, 2024 01:39
lhotari
lhotari approved these changes Nov 13, 2024
View reviewed changes

@lhotari lhotari left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, good catch

@shibd
shibd merged commit 0f934f2 into apache:master Nov 13, 2024
shibd added a commit that referenced this pull request Nov 13, 2024
@shibd
[fix][client] The partitionedProducer maxPendingMessages always is 0 (#…
2f6c1a3 
…23593)

(cherry picked from commit 0f934f2)
shibd added a commit that referenced this pull request Nov 13, 2024
@shibd
[fix][client] The partitionedProducer maxPendingMessages always is 0 (#…
8bc6045 
…23593)

(cherry picked from commit 0f934f2)
shibd added a commit that referenced this pull request Nov 13, 2024
@shibd
[fix][client] The partitionedProducer maxPendingMessages always is 0 (#…
d0ea6e2 
…23593)

(cherry picked from commit 0f934f2)
@Technoboy- Technoboy- added this to the 4.1.0 milestone Nov 20, 2024
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request Nov 20, 2024
@shibd @nikhil-ctds
[fix][client] The partitionedProducer maxPendingMessages always is 0 (a…
afd9de5 
…pache#23593)

(cherry picked from commit 0f934f2)
(cherry picked from commit 2f6c1a3)
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Nov 21, 2024
@shibd @srinath-ctds
[fix][client] The partitionedProducer maxPendingMessages always is 0 (a…
bd74b9d 
…pache#23593)

(cherry picked from commit 0f934f2)
(cherry picked from commit 2f6c1a3)
hanmz pushed a commit to hanmz/pulsar that referenced this pull request Feb 12, 2025
@shibd @hanmz
[fix][client] The partitionedProducer maxPendingMessages always is 0 (a…
885a3c8 
…pache#23593)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lhotari lhotari lhotari approved these changes
@poorbarcode poorbarcode poorbarcode approved these changes
@HQebupt HQebupt Awaiting requested review from HQebupt
@Technoboy- Technoboy- Awaiting requested review from Technoboy-

Assignees

@shibd shibd

Labels

cherry-picked/branch-3.0 cherry-picked/branch-3.3 cherry-picked/branch-4.0 doc-not-needed Your PR changes do not impact docs ready-to-test release/3.0.8 release/3.3.3 release/4.0.1 type/bug The PR fixed a bug or issue reported a bug

Projects

None yet

Milestone

4.1.0

Development

Successfully merging this pull request may close these issues.

4 participants

@shibd @lhotari @poorbarcode @Technoboy-