Skip to content

[fix][broker] Move pending acks cleanup to selected mark-delete callbacks - #25592

Merged
nodece merged 5 commits into
apache:masterfrom
nodece:move-pending-acks-cleanup
Apr 29, 2026
Merged

[fix][broker] Move pending acks cleanup to selected mark-delete callbacks#25592
nodece merged 5 commits into
apache:masterfrom
nodece:move-pending-acks-cleanup

Conversation

@nodece

@nodece nodece commented Apr 28, 2026

Copy link
Copy Markdown
Member

Motivation

Pending-ack cleanup was previously executed in the readMoreEntries dispatch path. This introduced unnecessary overhead in a hot path and also risked concurrent access to the same pending-ack entries from both dispatch and ack threads.

Cleanup is only required when messages are removed through mark-delete advancing operations such as expiry, skip, or clear-backlog. Ack-triggered mark-delete operations are much more frequent and should avoid carrying additional cleanup work.

Modifications

  • Introduced atomic remaining-unacked tracking for both non-batch and batch messages.

    • PendingAcksMap now stores and updates the actual remainingUnacked count instead of relying on batch size–based accounting.
    • Added atomic helper methods such as updateRemainingUnacked and removePendingAckAndGet to simplify and centralize unacked-message accounting.
    • Removed legacy logic that recalculated unacked counts from cursor state or batch size.
  • Refactored pending-ack cleanup to leverage the existing markDeletePositionMoveForward hook.

    • Removed pending-ack cleanup from the readMoreEntries dispatch path.

    • Cleanup of pending acks and redelivery state is now triggered only after mark-delete advancement operations such as:

      • message expiry
      • skip entries
      • clear backlog
    • Ack-triggered paths do not invoke markDeletePositionMoveForward; they continue using afterAckMessages to avoid additional overhead on the high-frequency ack path.

  • Updated pending-ack iteration and removal paths to consistently use the maintained remainingUnacked count directly, improving correctness and concurrency handling for partial batch acknowledgments.

Comment thread pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Dispatcher.java Outdated

@lhotari lhotari left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, follow up issue #25613

@nodece
nodece merged commit 138595f into apache:master Apr 29, 2026
82 of 84 checks passed
@nodece
nodece deleted the move-pending-acks-cleanup branch April 29, 2026 15:19
@lhotari lhotari added this to the 4.3.0 milestone Apr 30, 2026
poorbarcode pushed a commit to poorbarcode/pulsar that referenced this pull request May 6, 2026
lhotari pushed a commit that referenced this pull request May 13, 2026
nodece added a commit to ascentstream/pulsar that referenced this pull request May 27, 2026
…acks (apache#25592)

(cherry-pick from commit 138595f)

Signed-off-by: Zixuan Liu <nodeces@gmail.com>
priyanshu-ctds pushed a commit to datastax/pulsar that referenced this pull request Jun 9, 2026
…acks (apache#25592)

(cherry-pick from commit 138595f)

(cherry picked from commit 4330c2f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants