[fix][broker] Avoid introducing delay when there are delayed messages or marker messages

[lhotari]

Motivation

PR #23226 introduced a change to introduce a delay before calling readMoreEntries in PersistentDispatcherMultipleConsumers classes when no messages were dispatched.
This logic didn't take into account the case when all messages are delayed messages, marker messages or messages from an aborted transaction. In those cases additional delays would be introduced, causing performance regressions.

Modifications

• add a failing test case that simulates dispatching a delayed message that gets filtered
  • no delay should be introduced before the next call to readMoreEntries
• Fix the regression by tracking how many entries were "processed" by consumers in the last round instead of tracking how many entries were dispatched.
  • lastNumberOfEntriesProcessed tracks how many entries were processed by consumers in the last trySendMessagesToConsumers call
  • the number includes also delayed messages, marker messages, aborted txn messages and filtered messages
  • When no messages were processed, the value is 0. This is also an indication that the dispatcher didn't make progress in the last trySendMessagesToConsumers call. Only in those cases should the retry happen after a delay with increasing backoff.

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 95.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 74.56%. Comparing base (bbc6224) to head (957117d).
Report is 599 commits behind head on master.

Files with missing lines	Patch %	Lines
...sistent/PersistentDispatcherMultipleConsumers.java	92.85%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #23343      +/-   ##
============================================
+ Coverage     73.57%   74.56%   +0.99%     
- Complexity    32624    34443    +1819     
============================================
  Files          1877     1934      +57     
  Lines        139502   144990    +5488     
  Branches      15299    15838     +539     
============================================
+ Hits         102638   108116    +5478     
+ Misses        28908    28591     -317     
- Partials       7956     8283     +327     

Flag	Coverage Δ
inttests	27.57% <65.00%> (+2.99%)	⬆️
systests	24.57% <65.00%> (+0.25%)	⬆️
unittests	73.91% <95.00%> (+1.06%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...ersistentStickyKeyDispatcherMultipleConsumers.java	89.15% <100.00%> (+3.51%)	⬆️
...sistent/PersistentDispatcherMultipleConsumers.java	75.50% <92.85%> (+1.17%)	⬆️

... and 603 files with indirect coverage changes

[dao-jun]

lgtm