[improve][broker] PIP-406: Introduce metrics related to dispatch throttled events

[shibd]

Motivation

Refer to #23945

Modifications

1. Maintain rate limit event fields in AbstractBaseDispatcher.
2. Output these fields when retrieving topic stats and metrics.

Verifying this change

• Add testSubscriptionStatsDispatchThrottled, testBrokerDispatchThrottledMetrics , testTopicDispatchThrottledMetrics to cover it

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

[shibd]

@RobertIndie @lhotari @poorbarcode Could help review this PR, thanks.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 91.11111% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.27%. Comparing base (bbc6224) to head (a7c3f3a).
⚠️ Report is 1269 commits behind head on master.

Files with missing lines	Patch %	Lines
...a/org/apache/pulsar/broker/service/Dispatcher.java	0.00%	6 Missing ⚠️
.../pulsar/broker/service/AbstractBaseDispatcher.java	91.66%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #23946      +/-   ##
============================================
+ Coverage     73.57%   74.27%   +0.69%     
+ Complexity    32624    32087     -537     
============================================
  Files          1877     1863      -14     
  Lines        139502   144403    +4901     
  Branches      15299    16461    +1162     
============================================
+ Hits         102638   107254    +4616     
+ Misses        28908    28688     -220     
- Partials       7956     8461     +505     

Flag	Coverage Δ
inttests	26.73% <42.22%> (+2.14%)	⬆️
systests	23.23% <28.88%> (-1.10%)	⬇️
unittests	73.76% <91.11%> (+0.92%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...rvice/nonpersistent/NonPersistentSubscription.java	57.89% <100.00%> (+4.62%)	⬆️
...ker/service/persistent/PersistentSubscription.java	77.04% <100.00%> (+0.35%)	⬆️
...ker/stats/prometheus/AggregatedNamespaceStats.java	100.00% <100.00%> (ø)
.../stats/prometheus/AggregatedSubscriptionStats.java	100.00% <ø> (ø)
...ker/stats/prometheus/NamespaceStatsAggregator.java	94.33% <100.00%> (+3.93%)	⬆️
...che/pulsar/broker/stats/prometheus/TopicStats.java	98.48% <100.00%> (+4.28%)	⬆️
...mon/policies/data/stats/SubscriptionStatsImpl.java	94.05% <100.00%> (-0.06%)	⬇️
.../pulsar/broker/service/AbstractBaseDispatcher.java	91.70% <91.66%> (-1.60%)	⬇️
...a/org/apache/pulsar/broker/service/Dispatcher.java	50.00% <0.00%> (-10.00%)	⬇️

... and 1055 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[lhotari]

LGTM, good work @shibd

[lhotari]

@shibd It would be great to have this metric available in 4.0.x. Any plans for that?

[shibd]

@shibd It would be great to have this metric available in 4.0.x. Any plans for that?

I think we can cherry-picked to 4.0. Once this PR is merged, I'll submit a separate PR and then vote on it via email.