Skip to content

[improve][test] Fix flaky test SimpleProducerConsumerStatTest#testMsgRateExpired - #20629

Merged
tisonkun merged 2 commits into
apache:masterfrom
massakam:fix-flaky-test
Jun 30, 2023
Merged

[improve][test] Fix flaky test SimpleProducerConsumerStatTest#testMsgRateExpired#20629
tisonkun merged 2 commits into
apache:masterfrom
massakam:fix-flaky-test

Conversation

@massakam

@massakam massakam commented Jun 21, 2023

Copy link
Copy Markdown
Contributor

Fixes #20615

Motivation

SimpleProducerConsumerStatTest#testMsgRateExpired seems flakey, so I'll try to improve it.

Modifications

Before this fix, we checked if the messages expired by referring to the msgRateExpired value in the subscription stats. However, this value resets to 0 over time, so we instead refer to the totalMsgExpired value, which never resets.

Additionally, we get the stats directly from the instances in the broker server and not from the admin API.

Verifying this change

  • Make sure that the change passes the CI checks.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

@massakam massakam added type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages area/test doc-not-needed Your PR changes do not impact docs ready-to-test labels Jun 21, 2023
@massakam massakam added this to the 3.1.0 milestone Jun 21, 2023
@massakam massakam self-assigned this Jun 21, 2023
Awaitility.await().ignoreExceptions().timeout(10, TimeUnit.SECONDS)
.until(() -> pulsar.getBrokerService().getTopicStats().get(topicName).getSubscriptions().get(subName).getTotalMsgExpired() > 0);

Thread.sleep(2000);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove this sleep

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Technoboy- Removed. Thanks.

@codecov-commenter

codecov-commenter commented Jun 29, 2023

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.07%. Comparing base (2b01f83) to head (04261c8).
⚠️ Report is 2376 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@              Coverage Diff              @@
##             master   #20629       +/-   ##
=============================================
+ Coverage     33.58%   73.07%   +39.48%     
+ Complexity    12127     3701     -8426     
=============================================
  Files          1613     1867      +254     
  Lines        126241   138684    +12443     
  Branches      13770    15240     +1470     
=============================================
+ Hits          42396   101338    +58942     
+ Misses        78331    29310    -49021     
- Partials       5514     8036     +2522     
Flag Coverage Δ
inttests 24.15% <ø> (-0.04%) ⬇️
systests 24.97% <ø> (?)
unittests 72.36% <ø> (+40.32%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1521 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@tisonkun tisonkun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@tisonkun
tisonkun merged commit 9526382 into apache:master Jun 30, 2023
@massakam
massakam deleted the fix-flaky-test branch June 30, 2023 11:35
lhotari pushed a commit that referenced this pull request Jun 19, 2025
nodece pushed a commit to nodece/pulsar that referenced this pull request Jun 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/test cherry-picked/branch-3.0 doc-not-needed Your PR changes do not impact docs ready-to-test release/3.0.13 type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky-test: SimpleProducerConsumerStatTest.testMsgRateExpired

7 participants