[improve][ci] Detect test thread leaks in CI builds and add tooling for resource leak investigation

[lhotari]

Motivation

It should be ensured in each test that resources created by the test are properly cleaned up. Failing to do so can lead to memory leaks and, in some instances, unnecessary CPU consumption. These issues can, in turn, slow down test execution, increase Pulsar CI build durations, and cause flakiness. A significant source of memory leaks in Pulsar tests stems from thread leaks. This PR aims to add reporting and tooling to detect thread leaks in tests.

Issues / requirements:

• Thread leaks are hard to detect, leading to many tests leaking threads.
• The earlier solution ([Tests] Add TestNG listeners for resource cleanup, thread leak detection and "fail fast" #10195) had many false positives as it didn't account for waiting threads.
• Memory leaks in tests can sometimes be unrelated to thread leaks and there is a need for tooling for debugging test memory leaks.

Proposed Solutions:

• Enhance thread leak detection so that it becomes useful and has less false positives.
• Introduce warnings in Pulsar CI for classes causing thread leaks, allowing for early detection.
• Offer an option in Pulsar CI to collect heap dumps or histograms to identify memory leak sources.

Heap dump tooling for tests:

This PR includes a helper class that enables programmatic creation of heap dumps during tests. A heap dump offers a snapshot of the heap state, which can be invaluable for debugging certain types of issues. Tools such as Eclipse MAT can be used to inspect heap dumps. Eclipse MAT offers a query language to extract summaries. Moreover, the Calcite plugin for Eclipse MAT enables querying and summarizing heap dump information using SQL, which can be extremely handy at times.

Modifications

1. ThreadLeakDetectorListener Improvements:

   • Refine the detection algorithm to account for waiting threads.
   • Exclude known thread pools not spawned by the tests.
   • Introduce file-based reporting.

2. Pulsar CI Reporting Enhancements:

   • Leverage files generated by ThreadLeakDetectorListener for improved reporting.

3. TraceTestResourceCleanupListener Introduction:

   • Track test resource cleanup by generating thread dumps, heap histograms, and heap dumps before the TestNG JVM exits.
   • Add integration to Pulsar CI so that heap dump collection can be selected for manually triggered builds.

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

[codecov-commenter]

Codecov Report

❌ Patch coverage is 0% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 73.22%. Comparing base (e55de39) to head (b121aae).
⚠️ Report is 2044 commits behind head on master.

Files with missing lines	Patch %	Lines
.../org/apache/pulsar/common/util/ThreadDumpUtil.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #21450      +/-   ##
============================================
+ Coverage     73.14%   73.22%   +0.07%     
- Complexity    32570    32583      +13     
============================================
  Files          1890     1890              
  Lines        140357   140357              
  Branches      15425    15425              
============================================
+ Hits         102663   102774     +111     
+ Misses        29595    29499      -96     
+ Partials       8099     8084      -15     

Flag	Coverage Δ
inttests	24.18% <0.00%> (?)
systests	24.69% <0.00%> (-0.03%)	⬇️
unittests	72.52% <0.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
.../org/apache/pulsar/common/util/ThreadDumpUtil.java	59.45% <0.00%> (ø)

... and 92 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[eolivelli]

Great stuff

Lgtm