Skip to content

[draft][fix][build] More Heap for Maven Surefire Tests - #20774

Draft
dave2wave wants to merge 4 commits into
apache:masterfrom
dave2wave:more-surefire
Draft

[draft][fix][build] More Heap for Maven Surefire Tests#20774
dave2wave wants to merge 4 commits into
apache:masterfrom
dave2wave:more-surefire

Conversation

@dave2wave

Copy link
Copy Markdown
Member

Motivation

Builds on 3.0 and Master branches are failing during tests. SOme of the tests use up the heap and OOM.

Modifications

Changes surefire to -Xms1532M

Verifying this change

  • Make sure that the change passes the CI checks.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

@dave2wave
dave2wave requested a review from lhotari July 11, 2023 00:36
@github-actions github-actions Bot added the doc-not-needed Your PR changes do not impact docs label Jul 11, 2023
@dave2wave
dave2wave requested review from lhotari, nicoloboschi and tisonkun and removed request for lhotari and tisonkun July 11, 2023 00:36

@tisonkun tisonkun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have some profile to improve the tests in the other side? The total CI resources are limited IIRC.

@tisonkun

Copy link
Copy Markdown
Member

/pulsarbot run-failure-checks

@lhotari

lhotari commented Jul 11, 2023

Copy link
Copy Markdown
Member

Tests running into OOM is a sign of a resource leak. When the heap is increased, the possible resource leak would be tolerated, but the problem would remain.
For investigating test OOMs, I have added a feature which collects the heap dump that gets created at OOM time. Which build failed at OOM?

@JooHyukKim

JooHyukKim commented Jul 11, 2023

Copy link
Copy Markdown
Contributor

For investigating test OOMs, I have added a feature which collects the heap dump that gets created at OOM time. Which build failed at OOM?

If it's okay, can we have reference(link to) the feature that is being refered to? @lhotari

@JooHyukKim

Copy link
Copy Markdown
Contributor

This issue might bring us back to thread of @dave2wave in the mailing list

Comment thread pom.xml
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>${testJacocoAgentArgument} -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=${testHeapDumpPath} -XX:+ExitOnOutOfMemoryError -Xmx1G -XX:+UseZGC
<argLine>${testJacocoAgentArgument} -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=${testHeapDumpPath} -XX:+ExitOnOutOfMemoryError -Xmx1532M -XX:+UseZGC

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of changing this, make the parameter configurable so that this can be increased if someone is running all tests together. For Pulsar CI, it's better to keep this value at 1G.

So what you could do is adding a separate maven property "testMaxHeapSize" so that it can be passed from the command line with -DtestMaxHeapSize=1532M. Makes sense?

@lhotari

lhotari commented Jul 12, 2023

Copy link
Copy Markdown
Member

btw. One of the recently fixed resource leaks in tests is #20778 by @Technoboy-

@lhotari

lhotari commented Jul 13, 2023

Copy link
Copy Markdown
Member

#20799 fixes a major resource leak in tests.

@dave2wave

Copy link
Copy Markdown
Member Author

The two fixes are definite improvements. There are still a few issues to look into.

@dave2wave
dave2wave marked this pull request as draft July 16, 2023 19:15
@dave2wave dave2wave changed the title [fix][build] More Heap for Maven Surefire Tests [draft][fix][build] More Heap for Maven Surefire Tests Jul 16, 2023
@Technoboy- Technoboy- added this to the 3.2.0 milestone Jul 31, 2023
@github-actions

Copy link
Copy Markdown

The pr had no activity for 30 days, mark with Stale label.

@github-actions github-actions Bot added the Stale label Aug 31, 2023
@Technoboy- Technoboy- modified the milestones: 3.2.0, 3.3.0 Dec 22, 2023
@coderzc coderzc modified the milestones: 3.3.0, 3.4.0 May 8, 2024
@lhotari lhotari modified the milestones: 4.0.0, 4.1.0 Oct 11, 2024
@lhotari lhotari removed this from the 4.1.0 milestone Jan 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-not-needed Your PR changes do not impact docs ready-to-test Stale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants