Skip to content

[ZkTracer] feat: enable sampling for parameterized tests#2577

Open
DavePearce wants to merge 9 commits intomainfrom
2576-zktracer-feat-enable-koalabear_16-on-unit-replay-tests
Open

[ZkTracer] feat: enable sampling for parameterized tests#2577
DavePearce wants to merge 9 commits intomainfrom
2576-zktracer-feat-enable-koalabear_16-on-unit-replay-tests

Conversation

@DavePearce
Copy link
Contributor

@DavePearce DavePearce commented Mar 10, 2026

This adds sampling using a seed based on the day of the month to all parameterized tests. Specifically, sampling employs a cap to ensure that at most N individual tests are run for each parameterized test. Here, N is set by default to 100, but can be overriden as needed. This also enables KOALABEAR_16 on the unit & replay tests (which is the primary motivation for using sampling).

This PR implements issue(s) #2576


Note

Medium Risk
Moderate risk because it changes test execution coverage and determinism across CI by sampling parameterized inputs; failures may be harder to detect or reproduce if sampling is misconfigured.

Overview
Enables commit-hash-seeded sampling across a large set of parameterized tests by routing their @MethodSource argument lists through Utilities.randomSampleByCurrentCommitHash(...), reducing the number of executed cases per run.

Adds a global sampling ratio PARAMETERIZED_TESTS_SAMPLE_SIZE (env-controlled, default 0.2, minimum 15 cases) and wires CI to set this value in the reusable unit test workflow. Removes several per-test hardcoded sample-size knobs and refactors some generators to return List<Arguments> so sampling can be applied consistently; also drops a few ad-hoc nightly/5% shuffle-based samplers in favor of the unified mechanism.

Written by Cursor Bugbot for commit 94f3f47. This will update automatically on new commits. Configure here.

@DavePearce DavePearce linked an issue Mar 10, 2026 that may be closed by this pull request
@codecov-commenter
Copy link

codecov-commenter commented Mar 11, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 9.70%. Comparing base (bd97b6c) to head (08d8f5b).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #2577       +/-   ##
============================================
- Coverage     59.07%   9.70%   -49.37%     
============================================
  Files           454     454               
  Lines         18530   18530               
  Branches       2014    2014               
============================================
- Hits          10947    1799     -9148     
- Misses         6915   16731     +9816     
+ Partials        668       0      -668     
Flag Coverage Δ *Carryforward flag
hardhat 96.53% <ø> (ø) Carriedforward from a7b234e
kotlin 0.10% <ø> (-54.83%) ⬇️

*This pull request uses carry forward flags. Click here to find out more.
see 294 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@letypequividelespoubelles letypequividelespoubelles added Arithmetization Arithmetization team is in charge or involved in this task Test Adding or enhancing test coverage labels Mar 13, 2026
@DavePearce DavePearce force-pushed the 2576-zktracer-feat-enable-koalabear_16-on-unit-replay-tests branch from e568ba4 to 0ea4088 Compare March 19, 2026 03:29
@DavePearce DavePearce force-pushed the 2576-zktracer-feat-enable-koalabear_16-on-unit-replay-tests branch from 742d9f2 to 50f5ef6 Compare March 19, 2026 03:39
@DavePearce DavePearce force-pushed the 2576-zktracer-feat-enable-koalabear_16-on-unit-replay-tests branch from b9e190d to 72f68df Compare March 19, 2026 21:18
@DavePearce DavePearce force-pushed the 2576-zktracer-feat-enable-koalabear_16-on-unit-replay-tests branch from 72f68df to afd2d4d Compare March 19, 2026 22:08
@DavePearce DavePearce force-pushed the 2576-zktracer-feat-enable-koalabear_16-on-unit-replay-tests branch 3 times, most recently from a6b090f to f62d2f6 Compare March 20, 2026 01:06
@DavePearce DavePearce force-pushed the 2576-zktracer-feat-enable-koalabear_16-on-unit-replay-tests branch from 543e210 to a7b234e Compare March 20, 2026 01:29
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

@DavePearce DavePearce force-pushed the 2576-zktracer-feat-enable-koalabear_16-on-unit-replay-tests branch 3 times, most recently from c1419ce to 08d8f5b Compare March 20, 2026 04:52
This adds sampling using a seed based on the day of the month to all
parameterized tests.  Specifically, sampling employs a cap to ensure
that at most N individual tests are run for each parameterized test.
Here, N is set by default to 100, but can be overriden as needed.

This enables the KOALABEAR_16 field for unit & nightly tests in the CI
pipeline, but not for the blockchain reference tests.
@DavePearce DavePearce force-pushed the 2576-zktracer-feat-enable-koalabear_16-on-unit-replay-tests branch from 08d8f5b to 94f3f47 Compare March 20, 2026 07:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Arithmetization Arithmetization team is in charge or involved in this task Test Adding or enhancing test coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ZkTracer] feat: enable KOALABEAR_16 on Unit & Replay Tests

3 participants