[ZkTracer] feat: enable sampling for parameterized tests#2577
Open
DavePearce wants to merge 9 commits intomainfrom
Open
[ZkTracer] feat: enable sampling for parameterized tests#2577DavePearce wants to merge 9 commits intomainfrom
DavePearce wants to merge 9 commits intomainfrom
Conversation
...zation/src/test/java/net/consensys/linea/zktracer/module/ecdata/ecpairing/EcPairingTest.java
Outdated
Show resolved
Hide resolved
...on/src/test/java/net/consensys/linea/zktracer/instructionprocessing/callTests/Utilities.java
Outdated
Show resolved
Hide resolved
...on/src/test/java/net/consensys/linea/zktracer/instructionprocessing/callTests/Utilities.java
Fixed
Show fixed
Hide fixed
...on/src/test/java/net/consensys/linea/zktracer/instructionprocessing/callTests/Utilities.java
Outdated
Show resolved
Hide resolved
...t/java/net/consensys/linea/zktracer/instructionprocessing/callTests/prc/ecpairing/Tests.java
Outdated
Show resolved
Hide resolved
...on/src/test/java/net/consensys/linea/zktracer/instructionprocessing/callTests/Utilities.java
Fixed
Show fixed
Hide fixed
...on/src/test/java/net/consensys/linea/zktracer/instructionprocessing/callTests/Utilities.java
Show resolved
Hide resolved
...on/src/test/java/net/consensys/linea/zktracer/instructionprocessing/callTests/Utilities.java
Fixed
Show fixed
Hide fixed
...on/src/test/java/net/consensys/linea/zktracer/instructionprocessing/callTests/Utilities.java
Outdated
Show resolved
Hide resolved
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2577 +/- ##
============================================
- Coverage 59.07% 9.70% -49.37%
============================================
Files 454 454
Lines 18530 18530
Branches 2014 2014
============================================
- Hits 10947 1799 -9148
- Misses 6915 16731 +9816
+ Partials 668 0 -668
*This pull request uses carry forward flags. Click here to find out more. 🚀 New features to boost your workflow:
|
...on/src/test/java/net/consensys/linea/zktracer/instructionprocessing/callTests/Utilities.java
Outdated
Show resolved
Hide resolved
...on/src/test/java/net/consensys/linea/zktracer/instructionprocessing/callTests/Utilities.java
Fixed
Show fixed
Hide fixed
e568ba4 to
0ea4088
Compare
...hmetization/src/test/java/net/consensys/linea/zktracer/exceptions/OutOfGasExceptionTest.java
Outdated
Show resolved
Hide resolved
742d9f2 to
50f5ef6
Compare
...src/test/java/net/consensys/linea/zktracer/precompiles/LowGasStipendPrecompileCallTests.java
Outdated
Show resolved
Hide resolved
...rithmetization/src/test/java/net/consensys/linea/zktracer/module/mxp/MxpxThresholdTests.java
Outdated
Show resolved
Hide resolved
...on/src/test/java/net/consensys/linea/zktracer/instructionprocessing/callTests/Utilities.java
Fixed
Show fixed
Hide fixed
b9e190d to
72f68df
Compare
...on/src/test/java/net/consensys/linea/zktracer/instructionprocessing/callTests/Utilities.java
Fixed
Show fixed
Hide fixed
72f68df to
afd2d4d
Compare
...on/src/test/java/net/consensys/linea/zktracer/instructionprocessing/callTests/Utilities.java
Show resolved
Hide resolved
a6b090f to
f62d2f6
Compare
...on/src/test/java/net/consensys/linea/zktracer/instructionprocessing/callTests/Utilities.java
Show resolved
Hide resolved
...on/src/test/java/net/consensys/linea/zktracer/instructionprocessing/callTests/Utilities.java
Outdated
Show resolved
Hide resolved
...on/src/test/java/net/consensys/linea/zktracer/instructionprocessing/callTests/Utilities.java
Show resolved
Hide resolved
...on/src/test/java/net/consensys/linea/zktracer/instructionprocessing/callTests/Utilities.java
Outdated
Show resolved
Hide resolved
543e210 to
a7b234e
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
...on/src/test/java/net/consensys/linea/zktracer/instructionprocessing/callTests/Utilities.java
Outdated
Show resolved
Hide resolved
c1419ce to
08d8f5b
Compare
This adds sampling using a seed based on the day of the month to all parameterized tests. Specifically, sampling employs a cap to ensure that at most N individual tests are run for each parameterized test. Here, N is set by default to 100, but can be overriden as needed. This enables the KOALABEAR_16 field for unit & nightly tests in the CI pipeline, but not for the blockchain reference tests.
08d8f5b to
94f3f47
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

This adds sampling using a seed based on the day of the month to all parameterized tests. Specifically, sampling employs a cap to ensure that at most N individual tests are run for each parameterized test. Here, N is set by default to 100, but can be overriden as needed.
This also enablesKOALABEAR_16on the unit & replay tests (which is the primary motivation for using sampling).This PR implements issue(s) #2576
Note
Medium Risk
Moderate risk because it changes test execution coverage and determinism across CI by sampling parameterized inputs; failures may be harder to detect or reproduce if sampling is misconfigured.
Overview
Enables commit-hash-seeded sampling across a large set of parameterized tests by routing their
@MethodSourceargument lists throughUtilities.randomSampleByCurrentCommitHash(...), reducing the number of executed cases per run.Adds a global sampling ratio
PARAMETERIZED_TESTS_SAMPLE_SIZE(env-controlled, default0.2, minimum 15 cases) and wires CI to set this value in the reusable unit test workflow. Removes several per-test hardcoded sample-size knobs and refactors some generators to returnList<Arguments>so sampling can be applied consistently; also drops a few ad-hoc nightly/5% shuffle-based samplers in favor of the unified mechanism.Written by Cursor Bugbot for commit 94f3f47. This will update automatically on new commits. Configure here.