This repository was archived by the owner on Jan 23, 2023. It is now read-only.
Increase sample size of random number generation - #38728
Merged
Merged
Conversation
bartonjs
approved these changes
Jun 20, 2019
|
Hello @bartonjs! Because this pull request has the Do note that I've been instructed to only help merge pull requests of this repository that have been opened for at least 12 minutes, a condition that will be fulfilled in about 7 minutes. No worries though, I will be back when the time is right! 😉 p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (
|
stephentoub
approved these changes
Jun 20, 2019
Member
Author
|
Looks like all of the failures are in |
Member
|
Opened #38734 to track the faiures. OK to merge. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This increases the sample size of the random number generator tests to improve stability of the tests. The sample size increased by an order of a magnitude, but this only marginally increases test execution time. It adds about half a second to the entire test suite for me, though that is on a fairly strong desktop. I don't know how well this will perform on the ARM64 leg.
@bartonjs I looked at other approaches like variance/stddev but had similar issues, which is that the population of data is relatively small to make any strong conclusions about randomness. I'll continue to look at ways to improve these tests overall. However, in the mean time this simpler change will improve the reliability of these tests so folks can get green CI runs.
Fix #38402