Skip to content

Don't use Math.random in tests for reproducability #36241

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Dec 5, 2018
Merged

Don't use Math.random in tests for reproducability #36241

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
286ff22
Make sure test don't use Math.random for reproducability
Dec 4, 2018
a6462ad
iter
Dec 5, 2018
e67b972
Merge branch 'master' into fix-reproducable-randomness-tests
Dec 5, 2018
7a2b596
Add Math.random(() to forbidden-apis for tests
Dec 5, 2018
4af81db
Merge branch 'master' into fix-reproducable-randomness-tests
Dec 5, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions buildSrc/src/main/resources/forbidden/es-test-signatures.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,5 @@ org.apache.lucene.util.LuceneTestCase$Nightly @ We don't run nightly tests at th
com.carrotsearch.randomizedtesting.annotations.Nightly @ We don't run nightly tests at this point!

org.junit.Test @defaultMessage Just name your test method testFooBar

java.lang.Math#random() @ Use one of the various randomization methods from LuceneTestCase or ESTestCase for reproducibility
21 changes: 6 additions & 15 deletions server/src/test/java/org/elasticsearch/common/util/ArrayUtilsTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,15 +30,15 @@
public class ArrayUtilsTests extends ESTestCase {
public void testBinarySearch() throws Exception {
for (int j = 0; j < 100; j++) {
int index = Math.min(randomInt(0, 10), 9);
double tolerance = Math.random() * 0.01;
double lookForValue = randomFreq(0.9) ? -1 : Double.NaN; // sometimes we'll look for NaN
int index = randomIntBetween(0, 9);
double tolerance = randomDoubleBetween(0, 0.01, true);
double lookForValue = frequently() ? -1 : Double.NaN; // sometimes we'll look for NaN
double[] array = new double[10];
for (int i = 0; i < array.length; i++) {
double value;
if (randomFreq(0.9)) {
value = Math.random() * 10;
array[i] = value + ((randomFreq(0.5) ? 1 : -1) * Math.random() * tolerance);
if (frequently()) {
value = randomDoubleBetween(0, 9, true);
Copy link
Contributor

@jpountz jpountz Dec 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wasn't the test generating doubles between 0 and 10 before? (it might not matter)

Copy link
Member Author

@cbuescher cbuescher Dec 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Math.random() -> "a pseudorandom {@code double} greater than or equal to {@code 0.0} and less than {@code 1.0}", so I think 10 wasn't included. But yes, I agree it probably doesn't matter at all ;-)

array[i] = value + ((randomBoolean() ? 1 : -1) * randomDouble() * tolerance);

} else { // sometimes we'll have NaN in the array
value = Double.NaN;
Expand Down Expand Up @@ -73,15 +73,6 @@ public void testBinarySearch() throws Exception {
}
}

private boolean randomFreq(double freq) {
return Math.random() < freq;
}

private int randomInt(int min, int max) {
int delta = (int) (Math.random() * (max - min));
return min + delta;
}

public void testConcat() {
assertArrayEquals(new String[]{"a", "b", "c", "d"}, ArrayUtils.concat(new String[]{"a", "b"}, new String[]{"c", "d"}));
int firstSize = randomIntBetween(0, 10);
Expand Down
2 changes: 1 addition & 1 deletion server/src/test/java/org/elasticsearch/search/aggregations/support/PathTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ Tokens add(String name) {
}

Tokens add(String name, String key) {
if (Math.random() > 0.5) {
if (randomBoolean()) {
tokens.add(new AggregationPath.PathElement(name + "." + key, name, key));
} else {
tokens.add(new AggregationPath.PathElement(name + "[" + key + "]", name, key));
Expand Down