ml.integration.RegressionIT failures on CI

[gwbrown]

Hit this on a PR CI run, from build-stats it looks like this has failed a few times - first on the 9th in another PR build, then once on master, then again today.

Build Scan

Each time the failure occurs, a number of tests in this class fail. Here is an example of one test failure, the others are very similar, all failing at MlNativeDataFrameAnalyticsIntegTestCase.lambda$assertThatAuditMessagesMatch$3(MlNativeDataFrameAnalyticsIntegTestCase.java:199):

org.elasticsearch.xpack.ml.integration.RegressionIT > testWithOnlyTrainingRowsAndTrainingPercentIsHundred FAILED
java.lang.AssertionError: Messages: [Created analytics with analysis type [regression], Estimated memory usage for this analytics to be [3.3mb], Starting analytics on node [{integTest-0}{0-JhnP-KSvSoyOJyic4pdA}{MktKZojhSfeZCaVr_dWnrA}{127.0.0.1}{127.0.0.1:41272}{dilm}{testattr=test, ml.machine_memory=101204291584, ml.max_open_jobs=20, xpack.installed=true}], Started analytics, Creating destination index [regression_only_training_data_and_training_percent_is_100_source_index_results], Finished reindexing to destination index [regression_only_training_data_and_training_percent_is_100_source_index_results], Stored trained model with id [regression_only_training_data_and_training_percent_is_100-1571171868834], Finished analysis]
Expected: a collection with size <7>
but: collection size was <8>
at __randomizedtesting.SeedInfo.seed([31EC486CC8A422:5829A18793B7BD34]:0)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
at org.junit.Assert.assertThat(Assert.java:956)
at org.elasticsearch.xpack.ml.integration.MlNativeDataFrameAnalyticsIntegTestCase.lambda$assertThatAuditMessagesMatch$3(MlNativeDataFrameAnalyticsIntegTestCase.java:199)
at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:867)
at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:840)
at org.elasticsearch.xpack.ml.integration.MlNativeDataFrameAnalyticsIntegTestCase.assertThatAuditMessagesMatch(MlNativeDataFrameAnalyticsIntegTestCase.java:196)
at org.elasticsearch.xpack.ml.integration.RegressionIT.testWithOnlyTrainingRowsAndTrainingPercentIsHundred(RegressionIT.java:162)

Reproduce line (does not reproduce locally):

./gradlew ':x-pack:plugin:ml:qa:native-multi-node-tests:integTestRunner' --tests "org.elasticsearch.xpack.ml.integration.RegressionIT.testWithOnlyTrainingRowsAndTrainingPercentIsHundred" -Dtests.seed=31EC486CC8A422 -Dtests.security.manager=true -Dtests.locale=lo-LA -Dtests.timezone=Asia/Dili -Dcompiler.java=12 -Druntime.java=11

[elasticmachine]

Pinging @elastic/ml-core (:ml)

[gwbrown]

I think this is a duplicate of #48085?

[droberts195]

I think this is a duplicate of #48085?

Yes, since this issue was raised the whole RegressionIT suite has been muted with a link to #48085. And #48084 should fix the problem.