[CI] Failures in ClusterDisruptionIT#testAckedIndexing

[cbuescher]

Build: https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+6.x+intake/356/console

Doesn't reproduce locally on 6.x so far for me:

./gradlew :server:integTest \
  -Dtests.seed=D8D753C5046576ED \
  -Dtests.class=org.elasticsearch.discovery.ClusterDisruptionIT \
  -Dtests.method="testAckedIndexing" \
  -Dtests.security.manager=true \
  -Dtests.locale=it \
  -Dtests.timezone=America/El_Salvador \
  -Dcompiler.java=11 \
  -Druntime.java=8

java.lang.AssertionError: AcknowledgedResponse failed - not acked
Expected: <true>
     but: was <false>
	at __randomizedtesting.SeedInfo.seed([D8D753C5046576ED:5216E736581890A6]:0)
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
	at org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertAcked(ElasticsearchAssertions.java:106)
	at org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertAcked(ElasticsearchAssertions.java:110)
	at org.elasticsearch.test.TestCluster.wipeIndices(TestCluster.java:142)
	at org.elasticsearch.test.TestCluster.wipe(TestCluster.java:79)
	at org.elasticsearch.test.ESIntegTestCase.afterInternal(ESIntegTestCase.java:586)
	at org.elasticsearch.test.ESIntegTestCase.cleanUpCluster(ESIntegTestCase.java:2199)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748) 

[elasticmachine]

Pinging @elastic/es-distributed

[cbuescher]

Also found this on 6.5 which looks related:

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+6.5+multijob-unix-compatibility/os=ubuntu&&virtual/57/console

[cbuescher]

Another older one (Nov. 7th): https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+6.x+multijob-unix-compatibility/os=centos/46/console

[astefan]

A newer one: https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+6.x+multijob-unix-compatibility/os=fedora/113/console
Didn't manage to reproduce locally in 50 attempts.

gradlew :server:integTest -Dtests.seed=19F2BE5C71D86152 -Dtests.class=org.elasticsearch.discovery.ClusterDisruptionIT -Dtests.method="testAckedIndexing" -Dtests.security.manager=true -Dtests.locale=pt -Dtests.timezone=America/Argentina/Mendoza -Dcompiler.java=11 -Druntime.java=8

09:04:09  FAILURE 5.98s J6 | ClusterDisruptionIT.testAckedIndexing <<< FAILURES!
09:04:09     > Throwable #1: java.lang.AssertionError: AcknowledgedResponse failed - not acked
09:04:09     > Expected: <true>
09:04:09     >      but: was <false>
09:04:09     > 	at __randomizedtesting.SeedInfo.seed([19F2BE5C71D86152:93330AAF2DA58719]:0)
09:04:09     > 	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
09:04:09     > 	at org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertAcked(ElasticsearchAssertions.java:106)
09:04:09     > 	at org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertAcked(ElasticsearchAssertions.java:110)
09:04:09     > 	at org.elasticsearch.test.TestCluster.wipeIndices(TestCluster.java:142)
09:04:09     > 	at org.elasticsearch.test.TestCluster.wipe(TestCluster.java:79)
09:04:09     > 	at org.elasticsearch.test.ESIntegTestCase.afterInternal(ESIntegTestCase.java:586)
09:04:09     > 	at org.elasticsearch.test.ESIntegTestCase.cleanUpCluster(ESIntegTestCase.java:2199)
09:04:09     > 	at java.lang.Thread.run(Thread.java:748)

[dnhatn]

Logs of these failures are gone. Let's reopen if this fails again.