CoordinatorTests testAckListenerReceivesNacksIfPublicationTimesOut failure

[davidkyle]

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+7.x+multijob+fast+part1/95/console
https://gradle.com/s/i5matu3ca3yl6

java.lang.AssertionError: leaders
Expected: not an empty collection
     but: was <[]>
	at __randomizedtesting.SeedInfo.seed([1D931938A0E635D1:97D103FE47BECADC]:0)
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
	at org.junit.Assert.assertThat(Assert.java:956)
	at org.elasticsearch.cluster.coordination.AbstractCoordinatorTestCase$Cluster.getAnyLeader(AbstractCoordinatorTestCase.java:605)
	at org.elasticsearch.cluster.coordination.AbstractCoordinatorTestCase$Cluster.stabilise(AbstractCoordinatorTestCase.java:475)
	at org.elasticsearch.cluster.coordination.AbstractCoordinatorTestCase$Cluster.stabilise(AbstractCoordinatorTestCase.java:463)
	at org.elasticsearch.cluster.coordination.CoordinatorTests.testAckListenerReceivesNacksIfPublicationTimesOut(CoordinatorTests.java:622)

This actually reproduces on 7.x

./gradlew :server:test --tests "org.elasticsearch.cluster.coordination.CoordinatorTests.testAckListenerReceivesNacksIfPublicationTimesOut" \
  -Dtests.seed=1D931938A0E635D1 \
  -Dtests.security.manager=true \
  -Dtests.locale=fr-LU \
  -Dtests.timezone=America/Mazatlan \
  -Dcompiler.java=12 \
  -Druntime.java=8

[elasticmachine]

Pinging @elastic/es-distributed

[DaveCTurner]

"actually reproduces" lol. It's almost like we tried to make these tests deterministic 😁

I opened #44077 which should fix this.

[davidkyle]

Similar failure on master

  2> java.lang.AssertionError: leaders
    Expected: not an empty collection
         but: was <[]>
        at __randomizedtesting.SeedInfo.seed([C6A4B2315799684B:DCC78B49D0215FE1]:0)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
        at org.junit.Assert.assertThat(Assert.java:956)
        at org.elasticsearch.cluster.coordination.AbstractCoordinatorTestCase$Cluster.getAnyLeader(AbstractCoordinatorTestCase.java:605)
        at org.elasticsearch.cluster.coordination.AbstractCoordinatorTestCase$Cluster.stabilise(AbstractCoordinatorTestCase.java:475)
        at org.elasticsearch.cluster.coordination.AbstractCoordinatorTestCase$Cluster.stabilise(AbstractCoordinatorTestCase.java:463)
        at org.elasticsearch.cluster.coordination.CoordinatorTests.testCanShrinkFromFiveNodesToThree(CoordinatorTests.java:274)

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+intake/542/console
https://scans.gradle.com/s/rupwqezqp6kd6

And another win for reproducibility!

/gradlew :server:test --tests "org.elasticsearch.cluster.coordination.CoordinatorTests.testCanShrinkFromFiveNodesToThree" \
  -Dtests.seed=C6A4B2315799684B \
  -Dtests.security.manager=true \
  -Dtests.locale=fr-GF \
  -Dtests.timezone=Pacific/Auckland \
  -Dcompiler.java=12 \
  -Druntime.java=11

[DaveCTurner]

Yep #44077 fixes that one too. The too-short timeout is systemic rather than a problem with any particular test so if you're muting things you'll be best off muting all suites that derive from AbstractCoordinatorTestCase.