[CI] BlobStoreIncrementalityIT.testIncrementalBehaviorOnPrimaryFailover failing

[mark-vieira]

This looks to be a recently added test. It's failed twice now in 7.x, not sure if whatever the problem is also exists in master. The below failure reproduced locally for me:

org.elasticsearch.snapshots.BlobStoreIncrementalityIT > testIncrementalBehaviorOnPrimaryFailover FAILED
    java.lang.AssertionError: 
    Expected: is <4>
         but: was <0>
        at __randomizedtesting.SeedInfo.seed([33E415A7F24AEA0F:F57F696CCE8F8F00]:0)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
        at org.junit.Assert.assertThat(Assert.java:956)
        at org.junit.Assert.assertThat(Assert.java:923)
        at org.elasticsearch.snapshots.BlobStoreIncrementalityIT.assertTwoIdenticalShardSnapshots(BlobStoreIncrementalityIT.java:199)
        at org.elasticsearch.snapshots.BlobStoreIncrementalityIT.testIncrementalBehaviorOnPrimaryFailover(BlobStoreIncrementalityIT.java:127)
REPRODUCE WITH: ./gradlew ':server:integTest' --tests "org.elasticsearch.snapshots.BlobStoreIncrementalityIT.testIncrementalBehaviorOnPrimaryFailover" -Dtests.seed=33E415A7F24AEA0F -Dtests.security.manager=true -Dtests.locale=es-HN -Dtests.timezone=Pacific/Norfolk -Dcompiler.java=13

[elasticmachine]

Pinging @elastic/es-distributed (:Distributed/Snapshot/Restore)

[mark-vieira]

@original-brownbear since you authored this test do you mind taking a look?

[mark-vieira]

I've muted this in 7.x with be1b34c

[jasontedor]

I muted this in master with b56b9fb.

[mark-vieira]

This just failed again in master.

https://gradle-enterprise.elastic.co/s/ah4djmwvojm3c/tests/kyv2y2z3r4v7k-ugkfmsdgmdana