[ML] DataFrameTransformsCheckpointServiceTests.testExtractIndexCheckpointsInconsistentGlobalCheckpoints fails reproducibly

[benwtrent]

Able to reproduce locally from master and 7.x:

./gradlew :x-pack:plugin:data-frame:unitTest -Dtests.seed=80ACEF73CA827F27 -Dtests.class=org.elasticsearch.xpack.dataframe.checkpoint.DataFrameTransformsCheckpointServiceTests -Dtests.method="testExtractIndexCheckpointsInconsistentGlobalCheckpoints"

Trace:

FAILURE 0.49s | DataFrameTransformsCheckpointServiceTests.testExtractIndexCheckpointsInconsistentGlobalCheckpoints <<< FAILURES!
   > Throwable #1: junit.framework.AssertionFailedError: Expected exception CheckpointException but no exception was thrown
   >    at __randomizedtesting.SeedInfo.seed([80ACEF73CA827F27:EA2959BCB3423D4B]:0)
   >    at org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2685)
   >    at org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2670)
   >    at org.elasticsearch.xpack.dataframe.checkpoint.DataFrameTransformsCheckpointServiceTests.testExtractIndexCheckpointsInconsistentGlobalCheckpoints(DataFrameTransformsCheckpointServiceTests.java:92)
   >    at java.lang.Thread.run(Thread.java:748)

[elasticmachine]

Pinging @elastic/ml-core

[benwtrent]

Occurred again, but with a different seed:

./gradlew :x-pack:plugin:data-frame:unitTest -Dtests.seed=FE5FC63B63A4D079 -Dtests.class=org.elasticsearch.xpack.dataframe.checkpoint.DataFrameTransformsCheckpointServiceTests -Dtests.method="testExtractIndexCheckpointsInconsistentGlobalCheckpoints" -Dtests.security.manager=true -Dtests.locale=ar-JO -Dtests.timezone=America/North_Dakota/Beulah -Dcompiler.java=12 -Druntime.java=8

Same error as before, expecting it to throw but it does not.

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+intake/2774/console