[CI] Constant keyword tests failing reproducibly

[mark-vieira]

The introduction of a constant keyword field in #49713 added some additional tests which have failed a few times now in CI. The same four tests seem to reliably fail with this error:

    java.lang.AssertionError: Failure at [constant_keyword/10_basic:106]: _shards.skipped didn't match expected value:
                   _shards.skipped: expected Integer [1] but was Integer [2]
        at __randomizedtesting.SeedInfo.seed([343B2640EC8A2EB4:BC6F199A4276434C]:0)
        at org.elasticsearch.test.rest.yaml.ESClientYamlSuiteTestCase.executeSection(ESClientYamlSuiteTestCase.java:405)
        at org.elasticsearch.test.rest.yaml.ESClientYamlSuiteTestCase.test(ESClientYamlSuiteTestCase.java:382)

Here are some examples of build failures:
https://gradle-enterprise.elastic.co/s/lvbzfzye7qatw/tests/failed
https://gradle-enterprise.elastic.co/s/q56aekioba46m/tests/failed
https://gradle-enterprise.elastic.co/s/5qomgr7ai3c7e/tests/failed

I was able to locally reproduce these failures with the following:

./gradlew ':x-pack:plugin:integTestRunner' --tests "org.elasticsearch.xpack.test.rest.XPackRestIT.*constant_keyword*" -Dtests.seed=343B2640EC8A2EB4 -Dtests.security.manager=true -Dtests.locale=fr-CH -Dtests.timezone=NST -Dcompiler.java=13

@jpountz since you added these tests do you mind taking a look?

[elasticmachine]

Pinging @elastic/es-search (:Search/Search)

[mark-vieira]

Since this reproduced for me I've muted these tests in master with 883104f.

Interestingly enough, these tests have not yet failed in 7.x and the same reproduction line above passes for me when run against 7.x.

[jpountz]

Thanks @mark-vieira, I'm assigning myself.