[TEST] TokenBackwardsCompatibilityIT.testMixedCluster failing #31195
Description
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+6.x+intake/1846/console
Reproduction steps:
./gradlew :x-pack:qa:rolling-upgrade:with-system-key:v5.6.10-SNAPSHOT#twoThirdsUpgradedTestRunner -Dtests.seed=7C807A34A424F2C8 -Dtests.class=org.elasticsearch.upgrades.TokenBackwardsCompatibilityIT -Dtests.method="testMixedCluster" -Dtests.security.manager=true -Dtests.locale=pt-BR -Dtests.timezone=Africa/Bissau
Relevant output:
> Throwable #1: org.elasticsearch.client.ResponseException: method [GET], host [http://[::1]:26779], URI [_xpack/security/_authenticate], status line [HTTP/1.1 401 Unauthorized]
> {"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [_xpack/security/_authenticate]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication token for REST request [_xpack/security/_authenticate]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}
> at __randomizedtesting.SeedInfo.seed([7C807A34A424F2C8:71E12A5C778C8339]:0)
> at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:821)
> at org.elasticsearch.client.RestClient.performRequest(RestClient.java:182)
> at org.elasticsearch.client.RestClient.performRequest(RestClient.java:249)
> at org.elasticsearch.upgrades.TokenBackwardsCompatibilityIT.assertTokenWorks(TokenBackwardsCompatibilityIT.java:175)
> at org.elasticsearch.upgrades.TokenBackwardsCompatibilityIT.testMixedCluster(TokenBackwardsCompatibilityIT.java:103)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.elasticsearch.client.ResponseException: method [GET], host [http://[::1]:26779], URI [_xpack/security/_authenticate], status line [HTTP/1.1 401 Unauthorized]
> {"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [_xpack/security/_authenticate]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication token for REST request [_xpack/security/_authenticate]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}
> at org.elasticsearch.client.RestClient$1.completed(RestClient.java:495)
> at org.elasticsearch.client.RestClient$1.completed(RestClient.java:484)
> at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119)
> at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177)
> at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436)
> at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326)
> at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
> at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
> at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
> at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
> at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
> at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
> at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
> at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
> at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
> at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
> ... 1 more
1> [2018-06-08T01:16:41,322][INFO ][o.e.u.TokenBackwardsCompatibilityIT] [testGeneratingTokenInOldCluster]: before test
1> [2018-06-08T01:16:41,349][INFO ][o.e.u.TokenBackwardsCompatibilityIT] [testGeneratingTokenInOldCluster]: after test
IGNOR/A 0.04s | TokenBackwardsCompatibilityIT.testGeneratingTokenInOldCluster
> Assumption #1: this test should only run against the old cluster
1> [2018-06-08T01:16:41,358][INFO ][o.e.u.TokenBackwardsCompatibilityIT] [testTokenWorksInMixedOrUpgradedCluster]: before test
1> [2018-06-08T01:16:41,395][INFO ][o.e.u.TokenBackwardsCompatibilityIT] [testTokenWorksInMixedOrUpgradedCluster]: after test
2> NOTE: leaving temporary files on disk at: /var/lib/jenkins/workspace/elastic+elasticsearch+6.x+intake/x-pack/qa/rolling-upgrade/with-system-key/build/testrun/v5.6.10-SNAPSHOT#twoThirdsUpgradedTestRunner/J0/temp/org.elasticsearch.upgrades.TokenBackwardsCompatibilityIT_7C807A34A424F2C8-001
2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1605, maxMBSortInHeap=6.210594783023145, sim=RandomSimilarity(queryNorm=true): {}, locale=pt-BR, timezone=Africa/Bissau
2> NOTE: Linux 3.16.0-4-amd64 amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=16,threads=1,free=350787576,total=514850816
2> NOTE: All tests run in this JVM: [IndexAuditUpgradeIT, WatchBackwardsCompatibilityIT, TokenBackwardsCompatibilityIT]
Completed [3/5] in 2.23s, 4 tests, 1 error, 2 skipped <<< FAILURES!
There are no files in the temporary directory:
jenkins@worker-02a46c5b7f9fd3b1f:~/workspace/elastic+elasticsearch+6.x+intake/x-pack/qa/rolling-upgrade/with-system-key/build/testrun/v5.6.10-SNAPSHOT#twoThirdsUpgradedTestRunner/J0/temp/org.elasticsearch.upgrades.TokenBackwardsCompatibilityIT_7C807A34A424F2C8-001$ find .
.
./tempDir-001