[CI] 2 failures in SamlAuthenticationIT

[original-brownbear]

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-unix-compatibility/os=centos/42/console

REPRODUCE WITH: ./gradlew :x-pack:qa:saml-idp-tests:integTestRunner \
  -Dtests.seed=EF941323E50925A6 \
  -Dtests.class=org.elasticsearch.xpack.security.authc.saml.SamlAuthenticationIT \
  -Dtests.method="testLoginUserWithAuthorizingRealm" \
  -Dtests.security.manager=true \
  -Dtests.locale=ar-TN \
  -Dtests.timezone=Asia/Khandyga \
  -Dcompiler.java=11 \
  -Druntime.java=8

REPRODUCE WITH: ./gradlew :x-pack:qa:saml-idp-tests:integTestRunner \
  -Dtests.seed=EF941323E50925A6 \
  -Dtests.class=org.elasticsearch.xpack.security.authc.saml.SamlAuthenticationIT \
  -Dtests.method="testLoginUserWithSamlRoleMapping" \
  -Dtests.security.manager=true \
  -Dtests.locale=ar-TN \
  -Dtests.timezone=Asia/Khandyga \
  -Dcompiler.java=11 \
  -Druntime.java=8

[elasticmachine]

Pinging @elastic/es-security

[jkakavas]

Errors are

2> REPRODUCE WITH: ./gradlew :x-pack:qa:saml-idp-tests:integTestRunner -Dtests.seed=EF941323E50925A6 -Dtests.class=org.elasticsearch.xpack.security.authc.saml.SamlAuthenticationIT -Dtests.method="testLoginUserWithSamlRoleMapping" -Dtests.
security.manager=true -Dtests.locale=ar-TN -Dtests.timezone=Asia/Khandyga -Dcompiler.java=11 -Druntime.java=8
  2> NOTE: leaving temporary files on disk at: /var/lib/jenkins/workspace/elastic+elasticsearch+master+multijob-unix-compatibility/os/centos/x-pack/qa/saml-idp-tests/build/testrun/integTestRunner/J0/temp/org.elasticsearch.xpack.security.aut
hc.saml.SamlAuthenticationIT_EF941323E50925A6-001
  2> NOTE: test params are: codec=Asserting(Lucene80): {}, docValues:{}, maxPointsInLeafNode=922, maxMBSortInHeap=5.702762557240961, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@53100397), locale=ar-TN, timezone=A
sia/Khandyga
  2> NOTE: Linux 3.10.0-862.3.2.el7.x86_64 amd64/Oracle Corporation 1.8.0_181 (64-bit)/cpus=8,threads=1,free=423419192,total=514850816
  2> NOTE: All tests run in this JVM: [SamlAuthenticationIT]
   >    at __randomizedtesting.SeedInfo.seed([EF941323E50925A6:BDE91C5C41A4571E]:0)
   >    at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:687)
   >    at org.elasticsearch.client.RestClient.performRequest(RestClient.java:218)
   >    at org.elasticsearch.xpack.security.authc.saml.SamlAuthenticationIT.verifyElasticsearchRefreshToken(SamlAuthenticationIT.java:331)
   >    at org.elasticsearch.xpack.security.authc.saml.SamlAuthenticationIT.testLoginUserWithAuthorizingRealm(SamlAuthenticationIT.java:255)
   >    at java.lang.Thread.run(Thread.java:748)
   > Caused by: org.elasticsearch.client.ResponseException: method [POST], host [http://[::1]:33482], URI [/_xpack/security/oauth2/token], status line [HTTP/1.1 400 Bad Request]
   > {"error":"invalid_grant","error_description":"token is missing metadata"}
   >    at org.elasticsearch.client.RestClient$1.completed(RestClient.java:307)
   >    at org.elasticsearch.client.RestClient$1.completed(RestClient.java:292)
   >    at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119)
   >    at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177)
   >    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436)
   >    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326)
   >    at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
   >    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
   >    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
   >    at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
   >    at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
   >    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
   >    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
   >    at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
   >    at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
   >    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
   >    ... 1 more

There are some warnings right above that (probably unrelated)

.security.manager=true -Dtests.locale=ar-TN -Dtests.timezone=Asia/Khandyga -Dcompiler.java=11 -Druntime.java=8
  1> [2018-10-05T15:11:09,522][WARN ][o.e.b.JNANatives         ] [[SUITE-SamlAuthenticationIT-seed#[EF941323E50925A6]]] Unable to lock JVM Memory: error=12, reason=Cannot allocate memory
  1> [2018-10-05T15:11:09,524][WARN ][o.e.b.JNANatives         ] [[SUITE-SamlAuthenticationIT-seed#[EF941323E50925A6]]] This can result in part of the JVM being swapped out.
  1> [2018-10-05T15:11:09,524][WARN ][o.e.b.JNANatives         ] [[SUITE-SamlAuthenticationIT-seed#[EF941323E50925A6]]] Increase RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536
  1> [2018-10-05T15:11:09,524][WARN ][o.e.b.JNANatives         ] [[SUITE-SamlAuthenticationIT-seed#[EF941323E50925A6]]] These can be adjusted by modifying /etc/security/limits.conf, for example: 

[tvernum]

This seems to be caused by #34290, and I have a pretty good theory about why.