[CI] XDocsClientYamlTestSuiteIT test {yaml=en/rest-api/watcher/stats/line_153} fails #33864
Description
Example build failure
Reproduction line
(does not) reproduce locally
./gradlew :x-pack:docs:integTestRunner \
-Dtests.seed=7088A82F5379EBE9 \
-Dtests.class=org.elasticsearch.smoketest.XDocsClientYamlTestSuiteIT \
-Dtests.method="test {yaml=en/rest-api/watcher/stats/line_153}" \
-Dtests.security.manager=true \
-Dtests.jvm.argline="-XX:-UseConcMarkSweepGC -XX:+UseG1GC" \
-Dtests.locale=sr-Latn \
-Dtests.timezone=Australia/Melbourne \
-Dcompiler.java=10 \
-Druntime.java=8
Example relevant log:
19:47:10 1> [2018-09-19T05:46:57,699][ERROR][o.e.s.XDocsClientYamlTestSuiteIT] [test] This failing test was generated by documentation starting at en/rest-api/watcher/stats.asciidoc:line_153. It may include many snippets. See Elasticsearch's docs/README.asciidoc for an explanation of test generation.
19:47:10 ERROR 0.39s | XDocsClientYamlTestSuiteIT.test {yaml=en/rest-api/watcher/stats/line_153} <<< FAILURES!
19:47:10 > Throwable #1: org.elasticsearch.test.rest.yaml.ClientYamlTestResponseException: org.elasticsearch.client.ResponseException: method [GET], host [http://[::1]:40253], URI [/_xpack/security/user?error_trace=true], status line [HTTP/1.1 503 Service Unavailable]
19:47:10 > {"error":{"root_cause":[],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[],"stack_trace":"Failed to execute phase [query], all shards failed\n\tat org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:293)\n\tat org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:133)\n\tat org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:254)\n\tat org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:106)\n\tat org.elasticsearch.action.search.InitialSearchPhase.access$200(InitialSearchPhase.java:50)\n\tat org.elasticsearch.action.search.InitialSearchPhase$2.onFailure(InitialSearchPhase.java:277)\n\tat org.elasticsearch.action.search.SearchExecutionStatsCollector.onFailure(SearchExecutionStatsCollector.java:73)\n\tat org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:51)\n\tat org.elasticsearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:487)\n\tat org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1121)\n\tat org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1218)\n\tat org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1202)\n\tat org.elasticsearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:66)\n\tat org.elasticsearch.action.search.SearchTransportService$3.onFailure(SearchTransportService.java:364)\n\tat org.elasticsearch.search.SearchService$2.onFailure(SearchService.java:342)\n\tat org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:336)\n\tat org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:330)\n\tat org.elasticsearch.search.SearchService$3.doRun(SearchService.java:1021)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:723)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)\n\tat org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n\tat java.base/java.lang.Thread.run(Thread.java:844)\n"},"status":503}
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+g1gc/13/consoleFull
Frequency
Happens with low frequency, only one failure exactly like before, but a few others before.
Since it's in docs I wanted to make sure it's recorded.