[CI] org.elasticsearch.node.NodeTests failures

[iverase]

This test seems to be failing the time to time with the following error:

com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=716, name=elasticsearch[mock_ node][search][T#1], state=RUNNABLE, group=TGRP-NodeTests]
	at __randomizedtesting.SeedInfo.seed([60CD27ABD7612595:1A041A5993EEF0D6]:0)
Caused by: java.lang.IllegalStateException: threadcontext is already closed
	at __randomizedtesting.SeedInfo.seed([60CD27ABD7612595]:0)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextThreadLocal.ensureOpen(ThreadContext.java:658)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextThreadLocal.get(ThreadContext.java:651)
	at org.elasticsearch.common.util.concurrent.ThreadContext.stashContext(ThreadContext.java:132)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:754)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

Examples for this failure are:

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+7.x+multijob-unix-compatibility/os=debian-9/230/console

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+7.2+matrix-java-periodic/ES_BUILD_JAVA=openjdk12,ES_RUNTIME_JAVA=zulu11,nodes=immutable&&linux&&docker/5/console

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+7.x+release-tests/400/console

every time complains from a different test method.

Relates to #42350

[elasticmachine]

Pinging @elastic/es-core-infra

[danielmitterdorfer]

We have several more build failures that all fail with the same symptoms described above. Attached is a list of all build failures so far (extraced from build-stats via the query class:"org.elasticsearch.node.NodeTests" AND stacktrace:"threadcontext is already closed"). We see that the failures have started on April 30, 2019.

Example reproduction line:

./gradlew :server:test --tests "org.elasticsearch.node.NodeTests" -Dtests.seed=A00CF79A07E268E5 -Dtests.security.manager=true -Dtests.locale=en-US -Dtests.timezone=UTC -Dcompiler.java=12 -Druntime.java=12

This one was taken from build failure https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+7.x+matrix-java-periodic/ES_BUILD_JAVA=openjdk12,ES_RUNTIME_JAVA=zulu12,nodes=immutable&&linux&&docker/200. I was not able to reproduce this failure locally (100 iterations).