GCS repo tests (testSnapshotWithLargeSegmentFiles) failed with "Stream closed"

[DaveCTurner]

This build failed on 7.5 as follows:

NotSerializableExceptionWrapper[storage_exception: Stream closed.]; nested: IOException[Stream closed.];
	at __randomizedtesting.SeedInfo.seed([34A3AA4663B303DB:2399A0BBFD2957A5]:0)
	at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:227)
	at com.google.cloud.storage.spi.v1.HttpStorageRpc.read(HttpStorageRpc.java:690)
	at com.google.cloud.storage.BlobReadChannel$1.call(BlobReadChannel.java:127)
	at com.google.cloud.storage.BlobReadChannel$1.call(BlobReadChannel.java:124)
	at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
	at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
	at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
	at com.google.cloud.storage.BlobReadChannel.read(BlobReadChannel.java:123)
	at org.elasticsearch.repositories.gcs.GoogleCloudStorageBlobStore$1.lambda$read$0(GoogleCloudStorageBlobStore.java:189)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.elasticsearch.repositories.gcs.SocketAccess.doPrivilegedIOException(SocketAccess.java:44)
	at org.elasticsearch.repositories.gcs.GoogleCloudStorageBlobStore$1.read(GoogleCloudStorageBlobStore.java:189)
	at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
	at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
	at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
	at com.fasterxml.jackson.core.json.ByteSourceJsonBootstrapper.ensureLoaded(ByteSourceJsonBootstrapper.java:522)
	at com.fasterxml.jackson.core.json.ByteSourceJsonBootstrapper.detectEncoding(ByteSourceJsonBootstrapper.java:129)
	at com.fasterxml.jackson.core.json.ByteSourceJsonBootstrapper.constructParser(ByteSourceJsonBootstrapper.java:246)
	at com.fasterxml.jackson.core.JsonFactory._createParser(JsonFactory.java:1271)
	at com.fasterxml.jackson.core.JsonFactory.createParser(JsonFactory.java:810)
	at org.elasticsearch.common.xcontent.json.JsonXContent.createParser(JsonXContent.java:91)
	at org.elasticsearch.repositories.blobstore.BlobStoreRepository.getRepositoryData(BlobStoreRepository.java:909)
	at org.elasticsearch.repositories.blobstore.BlobStoreRepository.getRepositoryData(BlobStoreRepository.java:888)
	at org.elasticsearch.snapshots.RestoreService.restoreSnapshot(RestoreService.java:179)
	at org.elasticsearch.action.admin.cluster.snapshots.restore.TransportRestoreSnapshotAction.masterOperation(TransportRestoreSnapshotAction.java:80)
	at org.elasticsearch.action.admin.cluster.snapshots.restore.TransportRestoreSnapshotAction.masterOperation(TransportRestoreSnapshotAction.java:41)
	at org.elasticsearch.action.support.master.TransportMasterNodeAction.masterOperation(TransportMasterNodeAction.java:98)
	at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction.lambda$doStart$3(TransportMasterNodeAction.java:169)
	at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:73)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:773)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Stream closed.
	at java.net.AbstractPlainSocketImpl.available(AbstractPlainSocketImpl.java:470)
	at java.net.SocketInputStream.available(SocketInputStream.java:259)
	at java.io.BufferedInputStream.available(BufferedInputStream.java:410)
	at sun.net.www.MeteredStream.available(MeteredStream.java:170)
	at sun.net.www.http.KeepAliveStream.close(KeepAliveStream.java:85)
	at java.io.FilterInputStream.close(FilterInputStream.java:181)
	at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.close(HttpURLConnection.java:3523)
	at java.io.FilterInputStream.close(FilterInputStream.java:181)
	at com.google.api.client.util.IOUtils.copy(IOUtils.java:90)
	at com.google.api.client.util.IOUtils.copy(IOUtils.java:59)
	at com.google.api.client.http.HttpResponse.download(HttpResponse.java:386)
	at com.google.cloud.storage.spi.v1.HttpStorageRpc.read(HttpStorageRpc.java:684)
	... 32 more

The alleged REPRODUCE WITH line does not reproduce this locally:

REPRODUCE WITH: ./gradlew ':plugins:repository-gcs:test' --tests "org.elasticsearch.repositories.gcs.GoogleCloudStorageBlobStoreRepositoryTests.testSnapshotWithLargeSegmentFiles" \
  -Dtests.seed=34A3AA4663B303DB \
  -Dtests.security.manager=true \
  -Dtests.locale=no \
  -Dtests.timezone=Africa/Maseru \
  -Dcompiler.java=12 \
  -Druntime.java=8

[elasticmachine]

Pinging @elastic/es-distributed (:Distributed/Snapshot/Restore)

[tlrx]

This looks very similar to #48978.

[tlrx]

Thanks to @original-brownbear, #49518 should fix this issues. I'm closing for now and we'll reopen or reference if this issue happens again.