Piping "curl --retry 8" direct to "tar" negates benefits of retrying

[droberts195]

On the 6.8 branch the Docker build has to download a JDK:

elasticsearch/distribution/docker/src/docker/Dockerfile

Line 19 in 74fe2f4

RUN curl -L --retry 8 -s ${jdkUrl} | tar -C /opt -zxf -

Two builds recently failed on this line:

  Step 4/29 : RUN curl -L --retry 8 -s https://github.com/AdoptOpenJDK/openjdk13-binaries/releases/download/jdk-13%2B33/OpenJDK13U-jdk_x64_linux_hotspot_13_33.tar.gz | tar -C /opt -zxf -
   ---> Running in 29784e13f74f
  
  gzip: stdin: unexpected end of file
  tar: Unexpected EOF in archive
  tar: Unexpected EOF in archive
  tar: Error is not recoverable: exiting now

I suspect the benefit of the --retry is being lost because if a retry occurs after some of the previous attempt has been piped to tar then the overall input to tar will be corrupt. To get the benefit of --retry maybe it's better to download to a temporary file, then extract that only after curl has returned success, then finally delete the temporary file?

[elasticmachine]

Pinging @elastic/es-core-infra (:Core/Infra/Packaging)

[mark-vieira]

To get the benefit of --retry maybe it's better to download to a temporary file, then extract that only after curl has returned success, then finally delete the temporary file?

+1, we can still do this in a single Docker step to avoid adding additional layers by combining these steps with &&.

[alpar-t]

The linked PR didn't help, we'll need something like

elasticsearch/distribution/docker/src/docker/Dockerfile

Lines 44 to 45 in 18d87e3

	yum clean all && exit_code=0 && break || exit_code=\$? && echo "yum error: retry \$iter in 10s" && sleep 10; done; \
	(exit \$exit_code)

[tlrx]

Similar failures on CI today if that can be helpful:
https://gradle-enterprise.elastic.co/s/7z2rp4ql3bi2o/failure?openFailures=WzBd&openStackTraces=WzFd#top=0
https://gradle-enterprise.elastic.co/s/nrtc554ovrbxw/failure?openFailures=WzBd&openStackTraces=WzFd#top=0

[alpar-t]

fixed in #48560