[SPARK-28938][K8S][2.4] Move to supported OpenJDK docker image for Kubernetes #26046
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Test build #111851 has finished for PR 26046 at commit
|
|
Kubernetes integration test starting |
|
Kubernetes integration test status success |
dongjoon-hyun
approved these changes
Oct 7, 2019
Member
dongjoon-hyun
left a comment
dongjoon-hyun
left a comment
There was a problem hiding this comment.
+1, LGTM. Thank you, @viirya .
Merged to branch-2.4.
dongjoon-hyun
pushed a commit
that referenced
this pull request
Oct 7, 2019
…bernetes ### What changes were proposed in this pull request? The current docker image used by Kubernetes is `openjdk:8-alpine`. It was not supported and was removed with the commit docker-library/openjdk@3eb0351#diff-f95ffa3d1377774732c33f7b8368e099. This PR proposes to move to a supported docker image. ### Why are the changes needed? I think there are at least two reasons: 1. According to the commit, Alpine/musl is not officially supported by the OpenJDK project. 2. As no more OpenJDK 8 Alpine images, new JDK updates including security fixes , are not applied to it. See below: ``` docker run -it --rm openjdk:8-alpine java -version openjdk version "1.8.0_212" OpenJDK Runtime Environment (IcedTea 3.12.0) (Alpine 8.212.04-r0) OpenJDK 64-Bit Server VM (build 25.212-b04, mixed mode) ``` ``` docker run -it --rm openjdk:8-jdk-slim java -version openjdk version "1.8.0_222" OpenJDK Runtime Environment (build 1.8.0_222-b10) OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode) ``` ### Does this PR introduce any user-facing change? Yes. This changes the base docker image of Spark. ### How was this patch tested? Existing tests. Closes #26046 from viirya/SPARK-28938-2.4. Authored-by: Liang-Chi Hsieh <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
Member
Author
|
Thanks @dongjoon-hyun |
|
@viirya In this backport, Line 48 of the file As a workaround, the entrypoint can be overridden in k8s with |
This was referenced Oct 29, 2019
dongjoon-hyun
pushed a commit
that referenced
this pull request
Oct 29, 2019
…in/tini` ### What changes were proposed in this pull request? Change entrypoint.sh script for the kubernetes manager image to reference /usr/sbin/tini ### Why are the changes needed? This makes running commands like /bin/bash via pass-through work. This was missing from #26046 ### Does this PR introduce any user-facing change? It makes pass-through work. ### How was this patch tested? I built an image and verified that the following worked: `docker run -it --rm image:version /bin/bash` Closes #26296 from jkleckner/fix-pass-through-38938. Authored-by: Jim Kleckner <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
jkleckner
pushed a commit
to jkleckner/spark-on-k8s-operator
that referenced
this pull request
Dec 6, 2019
As outlined in kubeflow#669 this is needed to run properly because it includes the unreleased fixes from apache/spark#26046 and apache/spark#26296 which are needed for GKE authentication to work properly.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
The current docker image used by Kubernetes is
openjdk:8-alpine. It was not supported and was removed with the commit docker-library/openjdk@3eb0351#diff-f95ffa3d1377774732c33f7b8368e099.This PR proposes to move to a supported docker image.
Why are the changes needed?
I think there are at least two reasons:
, are not applied to it. See below:
Does this PR introduce any user-facing change?
Yes. This changes the base docker image of Spark.
How was this patch tested?
Existing tests.