Skip to content

Commit 6eee545

Browse files
andrushaMarcelo Vanzin
authored andcommitted
[SPARK-23449][K8S] Preserve extraJavaOptions ordering
For some JVM options, like `-XX:+UnlockExperimentalVMOptions` ordering is necessary. ## What changes were proposed in this pull request? Keep original `extraJavaOptions` ordering, when passing them through environment variables inside the Docker container. ## How was this patch tested? Ran base branch a couple of times and checked startup command in logs. Ordering differed every time. Added sorting, ordering was consistent to what user had in `extraJavaOptions`. Author: Andrew Korzhuev <[email protected]> Closes #20628 from andrusha/patch-2. (cherry picked from commit 185f5bc) Signed-off-by: Marcelo Vanzin <[email protected]>
1 parent 1f180cd commit 6eee545

File tree

1 file changed

+1
-1
lines changed
  • resource-managers/kubernetes/docker/src/main/dockerfiles/spark

1 file changed

+1
-1
lines changed

resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ fi
4141
shift 1
4242

4343
SPARK_CLASSPATH="$SPARK_CLASSPATH:${SPARK_HOME}/jars/*"
44-
env | grep SPARK_JAVA_OPT_ | sed 's/[^=]*=\(.*\)/\1/g' > /tmp/java_opts.txt
44+
env | grep SPARK_JAVA_OPT_ | sort -t_ -k4 -n | sed 's/[^=]*=\(.*\)/\1/g' > /tmp/java_opts.txt
4545
readarray -t SPARK_JAVA_OPTS < /tmp/java_opts.txt
4646
if [ -n "$SPARK_MOUNTED_CLASSPATH" ]; then
4747
SPARK_CLASSPATH="$SPARK_CLASSPATH:$SPARK_MOUNTED_CLASSPATH"

0 commit comments

Comments
 (0)