Improve the image building workflow #488
Conversation
| # with Kubernetes support. | ||
|
|
||
| declare -A path=( [spark-driver]=dockerfiles/driver/Dockerfile \ | ||
| [spark-executor]=dockerfiles/executor/Dockerfile \ |
There was a problem hiding this comment.
Execution failed
./build-push-docker-images.sh: line 22: declare: -A: invalid option
declare: usage: declare [-afFirtx] [-p] [name[=value] ...]
unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /Users/jimmy/Workspace/github/rootsongjc/spark/dist/sbin/dockerfiles: no such file or directory
unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /Users/jimmy/Workspace/github/rootsongjc/spark/dist/sbin/dockerfiles: no such file or directory
There was a problem hiding this comment.
I think that's because of an older bash version. https://stackoverflow.com/questions/6047648/bash-4-associative-arrays-error-declare-a-invalid-option
There was a problem hiding this comment.
MacOS bash version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin16)
Copyright (C) 2007 Free Software Foundation, Inc.
I updated it with this command:
brew install bash
Now bash version is 4.4.12(1)-release (x86_64-apple-darwin16.3.0)
The error disappeared.
There was a problem hiding this comment.
As this script located at sbin/build-push-docker-images.sh but the path in the script based on dockerfile, this directory not in the base path.
This file should be in dict directory.
There was a problem hiding this comment.
It is meant to be run from a release that is built and published.
There was a problem hiding this comment.
I put it to dist directory to run this script.
felixcheung
left a comment
felixcheung
left a comment
There was a problem hiding this comment.
LGTM.
we probably need some auth, image signing later on
|
Thanks for the review, merging when green. |
|
rerun unit tests please |
|
I didn't find a thread on this so I'll comment here - found out more about the process and it turns out ASF Infra has support for building and publishing image to the official Apache account/owner. I'll write up what I've found for the next chat. |
This will let us build and push the spark docker images using a single command from a downloaded release.
Also useful for folks trying to customize their images.
cc @ash211 @mccheah @erikerlandson @liyinan926