Skip to content

Commit a7b5319

Browse files
committed
updated python docs
1 parent 274c4d4 commit a7b5319

File tree

1 file changed

+6
-3
lines changed

1 file changed

+6
-3
lines changed

src/jekyll/running-on-kubernetes.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -169,28 +169,31 @@ Here is how you would execute a Spark-Pi example:
169169
bin/spark-submit \
170170
--deploy-mode cluster \
171171
--master k8s://https://<k8s-apiserver-host>:<k8s-apiserver-port> \
172-
--kubernetes-namespace default \
172+
--kubernetes-namespace <k8s-namespace> \
173173
--conf spark.executor.instances=5 \
174174
--conf spark.app.name=spark-pi \
175175
--conf spark.kubernetes.driver.docker.image=kubespark/driver-py:v2.1.0-kubernetes-0.3.0 \
176176
--conf spark.kubernetes.executor.docker.image=kubespark/executor-py:v2.1.0-kubernetes-0.3.0 \
177177
--conf spark.kubernetes.initcontainer.docker.image=kubespark/spark-init:v2.1.0-kubernetes-0.3.0 \
178-
local:///opt/spark/examples/src/main/python/pi.py 10
178+
--jars local:///opt/spark/examples/jars/spark-examples_2.11-2.1.0-k8s-0.3.0-SNAPSHOT.jar \
179+
local:///opt/spark/examples/src/main/python/pi.py 10
179180

180181
With Python support it is expected to distribute `.egg`, `.zip` and `.py` libraries to executors via the `--py-files` option.
181182
We support this as well, as seen with the following example:
182183

183184
bin/spark-submit \
184185
--deploy-mode cluster \
185186
--master k8s://https://<k8s-apiserver-host>:<k8s-apiserver-port> \
186-
--kubernetes-namespace default \
187+
--kubernetes-namespace <k8s-namespace> \
187188
--conf spark.executor.instances=5 \
188189
--conf spark.app.name=spark-pi \
189190
--conf spark.kubernetes.driver.docker.image=kubespark/driver-py:v2.1.0-kubernetes-0.3.0 \
190191
--conf spark.kubernetes.executor.docker.image=kubespark/executor-py:v2.1.0-kubernetes-0.3.0 \
191192
--conf spark.kubernetes.initcontainer.docker.image=kubespark/spark-init:v2.1.0-kubernetes-0.3.0 \
193+
--jars local:///opt/spark/examples/jars/spark-examples_2.11-2.1.0-k8s-0.3.0-SNAPSHOT.jar \
192194
--py-files local:///opt/spark/examples/src/main/python/sort.py \
193195
local:///opt/spark/examples/src/main/python/pi.py 10
196+
194197
195198
You may also customize your Docker images to use different `pip` packages that suit your use-case. As you can see
196199
with the current `driver-py` Docker image we have commented out the current pip module support that you can uncomment

0 commit comments

Comments
 (0)