File tree Expand file tree Collapse file tree 1 file changed +0
-16
lines changed Expand file tree Collapse file tree 1 file changed +0
-16
lines changed Original file line number Diff line number Diff line change 3737
3838DEPLOY_MODE=${DEPLOY_MODE:- " client" }
3939
40-
41- # This is a hack to make DStream.pyprint work.
42- # This will be removed after pyprint is moved to PythonDStream.
43- # Problem is that print function is in (Scala)DStream.
44- # Whenever python code is executed, we call PythonDStream which passes
45- # pythonExec(which python Spark should execute). pythonExec is used to call python.
46- # Since pyprint is located in DStream, Spark does not know which python should use.
47- # In that case, get python path from PYSPARK_PYTHON, environmental variable.
48-
49- # Figure out which Python executable to use
50- if [[ -z " $PYSPARK_PYTHON " ]]; then
51- PYSPARK_PYTHON=" python"
52- fi
53- export PYSPARK_PYTHON
54-
55-
5640if [ -n " $DRIVER_MEMORY " ] && [ $DEPLOY_MODE == " client" ]; then
5741 export SPARK_DRIVER_MEMORY=$DRIVER_MEMORY
5842fi
You can’t perform that action at this time.
0 commit comments