Skip to content

Commit cce7745

Browse files
andrewor14tdas
authored andcommitted
[SPARK-1896] Respect spark.master (and --master) before MASTER in spark-shell
The hierarchy for configuring the Spark master in the shell is as follows: ``` MASTER > --master > spark.master (spark-defaults.conf) ``` This is inconsistent with the way we run normal applications, which is: ``` --master > spark.master (spark-defaults.conf) > MASTER ``` I was trying to run a shell locally on a standalone cluster launched through the ec2 scripts, which automatically set `MASTER` in spark-env.sh. It was surprising to me that `--master` didn't take effect, considering that this is the way we tell users to set their masters [here](http://people.apache.org/~pwendell/spark-1.0.0-rc7-docs/scala-programming-guide.html#initializing-spark). Author: Andrew Or <[email protected]> Closes apache#846 from andrewor14/shell-master and squashes the following commits: 2cb81c9 [Andrew Or] Respect spark.master before MASTER in REPL
1 parent 8edbee7 commit cce7745

File tree

1 file changed

+2
-3
lines changed

1 file changed

+2
-3
lines changed

repl/src/main/scala/org/apache/spark/repl/SparkILoop.scala

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -962,11 +962,10 @@ class SparkILoop(in0: Option[BufferedReader], protected val out: JPrintWriter,
962962
private def getMaster(): String = {
963963
val master = this.master match {
964964
case Some(m) => m
965-
case None => {
965+
case None =>
966966
val envMaster = sys.env.get("MASTER")
967967
val propMaster = sys.props.get("spark.master")
968-
envMaster.orElse(propMaster).getOrElse("local[*]")
969-
}
968+
propMaster.orElse(envMaster).getOrElse("local[*]")
970969
}
971970
master
972971
}

0 commit comments

Comments
 (0)