Skip to content

Commit 041e720

Browse files
author
Andrew Or
committed
[SPARK-9649] Fix flaky test MasterSuite - randomize ports
``` Error Message Failed to bind to: /127.0.0.1:7093: Service 'sparkMaster' failed after 16 retries! Stacktrace java.net.BindException: Failed to bind to: /127.0.0.1:7093: Service 'sparkMaster' failed after 16 retries! at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272) at akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(NettyTransport.scala:393) at akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(NettyTransport.scala:389) at scala.util.Success$$anonfun$map$1.apply(Try.scala:206) at scala.util.Try$.apply(Try.scala:161) ``` Author: Andrew Or <[email protected]> Closes #7968 from andrewor14/fix-master-flaky-test and squashes the following commits: fcc42ef [Andrew Or] Randomize port Conflicts: core/src/test/scala/org/apache/spark/deploy/master/MasterSuite.scala
1 parent 8ce86b2 commit 041e720

File tree

3 files changed

+4
-1
lines changed

3 files changed

+4
-1
lines changed

core/src/test/scala/org/apache/spark/deploy/master/MasterSuite.scala

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,7 @@ class MasterSuite extends SparkFunSuite with Matchers with Eventually {
8181
conf.set("spark.deploy.recoveryMode", "CUSTOM")
8282
conf.set("spark.deploy.recoveryMode.factory",
8383
classOf[CustomRecoveryModeFactory].getCanonicalName)
84+
conf.set("spark.master.rest.enabled", "false")
8485

8586
val instantiationAttempts = CustomRecoveryModeFactory.instantiationAttempts
8687

@@ -135,7 +136,7 @@ class MasterSuite extends SparkFunSuite with Matchers with Eventually {
135136
)
136137

137138
val (actorSystem, port, uiPort, restPort) =
138-
Master.startSystemAndActor("127.0.0.1", 7077, 8080, conf)
139+
Master.startSystemAndActor("127.0.0.1", 0, 0, conf)
139140

140141
try {
141142
Await.result(actorSystem.actorSelection("/user/Master").resolveOne(10 seconds), 10 seconds)

pom.xml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1256,6 +1256,7 @@
12561256
<spark.test.home>${spark.test.home}</spark.test.home>
12571257
<spark.testing>1</spark.testing>
12581258
<spark.ui.enabled>false</spark.ui.enabled>
1259+
<spark.master.rest.enabled>false</spark.master.rest.enabled>
12591260
<spark.ui.showConsoleProgress>false</spark.ui.showConsoleProgress>
12601261
<spark.driver.allowMultipleContexts>true</spark.driver.allowMultipleContexts>
12611262
<spark.unsafe.exceptionOnMemoryLeak>true</spark.unsafe.exceptionOnMemoryLeak>

project/SparkBuild.scala

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -514,6 +514,7 @@ object TestSettings {
514514
javaOptions in Test += "-Dspark.testing=1",
515515
javaOptions in Test += "-Dspark.port.maxRetries=100",
516516
javaOptions in Test += "-Dspark.ui.enabled=false",
517+
javaOptions in Test += "-Dspark.master.rest.enabled=false",
517518
javaOptions in Test += "-Dspark.ui.showConsoleProgress=false",
518519
javaOptions in Test += "-Dspark.driver.allowMultipleContexts=true",
519520
javaOptions in Test += "-Dspark.unsafe.exceptionOnMemoryLeak=true",

0 commit comments

Comments
 (0)