Skip to content

Commit e7927ad

Browse files
committed
Merge pull request alteryx#169 from kayousterhout/mesos_fix
Don't ignore spark.cores.max when using Mesos Coarse mode totalCoresAcquired is decremented but never incremented, causing Spark to effectively ignore spark.cores.max in coarse grained Mesos mode. (cherry picked from commit 1a4cfbe) Signed-off-by: Reynold Xin <[email protected]>
1 parent 41dc566 commit e7927ad

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -181,6 +181,7 @@ private[spark] class CoarseMesosSchedulerBackend(
181181
!slaveIdsWithExecutors.contains(slaveId)) {
182182
// Launch an executor on the slave
183183
val cpusToUse = math.min(cpus, maxCores - totalCoresAcquired)
184+
totalCoresAcquired += cpusToUse
184185
val taskId = newMesosTaskId()
185186
taskIdToSlaveId(taskId) = slaveId
186187
slaveIdsWithExecutors += slaveId

0 commit comments

Comments
 (0)