Skip to content

Commit ec1c11b

Browse files
committed
Set the context class loader in the Mesos executor backend.
This fixes the `ClassNotFoundException` for Spark classes in the serializer.
1 parent 616be29 commit ec1c11b

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

core/src/main/scala/org/apache/spark/executor/MesosExecutorBackend.scala

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,11 @@ private[spark] class MesosExecutorBackend
6363

6464
logInfo(s"Registered with Mesos as executor ID $executorId with $cpusPerTask cpus")
6565
this.driver = driver
66+
// Set a context class loader to be picked up by the serializer. Without this call
67+
// the serializer would default to the null class loader, and fail to find Spark classes
68+
// See SPARK-10986.
69+
Thread.currentThread().setContextClassLoader(this.getClass.getClassLoader)
70+
6671
val properties = Utils.deserialize[Array[(String, String)]](executorInfo.getData.toByteArray) ++
6772
Seq[(String, String)](("spark.app.id", frameworkInfo.getId.getValue))
6873
val conf = new SparkConf(loadDefaults = true).setAll(properties)

0 commit comments

Comments
 (0)