Skip to content

Commit 08d4f70

Browse files
Brennon YorkJoshRosen
authored andcommitted
[SPARK-4298][Core] - The spark-submit cannot read Main-Class from Manifest.
Resolves a bug where the `Main-Class` from a .jar file wasn't being read in properly. This was caused by the fact that the `primaryResource` object was a URI and needed to be normalized through a call to `.getPath` before it could be passed into the `JarFile` object. Author: Brennon York <[email protected]> Closes #3561 from brennonyork/SPARK-4298 and squashes the following commits: 5e0fce1 [Brennon York] Use string interpolation for error messages, moved comment line from original code to above its necessary code segment 14daa20 [Brennon York] pushed mainClass assignment into match statement, removed spurious spaces, removed { } from case statements, removed return values c6dad68 [Brennon York] Set case statement to support multiple jar URI's and enabled the 'file' URI to load the main-class 8d20936 [Brennon York] updated to reset the error message back to the default a043039 [Brennon York] updated to split the uri and jar vals 8da7cbf [Brennon York] fixes SPARK-4298 (cherry picked from commit 8e14c5e) Signed-off-by: Josh Rosen <[email protected]> Conflicts: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
1 parent babcafa commit 08d4f70

File tree

1 file changed

+18
-8
lines changed

1 file changed

+18
-8
lines changed

core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala

Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ package org.apache.spark.deploy
1919

2020
import java.io.{File, FileInputStream, IOException}
2121
import java.util.Properties
22+
import java.net.URI
2223
import java.util.jar.JarFile
2324

2425
import scala.collection.JavaConversions._
@@ -126,14 +127,23 @@ private[spark] class SparkSubmitArguments(args: Seq[String], env: Map[String, St
126127

127128
// Try to set main class from JAR if no --class argument is given
128129
if (mainClass == null && !isPython && primaryResource != null) {
129-
try {
130-
val jar = new JarFile(primaryResource)
131-
// Note that this might still return null if no main-class is set; we catch that later
132-
mainClass = jar.getManifest.getMainAttributes.getValue("Main-Class")
133-
} catch {
134-
case e: Exception =>
135-
SparkSubmit.printErrorAndExit("Cannot load main class from JAR: " + primaryResource)
136-
return
130+
val uri = new URI(primaryResource)
131+
val uriScheme = uri.getScheme()
132+
133+
uriScheme match {
134+
case "file" =>
135+
try {
136+
val jar = new JarFile(uri.getPath)
137+
// Note that this might still return null if no main-class is set; we catch that later
138+
mainClass = jar.getManifest.getMainAttributes.getValue("Main-Class")
139+
} catch {
140+
case e: Exception =>
141+
SparkSubmit.printErrorAndExit(s"Cannot load main class from JAR $primaryResource")
142+
}
143+
case _ =>
144+
SparkSubmit.printErrorAndExit(
145+
s"Cannot load main class from JAR $primaryResource with URI $uriScheme. " +
146+
"Please specify a class through --class.")
137147
}
138148
}
139149

0 commit comments

Comments
 (0)