Skip to content

Commit 355ab6a

Browse files
KevinSmileHyukjinKwon
authored andcommitted
[SPARK-32804][LAUNCHER][FOLLOWUP] Fix SparkSubmitCommandBuilderSuite test failure without jars
### What changes were proposed in this pull request? It's a followup of #29653. Tests in `SparkSubmitCommandBuilderSuite` may fail if you didn't build first and have jars before test, so if `isTesting` we should set a dummy `SparkLauncher.NO_RESOURCE`. ### Why are the changes needed? Fix tests failure. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? mvn clean test (test without jars built first). Closes #29769 from KevinSmile/bug-fix-master. Authored-by: KevinSmile <[email protected]> Signed-off-by: HyukjinKwon <[email protected]>
1 parent 3bc13e6 commit 355ab6a

File tree

2 files changed

+11
-6
lines changed

2 files changed

+11
-6
lines changed

launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -404,12 +404,17 @@ private boolean isThriftServer(String mainClass) {
404404
}
405405

406406
private String findExamplesAppJar() {
407-
for (String exampleJar : findExamplesJars()) {
408-
if (new File(exampleJar).getName().startsWith("spark-examples")) {
409-
return exampleJar;
407+
boolean isTesting = "1".equals(getenv("SPARK_TESTING"));
408+
if (isTesting) {
409+
return SparkLauncher.NO_RESOURCE;
410+
} else {
411+
for (String exampleJar : findExamplesJars()) {
412+
if (new File(exampleJar).getName().startsWith("spark-examples")) {
413+
return exampleJar;
414+
}
410415
}
416+
throw new IllegalStateException("Failed to find examples' main app jar.");
411417
}
412-
throw new IllegalStateException("Failed to find examples' main app jar.");
413418
}
414419

415420
private List<String> findExamplesJars() {

launcher/src/test/java/org/apache/spark/launcher/SparkSubmitCommandBuilderSuite.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -259,8 +259,8 @@ public void testExamplesRunnerPrimaryResource() throws Exception {
259259
findArgValue(cmd, parser.CLASS));
260260
assertEquals("cluster", findArgValue(cmd, parser.DEPLOY_MODE));
261261
String primaryResource = cmd.get(cmd.size() - 2);
262-
assertTrue(new File(primaryResource).getName().startsWith("spark-examples"));
263-
assertFalse(cmd.contains(SparkLauncher.NO_RESOURCE));
262+
assertTrue(primaryResource.equals(SparkLauncher.NO_RESOURCE)
263+
|| new File(primaryResource).getName().startsWith("spark-examples"));
264264
}
265265

266266
@Test(expected = IllegalArgumentException.class)

0 commit comments

Comments
 (0)