Skip to content

Commit dffd92e

Browse files
viiryacloud-fan
authored andcommitted
[SPARK-29100][SQL] Fix compilation error in codegen with switch from InSet expression
### What changes were proposed in this pull request? When InSet generates Java switch-based code, if the input set is empty, we don't generate switch condition, but a simple expression that is default case of original switch. ### Why are the changes needed? SPARK-26205 adds an optimization to InSet that generates Java switch condition for certain cases. When the given set is empty, it is possibly that codegen causes compilation error: ``` [info] - SPARK-29100: InSet with empty input set *** FAILED *** (58 milliseconds) [info] Code generation of input[0, int, true] INSET () failed: [info] org.codehaus.janino.InternalCompilerException: failed to compile: org.codehaus.janino.InternalCompilerException: Compiling "GeneratedClass" in "generated.java": Compiling "apply(java.lang.Object _i)"; apply(java.lang.Object _i): Operand stack inconsistent at offset 45: Previous size 0, now 1 [info] org.codehaus.janino.InternalCompilerException: failed to compile: org.codehaus.janino.InternalCompilerException: Compiling "GeneratedClass" in "generated.java": Compiling "apply(java.lang.Object _i)"; apply(java.lang.Object _i): Operand stack inconsistent at offset 45: Previous size 0, now 1 [info] at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.org$apache$spark$sql$catalyst$expressions$codegen$CodeGenerator$$doCompile(CodeGenerator.scala:1308) [info] at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1386) [info] at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1383) ``` ### Does this PR introduce any user-facing change? Yes. Previously, when users have InSet against an empty set, generated code causes compilation error. This patch fixed it. ### How was this patch tested? Unit test added. Closes #25806 from viirya/SPARK-29100. Authored-by: Liang-Chi Hsieh <[email protected]> Signed-off-by: Wenchen Fan <[email protected]>
1 parent 95073fb commit dffd92e

File tree

2 files changed

+19
-5
lines changed

2 files changed

+19
-5
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -457,17 +457,25 @@ case class InSet(child: Expression, hset: Set[Any]) extends UnaryExpression with
457457
break;
458458
""")
459459

460+
val switchCode = if (caseBranches.size > 0) {
461+
code"""
462+
switch (${valueGen.value}) {
463+
${caseBranches.mkString("\n")}
464+
default:
465+
${ev.isNull} = $hasNull;
466+
}
467+
"""
468+
} else {
469+
s"${ev.isNull} = $hasNull;"
470+
}
471+
460472
ev.copy(code =
461473
code"""
462474
${valueGen.code}
463475
${CodeGenerator.JAVA_BOOLEAN} ${ev.isNull} = ${valueGen.isNull};
464476
${CodeGenerator.JAVA_BOOLEAN} ${ev.value} = false;
465477
if (!${valueGen.isNull}) {
466-
switch (${valueGen.value}) {
467-
${caseBranches.mkString("\n")}
468-
default:
469-
${ev.isNull} = $hasNull;
470-
}
478+
$switchCode
471479
}
472480
""")
473481
}

sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/PredicateSuite.scala

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -560,4 +560,10 @@ class PredicateSuite extends SparkFunSuite with ExpressionEvalHelper {
560560
assert(msg.contains("argument 1 requires boolean type"))
561561
}
562562
}
563+
564+
test("SPARK-29100: InSet with empty input set") {
565+
val row = create_row(1)
566+
val inSet = InSet(BoundReference(0, IntegerType, true), Set.empty)
567+
checkEvaluation(inSet, false, row)
568+
}
563569
}

0 commit comments

Comments
 (0)