Skip to content

Commit 49efd03

Browse files
gatorsmileyhuai
authored andcommitted
[SPARK-12138][SQL] Escape \u in the generated comments of codegen
When \u appears in a comment block (i.e. in /**/), code gen will break. So, in Expression and CodegenFallback, we escape \u to \\u. yhuai Please review it. I did reproduce it and it works after the fix. Thanks! Author: gatorsmile <[email protected]> Closes #10155 from gatorsmile/escapeU.
1 parent 04b6799 commit 49efd03

File tree

2 files changed

+12
-1
lines changed

2 files changed

+12
-1
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -220,7 +220,9 @@ abstract class Expression extends TreeNode[Expression] {
220220
* Returns the string representation of this expression that is safe to be put in
221221
* code comments of generated code.
222222
*/
223-
protected def toCommentSafeString: String = this.toString.replace("*/", "\\*\\/")
223+
protected def toCommentSafeString: String = this.toString
224+
.replace("*/", "\\*\\/")
225+
.replace("\\u", "\\\\u")
224226
}
225227

226228

sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CodeGenerationSuite.scala

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,4 +107,13 @@ class CodeGenerationSuite extends SparkFunSuite with ExpressionEvalHelper {
107107
true,
108108
InternalRow(UTF8String.fromString("*/")))
109109
}
110+
111+
test("\\u in the data") {
112+
// When \ u appears in a comment block (i.e. in /**/), code gen will break.
113+
// So, in Expression and CodegenFallback, we escape \ u to \\u.
114+
checkEvaluation(
115+
EqualTo(BoundReference(0, StringType, false), Literal.create("\\u", StringType)),
116+
true,
117+
InternalRow(UTF8String.fromString("\\u")))
118+
}
110119
}

0 commit comments

Comments
 (0)