This repository was archived by the owner on Nov 15, 2024. It is now read-only.
Commit f09a9e9
[SPARK-24007][SQL] EqualNullSafe for FloatType and DoubleType might generate a wrong result by codegen.
## What changes were proposed in this pull request?
`EqualNullSafe` for `FloatType` and `DoubleType` might generate a wrong result by codegen.
```scala
scala> val df = Seq((Some(-1.0d), None), (None, Some(-1.0d))).toDF()
df: org.apache.spark.sql.DataFrame = [_1: double, _2: double]
scala> df.show()
+----+----+
| _1| _2|
+----+----+
|-1.0|null|
|null|-1.0|
+----+----+
scala> df.filter("_1 <=> _2").show()
+----+----+
| _1| _2|
+----+----+
|-1.0|null|
|null|-1.0|
+----+----+
```
The result should be empty but the result remains two rows.
## How was this patch tested?
Added a test.
Author: Takuya UESHIN <[email protected]>
Closes apache#21094 from ueshin/issues/SPARK-24007/equalnullsafe.1 parent f81fa47 commit f09a9e9
File tree
2 files changed
+11
-2
lines changed- sql/catalyst/src
- main/scala/org/apache/spark/sql/catalyst/expressions/codegen
- test/scala/org/apache/spark/sql/catalyst/expressions
2 files changed
+11
-2
lines changedLines changed: 4 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
582 | 582 | | |
583 | 583 | | |
584 | 584 | | |
585 | | - | |
586 | | - | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
| 588 | + | |
587 | 589 | | |
588 | 590 | | |
589 | 591 | | |
| |||
Lines changed: 7 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
442 | 442 | | |
443 | 443 | | |
444 | 444 | | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
445 | 452 | | |
0 commit comments