Commit 3711a81
[SPARK-38185][SQL] Fix data incorrect if aggregate function is empty
Add `aggregateExpressions.nonEmpty` check in `groupOnly` function.
The group only condition should check if the aggregate expression is empty.
In DataFrame api, it is allowed to make a empty aggregations.
So the following query should return 1 rather than 0 because it's a global aggregate.
```scala
val emptyAgg = Map.empty[String, String]
spark.range(2).where("id > 2").agg(emptyAgg).limit(1).count
```
yes, bug fix
Add test
Closes #35490 from ulysses-you/SPARK-38185.
Authored-by: ulysses-you <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
(cherry picked from commit 25a4c5f)
Signed-off-by: Wenchen Fan <[email protected]>1 parent cd87ea8 commit 3711a81
File tree
2 files changed
+11
-1
lines changed- sql
- catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical
- core/src/test/scala/org/apache/spark/sql
2 files changed
+11
-1
lines changedLines changed: 6 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
982 | 982 | | |
983 | 983 | | |
984 | 984 | | |
985 | | - | |
| 985 | + | |
| 986 | + | |
| 987 | + | |
| 988 | + | |
| 989 | + | |
| 990 | + | |
986 | 991 | | |
987 | 992 | | |
988 | 993 | | |
| |||
Lines changed: 5 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1422 | 1422 | | |
1423 | 1423 | | |
1424 | 1424 | | |
| 1425 | + | |
| 1426 | + | |
| 1427 | + | |
| 1428 | + | |
| 1429 | + | |
1425 | 1430 | | |
1426 | 1431 | | |
1427 | 1432 | | |
| |||
0 commit comments