Commit 25a4c5f
[SPARK-38185][SQL] Fix data incorrect if aggregate function is empty
### What changes were proposed in this pull request?
Add `aggregateExpressions.nonEmpty` check in `groupOnly` function.
### Why are the changes needed?
The group only condition should check if the aggregate expression is empty.
In DataFrame api, it is allowed to make a empty aggregations.
So the following query should return 1 rather than 0 because it's a global aggregate.
```scala
val emptyAgg = Map.empty[String, String]
spark.range(2).where("id > 2").agg(emptyAgg).limit(1).count
```
### Does this PR introduce _any_ user-facing change?
yes, bug fix
### How was this patch tested?
Add test
Closes #35490 from ulysses-you/SPARK-38185.
Authored-by: ulysses-you <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>1 parent d4a2e5c commit 25a4c5f
File tree
2 files changed
+8
-1
lines changed- sql
- catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical
- core/src/test/scala/org/apache/spark/sql
2 files changed
+8
-1
lines changedLines changed: 3 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
984 | 984 | | |
985 | 985 | | |
986 | 986 | | |
987 | | - | |
| 987 | + | |
| 988 | + | |
| 989 | + | |
988 | 990 | | |
989 | 991 | | |
990 | 992 | | |
| |||
Lines changed: 5 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1443 | 1443 | | |
1444 | 1444 | | |
1445 | 1445 | | |
| 1446 | + | |
| 1447 | + | |
| 1448 | + | |
| 1449 | + | |
| 1450 | + | |
1446 | 1451 | | |
1447 | 1452 | | |
1448 | 1453 | | |
| |||
0 commit comments