Commit bca89e2
[SPARK-36183][SQL][FOLLOWUP] Fix push down limit 1 through Aggregate (#861)
* [SPARK-36183][SQL][FOLLOWUP] Fix push down limit 1 through Aggregate
### What changes were proposed in this pull request?
Use `Aggregate.aggregateExpressions` instead of `Aggregate.output` when pushing down limit 1 through Aggregate.
For example:
```scala
spark.range(10).selectExpr("id % 5 AS a", "id % 5 AS b").write.saveAsTable("t1")
spark.sql("SELECT a, b, a AS alias FROM t1 GROUP BY a, b LIMIT 1").explain(true)
```
Before this pr:
```
== Optimized Logical Plan ==
GlobalLimit 1
+- LocalLimit 1
+- !Project [a#227L, b#228L, alias#226L]
+- LocalLimit 1
+- Relation default.t1[a#227L,b#228L] parquet
```
After this pr:
```
== Optimized Logical Plan ==
GlobalLimit 1
+- LocalLimit 1
+- Project [a#227L, b#228L, a#227L AS alias#226L]
+- LocalLimit 1
+- Relation default.t1[a#227L,b#228L] parquet
```
### Why are the changes needed?
Fix bug.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Unit test.
Closes #35286 from wangyum/SPARK-36183-2.
Authored-by: Yuming Wang <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
(cherry picked from commit 9b12571)1 parent cdf792c commit bca89e2
File tree
2 files changed
+15
-2
lines changed- sql/catalyst/src
- main/scala/org/apache/spark/sql/catalyst/optimizer
- test/scala/org/apache/spark/sql/catalyst/optimizer
2 files changed
+15
-2
lines changedLines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
548 | 548 | | |
549 | 549 | | |
550 | 550 | | |
551 | | - | |
| 551 | + | |
552 | 552 | | |
553 | | - | |
| 553 | + | |
554 | 554 | | |
555 | 555 | | |
556 | 556 | | |
| |||
Lines changed: 13 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
182 | 182 | | |
183 | 183 | | |
184 | 184 | | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
185 | 198 | | |
186 | 199 | | |
187 | 200 | | |
| |||
0 commit comments