Commit 0ad35ba
[SPARK-33321][SQL] Migrate ANALYZE TABLE commands to use UnresolvedTableOrView to resolve the identifier
### What changes were proposed in this pull request?
This PR proposes to migrate `ANALYZE TABLE` and `ANALYZE TABLE ... FOR COLUMNS` to use `UnresolvedTableOrView` to resolve the table/view identifier. This allows consistent resolution rules (temp view first, etc.) to be applied for both v1/v2 commands. More info about the consistent resolution rule proposal can be found in [JIRA](https://issues.apache.org/jira/browse/SPARK-29900) or [proposal doc](https://docs.google.com/document/d/1hvLjGA8y_W_hhilpngXVub1Ebv8RsMap986nENCFnrg/edit?usp=sharing).
Note that `ANALYZE TABLE` is not supported for v2 tables.
### Why are the changes needed?
The changes allow consistent resolution behavior when resolving the table/view identifier. For example, the following is the current behavior:
```scala
sql("create temporary view t as select 1")
sql("create database db")
sql("create table db.t using csv as select 1")
sql("use db")
sql("ANALYZE TABLE t compute statistics") // Succeeds
```
With this change, ANALYZE TABLE above fails with the following:
```
org.apache.spark.sql.AnalysisException: t is a temp view not table or permanent view.; line 1 pos 0
at org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42)
at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveTempViews$$anonfun$apply$7.$anonfun$applyOrElse$40(Analyzer.scala:872)
at scala.Option.map(Option.scala:230)
at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveTempViews$$anonfun$apply$7.applyOrElse(Analyzer.scala:870)
at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveTempViews$$anonfun$apply$7.applyOrElse(Analyzer.scala:856)
```
, which is expected since temporary view is resolved first and ANALYZE TABLE doesn't support a temporary view.
### Does this PR introduce _any_ user-facing change?
After this PR, `ANALYZE TABLE t` is resolved to a temp view `t` instead of table `db.t`.
### How was this patch tested?
Updated existing tests.
Closes apache#30229 from imback82/parse_v1table.
Authored-by: Terry Kim <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>1 parent 1740b29 commit 0ad35ba
File tree
12 files changed
+120
-74
lines changed- sql
- catalyst/src
- main/scala/org/apache/spark/sql/catalyst
- analysis
- parser
- plans/logical
- test/scala/org/apache/spark/sql/catalyst/parser
- core/src
- main/scala/org/apache/spark/sql
- catalyst/analysis
- execution/datasources/v2
- test
- resources/sql-tests/results
- scala/org/apache/spark/sql
- connector
- execution
12 files changed
+120
-74
lines changedLines changed: 9 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
865 | 865 | | |
866 | 866 | | |
867 | 867 | | |
868 | | - | |
| 868 | + | |
869 | 869 | | |
870 | | - | |
| 870 | + | |
| 871 | + | |
| 872 | + | |
| 873 | + | |
| 874 | + | |
| 875 | + | |
871 | 876 | | |
872 | 877 | | |
873 | 878 | | |
| |||
926 | 931 | | |
927 | 932 | | |
928 | 933 | | |
929 | | - | |
| 934 | + | |
930 | 935 | | |
931 | 936 | | |
932 | 937 | | |
| |||
1026 | 1031 | | |
1027 | 1032 | | |
1028 | 1033 | | |
1029 | | - | |
| 1034 | + | |
1030 | 1035 | | |
1031 | 1036 | | |
1032 | 1037 | | |
| |||
Lines changed: 3 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
49 | | - | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
50 | 52 | | |
51 | 53 | | |
52 | 54 | | |
| |||
Lines changed: 10 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3216 | 3216 | | |
3217 | 3217 | | |
3218 | 3218 | | |
3219 | | - | |
| 3219 | + | |
3220 | 3220 | | |
3221 | 3221 | | |
3222 | 3222 | | |
| |||
3249 | 3249 | | |
3250 | 3250 | | |
3251 | 3251 | | |
3252 | | - | |
| 3252 | + | |
3253 | 3253 | | |
3254 | 3254 | | |
3255 | 3255 | | |
3256 | 3256 | | |
3257 | 3257 | | |
3258 | 3258 | | |
3259 | | - | |
| 3259 | + | |
| 3260 | + | |
| 3261 | + | |
| 3262 | + | |
3260 | 3263 | | |
3261 | 3264 | | |
3262 | | - | |
3263 | | - | |
| 3265 | + | |
| 3266 | + | |
| 3267 | + | |
| 3268 | + | |
3264 | 3269 | | |
3265 | 3270 | | |
3266 | 3271 | | |
| |||
Lines changed: 0 additions & 19 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
342 | 342 | | |
343 | 343 | | |
344 | 344 | | |
345 | | - | |
346 | | - | |
347 | | - | |
348 | | - | |
349 | | - | |
350 | | - | |
351 | | - | |
352 | | - | |
353 | | - | |
354 | | - | |
355 | | - | |
356 | | - | |
357 | | - | |
358 | | - | |
359 | | - | |
360 | | - | |
361 | | - | |
362 | | - | |
363 | | - | |
364 | 345 | | |
365 | 346 | | |
366 | 347 | | |
| |||
Lines changed: 22 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
567 | 567 | | |
568 | 568 | | |
569 | 569 | | |
| 570 | + | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
| 588 | + | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
Lines changed: 39 additions & 19 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1502 | 1502 | | |
1503 | 1503 | | |
1504 | 1504 | | |
1505 | | - | |
| 1505 | + | |
| 1506 | + | |
| 1507 | + | |
1506 | 1508 | | |
1507 | | - | |
| 1509 | + | |
| 1510 | + | |
| 1511 | + | |
1508 | 1512 | | |
1509 | | - | |
| 1513 | + | |
| 1514 | + | |
| 1515 | + | |
1510 | 1516 | | |
1511 | 1517 | | |
1512 | 1518 | | |
1513 | 1519 | | |
1514 | | - | |
1515 | | - | |
| 1520 | + | |
| 1521 | + | |
| 1522 | + | |
1516 | 1523 | | |
1517 | 1524 | | |
1518 | | - | |
1519 | | - | |
| 1525 | + | |
| 1526 | + | |
| 1527 | + | |
1520 | 1528 | | |
1521 | 1529 | | |
1522 | | - | |
| 1530 | + | |
| 1531 | + | |
| 1532 | + | |
1523 | 1533 | | |
1524 | 1534 | | |
1525 | | - | |
1526 | | - | |
| 1535 | + | |
| 1536 | + | |
| 1537 | + | |
1527 | 1538 | | |
1528 | 1539 | | |
1529 | | - | |
1530 | | - | |
| 1540 | + | |
| 1541 | + | |
| 1542 | + | |
1531 | 1543 | | |
1532 | 1544 | | |
1533 | | - | |
1534 | | - | |
| 1545 | + | |
| 1546 | + | |
| 1547 | + | |
1535 | 1548 | | |
1536 | 1549 | | |
1537 | | - | |
| 1550 | + | |
| 1551 | + | |
| 1552 | + | |
1538 | 1553 | | |
1539 | 1554 | | |
1540 | | - | |
| 1555 | + | |
| 1556 | + | |
| 1557 | + | |
1541 | 1558 | | |
1542 | 1559 | | |
1543 | 1560 | | |
| |||
1550 | 1567 | | |
1551 | 1568 | | |
1552 | 1569 | | |
1553 | | - | |
| 1570 | + | |
| 1571 | + | |
1554 | 1572 | | |
1555 | 1573 | | |
1556 | 1574 | | |
| |||
1559 | 1577 | | |
1560 | 1578 | | |
1561 | 1579 | | |
1562 | | - | |
| 1580 | + | |
| 1581 | + | |
1563 | 1582 | | |
1564 | 1583 | | |
1565 | 1584 | | |
| |||
1568 | 1587 | | |
1569 | 1588 | | |
1570 | 1589 | | |
1571 | | - | |
| 1590 | + | |
| 1591 | + | |
1572 | 1592 | | |
1573 | 1593 | | |
1574 | 1594 | | |
| |||
Lines changed: 17 additions & 19 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
| 25 | + | |
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
30 | | - | |
31 | 30 | | |
32 | 31 | | |
33 | 32 | | |
| |||
255 | 254 | | |
256 | 255 | | |
257 | 256 | | |
258 | | - | |
259 | | - | |
260 | | - | |
261 | | - | |
262 | 257 | | |
263 | | - | |
| 258 | + | |
264 | 259 | | |
265 | 260 | | |
266 | | - | |
267 | | - | |
268 | | - | |
269 | | - | |
270 | | - | |
| 261 | + | |
271 | 262 | | |
272 | 263 | | |
273 | 264 | | |
| |||
419 | 410 | | |
420 | 411 | | |
421 | 412 | | |
422 | | - | |
423 | | - | |
| 413 | + | |
| 414 | + | |
424 | 415 | | |
425 | | - | |
| 416 | + | |
426 | 417 | | |
427 | | - | |
| 418 | + | |
428 | 419 | | |
429 | 420 | | |
430 | | - | |
431 | | - | |
432 | | - | |
| 421 | + | |
| 422 | + | |
433 | 423 | | |
434 | 424 | | |
435 | 425 | | |
| |||
706 | 696 | | |
707 | 697 | | |
708 | 698 | | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
709 | 707 | | |
710 | 708 | | |
711 | 709 | | |
| |||
Lines changed: 4 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
231 | 231 | | |
232 | 232 | | |
233 | 233 | | |
234 | | - | |
| 234 | + | |
235 | 235 | | |
236 | 236 | | |
237 | 237 | | |
| |||
280 | 280 | | |
281 | 281 | | |
282 | 282 | | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
283 | 286 | | |
284 | 287 | | |
285 | 288 | | |
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
540 | 540 | | |
541 | 541 | | |
542 | 542 | | |
543 | | - | |
| 543 | + | |
544 | 544 | | |
545 | 545 | | |
546 | 546 | | |
| |||
Lines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
540 | 540 | | |
541 | 541 | | |
542 | 542 | | |
543 | | - | |
| 543 | + | |
544 | 544 | | |
545 | 545 | | |
546 | | - | |
| 546 | + | |
547 | 547 | | |
548 | 548 | | |
549 | 549 | | |
| |||
0 commit comments