Commit aa3a742
[SPARK-31159][SQL][FOLLOWUP] Move checking of the
### What changes were proposed in this pull request?
In the PR, I propose to refactor reading of timestamps of the `TIMESTAMP_MILLIS` logical type from Parquet files in `VectorizedColumnReader`, and move checking of the `rebaseDateTime` flag out of the internal loop.
### Why are the changes needed?
To avoid any additional overhead of the checking the SQL config `spark.sql.legacy.parquet.rebaseDateTime.enabled` introduced by the PR #27915.
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
By running the test suite `ParquetIOSuite`.
Closes #27973 from MaxGekk/rebase-parquet-datetime-followup.
Authored-by: Maxim Gekk <[email protected]>
Signed-off-by: HyukjinKwon <[email protected]>rebaseDateTime flag out of the loop in VectorizedColumnReader
1 parent d929c0d commit aa3a742
File tree
1 file changed
+15
-8
lines changed- sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet
1 file changed
+15
-8
lines changedLines changed: 15 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
462 | 462 | | |
463 | 463 | | |
464 | 464 | | |
465 | | - | |
466 | | - | |
467 | | - | |
468 | | - | |
469 | | - | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
| 476 | + | |
| 477 | + | |
| 478 | + | |
| 479 | + | |
470 | 480 | | |
471 | | - | |
472 | | - | |
473 | | - | |
474 | 481 | | |
475 | 482 | | |
476 | 483 | | |
| |||
0 commit comments