Skip to content

Commit f72220b

Browse files
cloud-fanMaxGekk
authored and
HyukjinKwon
committed
[SPARK-31606][SQL] Reduce the perf regression of vectorized parquet reader caused by datetime rebase
### What changes were proposed in this pull request? Push the rebase logic to the lower level of the parquet vectorized reader, to make the final code more vectorization-friendly. ### Why are the changes needed? Parquet vectorized reader is carefully implemented, to make it more likely to be vectorized by the JVM. However, the newly added datetime rebase degrade the performance a lot, as it breaks vectorization, even if the datetime values don't need to rebase (this is very likely as dates before 1582 is rare). ### Does this PR introduce any user-facing change? no ### How was this patch tested? Run part of the `DateTimeRebaseBenchmark` locally. The results: before this patch ``` [info] Load dates from parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative [info] ------------------------------------------------------------------------------------------------------------------------ [info] after 1582, vec on, rebase off 2677 2838 142 37.4 26.8 1.0X [info] after 1582, vec on, rebase on 3828 4331 805 26.1 38.3 0.7X [info] before 1582, vec on, rebase off 2903 2926 34 34.4 29.0 0.9X [info] before 1582, vec on, rebase on 4163 4197 38 24.0 41.6 0.6X [info] Load timestamps from parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative [info] ------------------------------------------------------------------------------------------------------------------------ [info] after 1900, vec on, rebase off 3537 3627 104 28.3 35.4 1.0X [info] after 1900, vec on, rebase on 6891 7010 105 14.5 68.9 0.5X [info] before 1900, vec on, rebase off 3692 3770 72 27.1 36.9 1.0X [info] before 1900, vec on, rebase on 7588 7610 30 13.2 75.9 0.5X ``` After this patch ``` [info] Load dates from parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative [info] ------------------------------------------------------------------------------------------------------------------------ [info] after 1582, vec on, rebase off 2758 2944 197 36.3 27.6 1.0X [info] after 1582, vec on, rebase on 2908 2966 51 34.4 29.1 0.9X [info] before 1582, vec on, rebase off 2840 2878 37 35.2 28.4 1.0X [info] before 1582, vec on, rebase on 3407 3433 24 29.4 34.1 0.8X [info] Load timestamps from parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative [info] ------------------------------------------------------------------------------------------------------------------------ [info] after 1900, vec on, rebase off 3861 4003 139 25.9 38.6 1.0X [info] after 1900, vec on, rebase on 4194 4283 77 23.8 41.9 0.9X [info] before 1900, vec on, rebase off 3849 3937 79 26.0 38.5 1.0X [info] before 1900, vec on, rebase on 7512 7546 55 13.3 75.1 0.5X ``` Date type is 30% faster if the values don't need to rebase, 20% faster if need to rebase. Timestamp type is 60% faster if the values don't need to rebase, no difference if need to rebase. Closes #28406 from cloud-fan/perf. Lead-authored-by: Wenchen Fan <[email protected]> Co-authored-by: Maxim Gekk <[email protected]> Signed-off-by: HyukjinKwon <[email protected]>
1 parent 7ef0b69 commit f72220b

File tree

8 files changed

+295
-160
lines changed

8 files changed

+295
-160
lines changed

‎sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/RebaseDateTime.scala‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,8 @@ object RebaseDateTime {
7171
-719164, -682945, -646420, -609895, -536845, -500320, -463795,
7272
-390745, -354220, -317695, -244645, -208120, -171595, -141427)
7373

74+
final val lastSwitchJulianDay: Int = julianGregDiffSwitchDay.last
75+
7476
// The first days of Common Era (CE) which is mapped to the '0001-01-01' date in Julian calendar.
7577
private final val julianCommonEraStartDay = julianGregDiffSwitchDay(0)
7678

@@ -416,6 +418,8 @@ object RebaseDateTime {
416418
// in the interval: [julianGregDiffSwitchMicros(i), julianGregDiffSwitchMicros(i+1))
417419
private val julianGregRebaseMap = loadRebaseRecords("julian-gregorian-rebase-micros.json")
418420

421+
final val lastSwitchJulianTs: Long = julianGregRebaseMap.values.map(_.switches.last).max
422+
419423
/**
420424
* An optimized version of [[rebaseJulianToGregorianMicros(ZoneId, Long)]]. This method leverages
421425
* the pre-calculated rebasing maps to save calculation. If the rebasing map doesn't contain

‎sql/core/benchmarks/DateTimeRebaseBenchmark-jdk11-results.txt‎

Lines changed: 52 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -2,93 +2,93 @@
22
Rebasing dates/timestamps in Parquet datasource
33
================================================================================================
44

5-
OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
5+
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
66
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
77
Save dates to parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
88
------------------------------------------------------------------------------------------------------------------------
9-
after 1582, noop 21171 21171 0 4.7 211.7 1.0X
10-
before 1582, noop 11036 11036 0 9.1 110.4 1.9X
11-
after 1582, rebase off 34321 34321 0 2.9 343.2 0.6X
12-
after 1582, rebase on 33269 33269 0 3.0 332.7 0.6X
13-
before 1582, rebase off 22016 22016 0 4.5 220.2 1.0X
14-
before 1582, rebase on 23338 23338 0 4.3 233.4 0.9X
9+
after 1582, noop 20073 20073 0 5.0 200.7 1.0X
10+
before 1582, noop 10985 10985 0 9.1 109.9 1.8X
11+
after 1582, rebase off 32245 32245 0 3.1 322.4 0.6X
12+
after 1582, rebase on 31434 31434 0 3.2 314.3 0.6X
13+
before 1582, rebase off 21590 21590 0 4.6 215.9 0.9X
14+
before 1582, rebase on 22963 22963 0 4.4 229.6 0.9X
1515

16-
OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
16+
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
1717
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
1818
Load dates from parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
1919
------------------------------------------------------------------------------------------------------------------------
20-
after 1582, vec off, rebase off 12791 13089 287 7.8 127.9 1.0X
21-
after 1582, vec off, rebase on 13203 13271 81 7.6 132.0 1.0X
22-
after 1582, vec on, rebase off 3709 3764 49 27.0 37.1 3.4X
23-
after 1582, vec on, rebase on 5082 5114 29 19.7 50.8 2.5X
24-
before 1582, vec off, rebase off 13059 13153 87 7.7 130.6 1.0X
25-
before 1582, vec off, rebase on 14211 14236 27 7.0 142.1 0.9X
26-
before 1582, vec on, rebase off 3687 3749 72 27.1 36.9 3.5X
27-
before 1582, vec on, rebase on 5449 5497 56 18.4 54.5 2.3X
20+
after 1582, vec off, rebase off 12815 12858 40 7.8 128.1 1.0X
21+
after 1582, vec off, rebase on 13030 13167 148 7.7 130.3 1.0X
22+
after 1582, vec on, rebase off 3705 3712 6 27.0 37.1 3.5X
23+
after 1582, vec on, rebase on 3788 3791 3 26.4 37.9 3.4X
24+
before 1582, vec off, rebase off 12873 12943 61 7.8 128.7 1.0X
25+
before 1582, vec off, rebase on 14072 14165 80 7.1 140.7 0.9X
26+
before 1582, vec on, rebase off 3694 3708 15 27.1 36.9 3.5X
27+
before 1582, vec on, rebase on 4403 4484 81 22.7 44.0 2.9X
2828

29-
OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
29+
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
3030
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
3131
Save timestamps to parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
3232
------------------------------------------------------------------------------------------------------------------------
33-
after 1582, noop 2831 2831 0 35.3 28.3 1.0X
34-
before 1582, noop 2816 2816 0 35.5 28.2 1.0X
35-
after 1582, rebase off 15543 15543 0 6.4 155.4 0.2X
36-
after 1582, rebase on 18391 18391 0 5.4 183.9 0.2X
37-
before 1582, rebase off 15747 15747 0 6.4 157.5 0.2X
38-
before 1582, rebase on 18846 18846 0 5.3 188.5 0.2X
33+
after 1900, noop 3032 3032 0 33.0 30.3 1.0X
34+
before 1900, noop 3043 3043 0 32.9 30.4 1.0X
35+
after 1900, rebase off 15634 15634 0 6.4 156.3 0.2X
36+
after 1900, rebase on 18233 18233 0 5.5 182.3 0.2X
37+
before 1900, rebase off 15820 15820 0 6.3 158.2 0.2X
38+
before 1900, rebase on 19921 19921 0 5.0 199.2 0.2X
3939

40-
OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
40+
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
4141
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
4242
Load timestamps from parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
4343
------------------------------------------------------------------------------------------------------------------------
44-
after 1582, vec off, rebase off 16126 16216 78 6.2 161.3 1.0X
45-
after 1582, vec off, rebase on 18277 18453 165 5.5 182.8 0.9X
46-
after 1582, vec on, rebase off 5030 5067 42 19.9 50.3 3.2X
47-
after 1582, vec on, rebase on 8553 8583 43 11.7 85.5 1.9X
48-
before 1582, vec off, rebase off 15828 15872 39 6.3 158.3 1.0X
49-
before 1582, vec off, rebase on 18899 18959 103 5.3 189.0 0.9X
50-
before 1582, vec on, rebase off 4961 5009 43 20.2 49.6 3.3X
51-
before 1582, vec on, rebase on 9099 9140 40 11.0 91.0 1.8X
44+
after 1900, vec off, rebase off 14987 15008 18 6.7 149.9 1.0X
45+
after 1900, vec off, rebase on 17500 17628 210 5.7 175.0 0.9X
46+
after 1900, vec on, rebase off 5030 5036 7 19.9 50.3 3.0X
47+
after 1900, vec on, rebase on 5066 5109 44 19.7 50.7 3.0X
48+
before 1900, vec off, rebase off 15094 15213 121 6.6 150.9 1.0X
49+
before 1900, vec off, rebase on 18098 18175 101 5.5 181.0 0.8X
50+
before 1900, vec on, rebase off 5008 5012 4 20.0 50.1 3.0X
51+
before 1900, vec on, rebase on 8803 8848 55 11.4 88.0 1.7X
5252

5353

5454
================================================================================================
5555
Rebasing dates/timestamps in ORC datasource
5656
================================================================================================
5757

58-
OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
58+
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
5959
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
6060
Save dates to ORC: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
6161
------------------------------------------------------------------------------------------------------------------------
62-
after 1582, noop 21026 21026 0 4.8 210.3 1.0X
63-
before 1582, noop 11040 11040 0 9.1 110.4 1.9X
64-
after 1582 28171 28171 0 3.5 281.7 0.7X
65-
before 1582 18955 18955 0 5.3 189.5 1.1X
62+
after 1582, noop 19593 19593 0 5.1 195.9 1.0X
63+
before 1582, noop 10581 10581 0 9.5 105.8 1.9X
64+
after 1582 27843 27843 0 3.6 278.4 0.7X
65+
before 1582 19435 19435 0 5.1 194.4 1.0X
6666

67-
OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
67+
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
6868
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
6969
Load dates from ORC: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
7070
------------------------------------------------------------------------------------------------------------------------
71-
after 1582, vec off 10876 10931 49 9.2 108.8 1.0X
72-
after 1582, vec on 3900 3913 20 25.6 39.0 2.8X
73-
before 1582, vec off 11165 11174 12 9.0 111.6 1.0X
74-
before 1582, vec on 4208 4214 7 23.8 42.1 2.6X
71+
after 1582, vec off 10395 10507 119 9.6 103.9 1.0X
72+
after 1582, vec on 3921 3945 22 25.5 39.2 2.7X
73+
before 1582, vec off 10762 10860 127 9.3 107.6 1.0X
74+
before 1582, vec on 4194 4226 41 23.8 41.9 2.5X
7575

76-
OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
76+
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
7777
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
7878
Save timestamps to ORC: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
7979
------------------------------------------------------------------------------------------------------------------------
80-
after 1582, noop 2924 2924 0 34.2 29.2 1.0X
81-
before 1582, noop 2820 2820 0 35.5 28.2 1.0X
82-
after 1582 22228 22228 0 4.5 222.3 0.1X
83-
before 1582 22590 22590 0 4.4 225.9 0.1X
80+
after 1900, noop 3003 3003 0 33.3 30.0 1.0X
81+
before 1900, noop 3016 3016 0 33.2 30.2 1.0X
82+
after 1900 21804 21804 0 4.6 218.0 0.1X
83+
before 1900 23920 23920 0 4.2 239.2 0.1X
8484

85-
OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
85+
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
8686
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
8787
Load timestamps from ORC: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
8888
------------------------------------------------------------------------------------------------------------------------
89-
after 1582, vec off 13591 13658 59 7.4 135.9 1.0X
90-
after 1582, vec on 7399 7488 126 13.5 74.0 1.8X
91-
before 1582, vec off 14065 14096 30 7.1 140.7 1.0X
92-
before 1582, vec on 7950 8127 249 12.6 79.5 1.7X
89+
after 1900, vec off 14112 14128 17 7.1 141.1 1.0X
90+
after 1900, vec on 7347 7459 134 13.6 73.5 1.9X
91+
before 1900, vec off 15170 15192 27 6.6 151.7 0.9X
92+
before 1900, vec on 8280 8312 52 12.1 82.8 1.7X
9393

9494

0 commit comments

Comments
 (0)