Skip to content

Commit b96fd44

Browse files
wangyumdongjoon-hyun
wangyum
and
dongjoon-hyun
committed
[SPARK-25476][SPARK-25510][TEST] Refactor AggregateBenchmark and add a new trait to better support Dataset and DataFrame API
## What changes were proposed in this pull request? This PR does 2 things: 1. Add a new trait(`SqlBasedBenchmark`) to better support Dataset and DataFrame API. 2. Refactor `AggregateBenchmark` to use main method. Generate benchmark result: ``` SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "sql/test:runMain org.apache.spark.sql.execution.benchmark.AggregateBenchmark" ``` ## How was this patch tested? manual tests Closes apache#22484 from wangyum/SPARK-25476. Lead-authored-by: Yuming Wang <[email protected]> Co-authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 30f5d0f commit b96fd44

File tree

3 files changed

+633
-513
lines changed

3 files changed

+633
-513
lines changed

‎sql/core/benchmarks/AggregateBenchmark-results.txt

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
================================================================================================
2+
aggregate without grouping
3+
================================================================================================
4+
5+
OpenJDK 64-Bit Server VM 1.8.0_181-b13 on Linux 3.10.0-862.3.2.el7.x86_64
6+
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
7+
agg w/o group: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
8+
------------------------------------------------------------------------------------------------
9+
agg w/o group wholestage off 65374 / 70665 32.1 31.2 1.0X
10+
agg w/o group wholestage on 1178 / 1209 1779.8 0.6 55.5X
11+
12+
13+
================================================================================================
14+
stat functions
15+
================================================================================================
16+
17+
OpenJDK 64-Bit Server VM 1.8.0_181-b13 on Linux 3.10.0-862.3.2.el7.x86_64
18+
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
19+
stddev: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
20+
------------------------------------------------------------------------------------------------
21+
stddev wholestage off 8667 / 8851 12.1 82.7 1.0X
22+
stddev wholestage on 1266 / 1273 82.8 12.1 6.8X
23+
24+
OpenJDK 64-Bit Server VM 1.8.0_181-b13 on Linux 3.10.0-862.3.2.el7.x86_64
25+
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
26+
kurtosis: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
27+
------------------------------------------------------------------------------------------------
28+
kurtosis wholestage off 41218 / 41231 2.5 393.1 1.0X
29+
kurtosis wholestage on 1347 / 1357 77.8 12.8 30.6X
30+
31+
32+
================================================================================================
33+
aggregate with linear keys
34+
================================================================================================
35+
36+
OpenJDK 64-Bit Server VM 1.8.0_181-b13 on Linux 3.10.0-862.3.2.el7.x86_64
37+
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
38+
Aggregate w keys: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
39+
------------------------------------------------------------------------------------------------
40+
codegen = F 9309 / 9389 9.0 111.0 1.0X
41+
codegen = T hashmap = F 4417 / 4435 19.0 52.7 2.1X
42+
codegen = T hashmap = T 1289 / 1298 65.1 15.4 7.2X
43+
44+
45+
================================================================================================
46+
aggregate with randomized keys
47+
================================================================================================
48+
49+
OpenJDK 64-Bit Server VM 1.8.0_181-b13 on Linux 3.10.0-862.3.2.el7.x86_64
50+
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
51+
Aggregate w keys: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
52+
------------------------------------------------------------------------------------------------
53+
codegen = F 11424 / 11426 7.3 136.2 1.0X
54+
codegen = T hashmap = F 6441 / 6496 13.0 76.8 1.8X
55+
codegen = T hashmap = T 2333 / 2344 36.0 27.8 4.9X
56+
57+
58+
================================================================================================
59+
aggregate with string key
60+
================================================================================================
61+
62+
OpenJDK 64-Bit Server VM 1.8.0_181-b13 on Linux 3.10.0-862.3.2.el7.x86_64
63+
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
64+
Aggregate w string key: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
65+
------------------------------------------------------------------------------------------------
66+
codegen = F 4751 / 4890 4.4 226.5 1.0X
67+
codegen = T hashmap = F 3146 / 3182 6.7 150.0 1.5X
68+
codegen = T hashmap = T 2211 / 2261 9.5 105.4 2.1X
69+
70+
71+
================================================================================================
72+
aggregate with decimal key
73+
================================================================================================
74+
75+
OpenJDK 64-Bit Server VM 1.8.0_181-b13 on Linux 3.10.0-862.3.2.el7.x86_64
76+
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
77+
Aggregate w decimal key: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
78+
------------------------------------------------------------------------------------------------
79+
codegen = F 3029 / 3062 6.9 144.4 1.0X
80+
codegen = T hashmap = F 1534 / 1569 13.7 73.2 2.0X
81+
codegen = T hashmap = T 575 / 578 36.5 27.4 5.3X
82+
83+
84+
================================================================================================
85+
aggregate with multiple key types
86+
================================================================================================
87+
88+
OpenJDK 64-Bit Server VM 1.8.0_181-b13 on Linux 3.10.0-862.3.2.el7.x86_64
89+
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
90+
Aggregate w multiple keys: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
91+
------------------------------------------------------------------------------------------------
92+
codegen = F 7506 / 7521 2.8 357.9 1.0X
93+
codegen = T hashmap = F 4791 / 4808 4.4 228.5 1.6X
94+
codegen = T hashmap = T 3553 / 3585 5.9 169.4 2.1X
95+
96+
97+
================================================================================================
98+
max function bytecode size of wholestagecodegen
99+
================================================================================================
100+
101+
OpenJDK 64-Bit Server VM 1.8.0_181-b13 on Linux 3.10.0-862.3.2.el7.x86_64
102+
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
103+
max function bytecode size: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
104+
------------------------------------------------------------------------------------------------
105+
codegen = F 608 / 656 1.1 927.1 1.0X
106+
codegen = T hugeMethodLimit = 10000 402 / 419 1.6 613.5 1.5X
107+
codegen = T hugeMethodLimit = 1500 616 / 619 1.1 939.9 1.0X
108+
109+
110+
================================================================================================
111+
cube
112+
================================================================================================
113+
114+
OpenJDK 64-Bit Server VM 1.8.0_181-b13 on Linux 3.10.0-862.3.2.el7.x86_64
115+
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
116+
cube: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
117+
------------------------------------------------------------------------------------------------
118+
cube wholestage off 3229 / 3237 1.6 615.9 1.0X
119+
cube wholestage on 1285 / 1306 4.1 245.2 2.5X
120+
121+
122+
================================================================================================
123+
hash and BytesToBytesMap
124+
================================================================================================
125+
126+
OpenJDK 64-Bit Server VM 1.8.0_181-b13 on Linux 3.10.0-862.3.2.el7.x86_64
127+
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
128+
BytesToBytesMap: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
129+
------------------------------------------------------------------------------------------------
130+
UnsafeRowhash 328 / 330 64.0 15.6 1.0X
131+
murmur3 hash 167 / 167 125.4 8.0 2.0X
132+
fast hash 84 / 85 249.0 4.0 3.9X
133+
arrayEqual 192 / 192 109.3 9.1 1.7X
134+
Java HashMap (Long) 144 / 147 145.9 6.9 2.3X
135+
Java HashMap (two ints) 147 / 153 142.3 7.0 2.2X
136+
Java HashMap (UnsafeRow) 785 / 788 26.7 37.4 0.4X
137+
LongToUnsafeRowMap (opt=false) 456 / 457 46.0 21.8 0.7X
138+
LongToUnsafeRowMap (opt=true) 125 / 125 168.3 5.9 2.6X
139+
BytesToBytesMap (off Heap) 885 / 885 23.7 42.2 0.4X
140+
BytesToBytesMap (on Heap) 860 / 864 24.4 41.0 0.4X
141+
Aggregate HashMap 56 / 56 373.9 2.7 5.8X
142+
143+

0 commit comments

Comments
 (0)