Skip to content

Benchmarks

Patrick Favre-Bulle edited this page Aug 14, 2022 · 8 revisions

Base64

i7 7700K

# JMH version: 1.21
# VM version: JDK 1.8.0_172, Java HotSpot(TM) 64-Bit Server VM, 25.172-b11
# Warmup: 3 iterations, 5 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# i7 7700K / 24G

EncodingJmhBenchmark.encodeBase64Apache              1  thrpt    5   1318131,937 ±   16339,198  ops/s
EncodingJmhBenchmark.encodeBase64Apache             16  thrpt    5   1069180,566 ±    1174,315  ops/s
EncodingJmhBenchmark.encodeBase64Apache             32  thrpt    5    952877,557 ±    1034,081  ops/s
EncodingJmhBenchmark.encodeBase64Apache            128  thrpt    5    470671,111 ±    3890,514  ops/s
EncodingJmhBenchmark.encodeBase64Apache            512  thrpt    5    172110,829 ±     213,959  ops/s
EncodingJmhBenchmark.encodeBase64Apache        1000000  thrpt    5       102,850 ±       3,624  ops/s
EncodingJmhBenchmark.encodeBase64Guava               1  thrpt    5  10807902,000 ±  909701,319  ops/s
EncodingJmhBenchmark.encodeBase64Guava              16  thrpt    5   5703135,415 ±    7063,849  ops/s
EncodingJmhBenchmark.encodeBase64Guava              32  thrpt    5   3875269,919 ±    5242,293  ops/s
EncodingJmhBenchmark.encodeBase64Guava             128  thrpt    5   1378103,987 ±    1333,459  ops/s
EncodingJmhBenchmark.encodeBase64Guava             512  thrpt    5    346530,050 ±     407,467  ops/s
EncodingJmhBenchmark.encodeBase64Guava         1000000  thrpt    5       232,082 ±       4,882  ops/s
EncodingJmhBenchmark.encodeBase64Okio                1  thrpt    5  14068294,672 ± 3751801,084  ops/s
EncodingJmhBenchmark.encodeBase64Okio               16  thrpt    5   6973717,092 ±   97091,658  ops/s
EncodingJmhBenchmark.encodeBase64Okio               32  thrpt    5   4641793,472 ±  107239,983  ops/s
EncodingJmhBenchmark.encodeBase64Okio              128  thrpt    5   1771447,895 ±   28595,096  ops/s
EncodingJmhBenchmark.encodeBase64Okio              512  thrpt    5    276164,010 ±    6808,837  ops/s
EncodingJmhBenchmark.encodeBase64Okio          1000000  thrpt    5       102,610 ±       5,899  ops/s
EncodingJmhBenchmark.encodeBase64OpenJdk             1  thrpt    5  11713907,451 ± 1928162,113  ops/s
EncodingJmhBenchmark.encodeBase64OpenJdk            16  thrpt    5   6816928,904 ±   30601,543  ops/s
EncodingJmhBenchmark.encodeBase64OpenJdk            32  thrpt    5   5087048,151 ±   14096,769  ops/s
EncodingJmhBenchmark.encodeBase64OpenJdk           128  thrpt    5   1905604,168 ±   40709,235  ops/s
EncodingJmhBenchmark.encodeBase64OpenJdk           512  thrpt    5    568632,043 ±     428,865  ops/s
EncodingJmhBenchmark.encodeBase64OpenJdk       1000000  thrpt    5       270,155 ±      29,743  ops/s
Benchmark                                     (byteLength)   Mode  Cnt        Score       Error  Units
EncodingJmhBenchmark.decodeOnlyBase64Okio              512  thrpt    3   275981,072 ±  4374,638  ops/s
EncodingJmhBenchmark.decodeOnlyBase64Okio          1000000  thrpt    3      130,277 ±     5,843  ops/s
EncodingJmhBenchmark.decodeOnlyBase64OpenJdk           512  thrpt    3   920051,263 ± 17679,192  ops/s
EncodingJmhBenchmark.decodeOnlyBase64OpenJdk       1000000  thrpt    3      424,115 ±     8,624  ops/s
EncodingJmhBenchmark.encodeOnlyBase64Okio              512  thrpt    3   973920,818 ± 23511,284  ops/s
EncodingJmhBenchmark.encodeOnlyBase64Okio          1000000  thrpt    3      485,362 ±    10,893  ops/s
EncodingJmhBenchmark.encodeOnlyBase64OpenJdk           512  thrpt    3  1345013,354 ± 49538,081  ops/s
EncodingJmhBenchmark.encodeOnlyBase64OpenJdk       1000000  thrpt    3      751,239 ±    64,040  ops/s

Hex Encoding

Ryzen 5900X

JDK 17

Result "at.favre.lib.bytes.EncodingHexJmhBenchmark.encodeStackOverflowCode2":
  1122,293 ±(99.9%) 22,228 ops/s [Average]
  (min, avg, max) = (1120,894, 1122,293, 1123,116), stdev = 1,218
  CI (99.9%): [1100,065, 1144,521] (assumes normal distribution)


# Run complete. Total time: 00:25:40

REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.

NOTE: Current JVM experimentally supports Compiler Blackholes, and they are in use. Please exercise
extra caution when trusting the results, look into the generated code to check the benchmark still
works, and factor in a small probability of new VM bugs. Additionally, while comparisons between
different JVMs are already problematic, the performance difference caused by different Blackhole
modes can be very significant. Please make sure you use the consistent Blackhole mode for comparisons.

Benchmark                                         (byteLength)   Mode  Cnt         Score          Error  Units
EncodingHexJmhBenchmark.encodeBigInteger                     4  thrpt    3  13428556,267 ±  2485252,488  ops/s
EncodingHexJmhBenchmark.encodeBigInteger                     8  thrpt    3   4944548,258 ±    36914,459  ops/s
EncodingHexJmhBenchmark.encodeBigInteger                    16  thrpt    3   2941403,105 ±    53464,757  ops/s
EncodingHexJmhBenchmark.encodeBigInteger                    32  thrpt    3   1389448,671 ±   133105,711  ops/s
EncodingHexJmhBenchmark.encodeBigInteger                   128  thrpt    3    242096,448 ±     4847,126  ops/s
EncodingHexJmhBenchmark.encodeBigInteger                   512  thrpt    3     43931,400 ±      408,381  ops/s
EncodingHexJmhBenchmark.encodeBigInteger               1000000  thrpt    3         5,335 ±        0,068  ops/s
EncodingHexJmhBenchmark.encodeBytesLib                       4  thrpt    3  49699573,884 ±   718083,037  ops/s
EncodingHexJmhBenchmark.encodeBytesLib                       8  thrpt    3  39837245,684 ±   194783,743  ops/s
EncodingHexJmhBenchmark.encodeBytesLib                      16  thrpt    3  31724981,604 ±   244938,749  ops/s
EncodingHexJmhBenchmark.encodeBytesLib                      32  thrpt    3  22786906,028 ±  1107024,474  ops/s
EncodingHexJmhBenchmark.encodeBytesLib                     128  thrpt    3   6197028,261 ±   704040,867  ops/s
EncodingHexJmhBenchmark.encodeBytesLib                     512  thrpt    3   1504695,048 ±    24497,811  ops/s
EncodingHexJmhBenchmark.encodeBytesLib                 1000000  thrpt    3       930,265 ±      130,086  ops/s
EncodingHexJmhBenchmark.encodeOldBytesLib                    4  thrpt    3  23410373,676 ±  7239888,483  ops/s
EncodingHexJmhBenchmark.encodeOldBytesLib                    8  thrpt    3  19364059,826 ± 29244077,987  ops/s
EncodingHexJmhBenchmark.encodeOldBytesLib                   16  thrpt    3  16227097,506 ±   424439,581  ops/s
EncodingHexJmhBenchmark.encodeOldBytesLib                   32  thrpt    3   7768193,947 ±   132317,714  ops/s
EncodingHexJmhBenchmark.encodeOldBytesLib                  128  thrpt    3   3279449,595 ±    34562,168  ops/s
EncodingHexJmhBenchmark.encodeOldBytesLib                  512  thrpt    3    929069,735 ±   138616,676  ops/s
EncodingHexJmhBenchmark.encodeOldBytesLib              1000000  thrpt    3       138,958 ±        0,908  ops/s
EncodingHexJmhBenchmark.encodeSchallinsky                    4  thrpt    3  48166926,547 ±  3266017,792  ops/s
EncodingHexJmhBenchmark.encodeSchallinsky                    8  thrpt    3  39284153,920 ± 11076984,360  ops/s
EncodingHexJmhBenchmark.encodeSchallinsky                   16  thrpt    3  31889491,493 ±  2807579,608  ops/s
EncodingHexJmhBenchmark.encodeSchallinsky                   32  thrpt    3  22880701,641 ±  2424083,371  ops/s
EncodingHexJmhBenchmark.encodeSchallinsky                  128  thrpt    3   7677099,242 ±  2594554,970  ops/s
EncodingHexJmhBenchmark.encodeSchallinsky                  512  thrpt    3   1862006,524 ±    71913,256  ops/s
EncodingHexJmhBenchmark.encodeSchallinsky              1000000  thrpt    3       802,081 ±      175,500  ops/s
EncodingHexJmhBenchmark.encodeStackOverflowCode1             4  thrpt    3  47413050,123 ±  2725992,565  ops/s
EncodingHexJmhBenchmark.encodeStackOverflowCode1             8  thrpt    3  40677495,172 ±  2817227,931  ops/s
EncodingHexJmhBenchmark.encodeStackOverflowCode1            16  thrpt    3  32959351,890 ±  3106728,795  ops/s
EncodingHexJmhBenchmark.encodeStackOverflowCode1            32  thrpt    3  23611836,744 ±   877589,728  ops/s
EncodingHexJmhBenchmark.encodeStackOverflowCode1           128  thrpt    3   7882848,672 ±   447275,782  ops/s
EncodingHexJmhBenchmark.encodeStackOverflowCode1           512  thrpt    3   1882316,791 ±    36826,620  ops/s
EncodingHexJmhBenchmark.encodeStackOverflowCode1       1000000  thrpt    3       949,783 ±       18,475  ops/s
EncodingHexJmhBenchmark.encodeStackOverflowCode2             4  thrpt    3  51237081,006 ±  6255497,646  ops/s
EncodingHexJmhBenchmark.encodeStackOverflowCode2             8  thrpt    3  42742454,781 ±  5897169,657  ops/s
EncodingHexJmhBenchmark.encodeStackOverflowCode2            16  thrpt    3  34528484,827 ±  1548421,555  ops/s
EncodingHexJmhBenchmark.encodeStackOverflowCode2            32  thrpt    3  25978276,665 ±  3060288,487  ops/s
EncodingHexJmhBenchmark.encodeStackOverflowCode2           128  thrpt    3   9887505,224 ± 10028617,360  ops/s
EncodingHexJmhBenchmark.encodeStackOverflowCode2           512  thrpt    3   2250028,032 ±   386443,795  ops/s
EncodingHexJmhBenchmark.encodeStackOverflowCode2       1000000  thrpt    3      1122,293 ±       22,228  ops/s

JDK 8

Result "at.favre.lib.bytes.EncodingHexJmhBenchmark.encodeStackOverflowCode2": 1049,595 ±(99.9%) 93,412 ops/s [Average] (min, avg, max) = (1045,195, 1049,595, 1055,215), stdev = 5,120 CI (99.9%): [956,183, 1143,007] (assumes normal distribution)

Run complete. Total time: 00:25:54

REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial experiments, perform baseline and negative tests that provide experimental control, make sure the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts. Do not assume the numbers tell you what you want them to tell.

Benchmark (byteLength) Mode Cnt Score Error Units EncodingHexJmhBenchmark.encodeBigInteger 4 thrpt 3 9333868,849 ± 3894397,371 ops/s EncodingHexJmhBenchmark.encodeBigInteger 8 thrpt 3 4047220,801 ± 429446,689 ops/s EncodingHexJmhBenchmark.encodeBigInteger 16 thrpt 3 2407526,211 ± 114239,702 ops/s EncodingHexJmhBenchmark.encodeBigInteger 32 thrpt 3 1220847,723 ± 42102,349 ops/s EncodingHexJmhBenchmark.encodeBigInteger 128 thrpt 3 211012,498 ± 8304,562 ops/s EncodingHexJmhBenchmark.encodeBigInteger 512 thrpt 3 40888,638 ± 1577,013 ops/s EncodingHexJmhBenchmark.encodeBigInteger 1000000 thrpt 3 5,401 ± 0,149 ops/s EncodingHexJmhBenchmark.encodeBytesLib 4 thrpt 3 30613483,876 ± 537736,621 ops/s EncodingHexJmhBenchmark.encodeBytesLib 8 thrpt 3 21432227,942 ± 2686923,886 ops/s EncodingHexJmhBenchmark.encodeBytesLib 16 thrpt 3 19390972,849 ± 665525,732 ops/s EncodingHexJmhBenchmark.encodeBytesLib 32 thrpt 3 15213485,653 ± 729118,879 ops/s EncodingHexJmhBenchmark.encodeBytesLib 128 thrpt 3 6591738,552 ± 197300,348 ops/s EncodingHexJmhBenchmark.encodeBytesLib 512 thrpt 3 1772463,045 ± 457442,875 ops/s EncodingHexJmhBenchmark.encodeBytesLib 1000000 thrpt 3 896,453 ± 29,679 ops/s EncodingHexJmhBenchmark.encodeOldBytesLib 4 thrpt 3 20947931,772 ± 23857593,063 ops/s EncodingHexJmhBenchmark.encodeOldBytesLib 8 thrpt 3 16514682,007 ± 695689,124 ops/s EncodingHexJmhBenchmark.encodeOldBytesLib 16 thrpt 3 12196405,364 ± 882219,900 ops/s EncodingHexJmhBenchmark.encodeOldBytesLib 32 thrpt 3 8589420,490 ± 1266396,533 ops/s EncodingHexJmhBenchmark.encodeOldBytesLib 128 thrpt 3 3113779,210 ± 96236,236 ops/s EncodingHexJmhBenchmark.encodeOldBytesLib 512 thrpt 3 838036,695 ± 9111,954 ops/s EncodingHexJmhBenchmark.encodeOldBytesLib 1000000 thrpt 3 143,409 ± 77,980 ops/s EncodingHexJmhBenchmark.encodeSchallinsky 4 thrpt 3 28925917,282 ± 1909118,263 ops/s EncodingHexJmhBenchmark.encodeSchallinsky 8 thrpt 3 24633538,637 ± 1976249,711 ops/s EncodingHexJmhBenchmark.encodeSchallinsky 16 thrpt 3 18627196,959 ± 161703,637 ops/s EncodingHexJmhBenchmark.encodeSchallinsky 32 thrpt 3 14708733,929 ± 5488378,578 ops/s EncodingHexJmhBenchmark.encodeSchallinsky 128 thrpt 3 6640536,630 ± 208149,440 ops/s EncodingHexJmhBenchmark.encodeSchallinsky 512 thrpt 3 1865910,408 ± 36249,944 ops/s EncodingHexJmhBenchmark.encodeSchallinsky 1000000 thrpt 3 875,940 ± 608,061 ops/s EncodingHexJmhBenchmark.encodeStackOverflowCode1 4 thrpt 3 30130846,282 ± 3670664,087 ops/s EncodingHexJmhBenchmark.encodeStackOverflowCode1 8 thrpt 3 25747757,359 ± 1837807,768 ops/s EncodingHexJmhBenchmark.encodeStackOverflowCode1 16 thrpt 3 19435881,479 ± 4796200,589 ops/s EncodingHexJmhBenchmark.encodeStackOverflowCode1 32 thrpt 3 15339205,263 ± 481716,466 ops/s EncodingHexJmhBenchmark.encodeStackOverflowCode1 128 thrpt 3 6732995,825 ± 150642,614 ops/s EncodingHexJmhBenchmark.encodeStackOverflowCode1 512 thrpt 3 1891953,715 ± 26048,908 ops/s EncodingHexJmhBenchmark.encodeStackOverflowCode1 1000000 thrpt 3 887,777 ± 125,344 ops/s EncodingHexJmhBenchmark.encodeStackOverflowCode2 4 thrpt 3 31648591,182 ± 953394,103 ops/s EncodingHexJmhBenchmark.encodeStackOverflowCode2 8 thrpt 3 26786077,183 ± 225195,917 ops/s EncodingHexJmhBenchmark.encodeStackOverflowCode2 16 thrpt 3 20598995,852 ± 1062913,974 ops/s EncodingHexJmhBenchmark.encodeStackOverflowCode2 32 thrpt 3 15871946,268 ± 992030,494 ops/s EncodingHexJmhBenchmark.encodeStackOverflowCode2 128 thrpt 3 7375691,213 ± 2440196,295 ops/s EncodingHexJmhBenchmark.encodeStackOverflowCode2 512 thrpt 3 2147273,997 ± 32629,513 ops/s EncodingHexJmhBenchmark.encodeStackOverflowCode2 1000000 thrpt 3 1049,595 ± 93,412 ops/s

Process finished with exit code 0

Clone this wiki locally