Rearrange SDK client supported checksum algorithms in the priority order of their performance

[trivikr]

Describe the feature

Rearrange SDK client supported checksum algorithms in the priority order of their performance

Use Case

When flexible checksums was implemented, the priority order algorithms were predetermined from different SDK, and they were not profiled.

There's a ToDo

aws-sdk-js-v3/packages/middleware-flexible-checksums/src/types.ts

Line 15 in 118987c

* ToDo: update the priority order based on profiling of JavaScript implementations.

Ordering SDK client supported checksum algorithms in the priority order of their performance will improve the time take for validation.

Proposed Solution

Write a benchmark similar to https://github.com/trivikr/benchmark-crc32 for all supported algorithms, and order them based on performance

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

SDK version used

All

Environment details (OS name and version, etc.)

Node.js

[trivikr]

The benchmarks comparing checksum implementations used by AWS SDK for JavaScript were added in trivikr/benchmark-checksums#1

Copying for visibility

Benchmark for buffer of size 16 KB:
crc32 x 5,847 ops/sec ±0.94% (97 runs sampled)
crc32c x 5,696 ops/sec ±2.56% (95 runs sampled)
sha1 x 115,813 ops/sec ±3.62% (89 runs sampled)
sha256 x 117,583 ops/sec ±2.80% (87 runs sampled)
Fastest is sha256,sha1

Benchmark for buffer of size 64 KB:
crc32 x 1,438 ops/sec ±0.58% (97 runs sampled)
crc32c x 1,378 ops/sec ±4.90% (93 runs sampled)
sha1 x 33,010 ops/sec ±2.74% (91 runs sampled)
sha256 x 34,076 ops/sec ±1.57% (91 runs sampled)
Fastest is sha256

Benchmark for buffer of size 256 KB:
crc32 x 361 ops/sec ±1.13% (93 runs sampled)
crc32c x 367 ops/sec ±0.43% (94 runs sampled)
sha1 x 9,096 ops/sec ±0.43% (101 runs sampled)
sha256 x 9,178 ops/sec ±0.58% (100 runs sampled)
Fastest is sha256

Benchmark for buffer of size 1024 KB:
crc32 x 90.94 ops/sec ±0.36% (79 runs sampled)
crc32c x 90.49 ops/sec ±0.47% (79 runs sampled)
sha1 x 2,261 ops/sec ±1.45% (96 runs sampled)
sha256 x 2,111 ops/sec ±7.35% (91 runs sampled)
Fastest is sha1

Based on benchmark results, the new priority order will be as follows:

export const PRIORITY_ORDER_ALGORITHMS = [
  ChecksumAlgorithm.SHA256,
  ChecksumAlgorithm.SHA1,
  ChecksumAlgorithm.CRC32,
  ChecksumAlgorithm.CRC32C,
];

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.