SHA3 keccak256 is super slow

[bogdan]

I am using hash function for photo and video files and it is very slow comparing to analogies in other programming languages like Javascript:

                    [1, 10, 100, 1000, 10000, 100_000, 1_000_000, 10_000_000].forEach { size in
                        _ = AppLogger.shared.benchmark("Calcualate \(size)") {
                            Data(SHA3(variant: .keccak256).calculate(for: [UInt8](Data(repeating: 0x01, count: size))))
                        }
                    }

APP| (2ms) Calcualate 1
APP| (0ms) Calcualate 10
APP| (0ms) Calcualate 100
APP| (1ms) Calcualate 1000
APP| (10ms) Calcualate 10000
APP| (98ms) Calcualate 100000
APP| (946ms) Calcualate 1000000
APP| (9462ms) Calcualate 10000000

Is there any way to improve performance?

[krzyzanowskim]

make sure you build "release". optimised compilation should be significantly faster.

swift debug build is very slow

[bogdan]

I checked the release build on my modern M1 desktop machine. Here are the stats:

APP| (1ms) Calcualate 1
APP| (0ms) Calcualate 10
APP| (0ms) Calcualate 100
APP| (1ms) Calcualate 1000
APP| (10ms) Calcualate 10000
APP| (92ms) Calcualate 100000
APP| (912ms) Calcualate 1000000

It still feels slow.

I tested 11MB image file on JavaScript using this implementation:
https://github.com/ethers-io/ethers.js/blob/48cdcef2925ffca4f60412697791dac268f06d0d/src.ts/crypto/keccak.ts#L44

It is 471ms in a browser: .
So the compiled build is 2 times slower than the slowest environment ever made.

Why would that be the case?

[krzyzanowskim]

slower than the slowest environment ever made

what is it? the "noble-hashes" aint slow (that would be Python)

Why would that be the case?

🤷🏻 I didn't debug it now. but feel free to give it a try

[krzyzanowskim]

I run a quick test on my machine (M3 Air) and for 10_000_000 bytes. It's ~12ms that looks good at first glance. Feel free to run the same test on your end to compare output:

  func testSHA3Perf() {
    let arr = Array<UInt8>(repeating: 0x01, count: 10_000_000)
    measureMetrics([XCTPerformanceMetric.wallClockTime], automaticallyStartMeasuring: true) {
      _ = SHA3(variant: .keccak256).calculate(for: arr)
    }
  }