Skip to content

Commit b81798d

Browse files
MetaphormeMetaphorme
committed
Update XXHASH and benchmark.
1 parent 33ea274 commit b81798d

File tree

5 files changed

+158
-66
lines changed

5 files changed

+158
-66
lines changed

‎README.md‎

Lines changed: 81 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -8,18 +8,18 @@
88
[中文文档](./README_zh.md)
99

1010
ezcheck(easy check) is an easy-to-use, lightweight, cross-platform, and high-performance tool for calculating,
11-
comparing, and verifying hash of strings and files.
11+
comparing, and verifying hash of strings and files. Designed to prevent content tampering and ensure file integrity.
1212

1313
ezcheck have three backends: [ring](https://docs.rs/ring), [hashes](https://docs.rs/hashes) and mix
1414
backend([ring](https://docs.rs/ring) and [hashes](https://docs.rs/hashes)), and you can only choose
1515
one of them. The main differences between them are:
1616

17-
| Features | ring | hashes | mix |
18-
|----------------------|------------------------------------------------------------|-----------------------------------------------------------------|-----------------------------------------------------------------|
19-
| Speed | Fast. | About 5 times slower than ring. | Use the fastest backend that supports the algorithm. |
20-
| Supported algorithms | SHA256, SHA384, SHA512, SHA512/256 | MD2, MD4, MD5, SHA1, SHA224, SHA256, SHA384, SHA512, SHA512/256 | MD2, MD4, MD5, SHA1, SHA224, SHA256, SHA384, SHA512, SHA512/256 |
21-
| Implement languages | Assembly, Rust, C and etc.. | Rust | Assembly, Rust, C and etc.. |
22-
| Compatibility | May not work on every machine with different architecture. | Works well with Rust. | Same to ring. |
17+
| Features | ring | hashes | mix(Recommended) |
18+
|----------------------|------------------------------------------------------------|-----------------------------------------------------------------|----------------------------------------------------------------------------------------|
19+
| Speed | Fast. | About 5 times slower than ring. | Use the fastest backend that supports the algorithm. |
20+
| Supported algorithms | SHA256, SHA384, SHA512, SHA512/256 | MD2, MD4, MD5, SHA1, SHA224, SHA256, SHA384, SHA512, SHA512/256 | MD2, MD4, MD5, SHA1, SHA224, SHA256, SHA384, SHA512, SHA512/256, XXH32, XXH64, XXH3_64 |
21+
| Implement languages | Assembly, Rust, C and etc.. | Rust | Assembly, Rust, C and etc.. |
22+
| Compatibility | May not work on every machine with different architecture. | Works well with Rust. | Same to ring. |
2323

2424
❗️ To achieve both fastest speed and maximum compatibility, the default backend is **mix backend**.
2525

@@ -98,6 +98,9 @@ Supported hash algorithms of different backends:
9898
| SHA384 | SHA384 | SHA384 (ring backend) |
9999
| SHA512 | SHA512 | SHA512 (ring backend) |
100100
| SHA512/256 | SHA512/256 | SHA512/256 (ring backend) |
101+
| | | XXH32 |
102+
| | | XXH64 |
103+
| | | XXH3_64 |
101104

102105
### Calculate
103106

@@ -198,37 +201,80 @@ image.jpg: SHA256 OK
198201

199202
## Benchmark
200203

201-
### Method
204+
### SHA256 Benchmark Tests
205+
206+
#### Method
207+
208+
* Device: MacBook Air M1 8GB
209+
210+
* Steps
211+
212+
1. Run and repeat 3 times:
213+
```bash
214+
$ count=10000 # Test size = 1MiB * $count
215+
$ # Bare, Speed of generating the data
216+
$ dd if=/dev/zero bs=1M count=$count | pv > /dev/null
217+
$ # ezcheck-hashes
218+
$ dd if=/dev/zero bs=1M count=$count | pv | ./ezcheck-hashes calculate sha256 -f -
219+
$ # ezcheck-ring
220+
$ dd if=/dev/zero bs=1M count=$count | pv | ./ezcheck-ring calculate sha256 -f -
221+
$ # sha256sum
222+
$ dd if=/dev/zero bs=1M count=$count | pv | sha256sum
223+
```
224+
225+
2. Calculate the average value.
226+
227+
#### Result
228+
229+
| Implementation / Speed(GiB/s) / Test size(MiB) | 1 | 100 | 500 | 1000 | 5000 | 10000 |
230+
|------------------------------------------------|------|------|------|------|------|-------|
231+
| Bare (Speed of generating the data) | 2.13 | 3.02 | 4.59 | 5.31 | 5.97 | 6.07 |
232+
| ezcheck-hashes | 0.13 | 0.28 | 0.29 | 0.30 | 0.30 | 0.30 |
233+
| ezcheck-ring | 0.58 | 1.24 | 1.57 | 1.63 | 1.68 | 1.68 |
234+
| sha256sum | 0.73 | 1.26 | 1.63 | 1.69 | 1.75 | 1.81 |
235+
236+
![benchmark](./benchmark-sha256.png)
237+
238+
### Speed vs. Algorithms, Implementations
239+
240+
#### Method
202241

203242
* Device: MacBook Air M1 8GB
204243

205-
* Steps:
206-
207-
1. Run and repeat 3 times:
208-
```bash
209-
$ count=10000 # Test size = 1MiB * $count
210-
$ # Bare, Speed of generating the data
211-
$ dd if=/dev/zero bs=1M count=$count | pv > /dev/null
212-
$ # ezcheck-hashes
213-
$ dd if=/dev/zero bs=1M count=$count | pv | ./ezcheck-hashes calculate sha256 -f -
214-
$ # ezcheck-ring
215-
$ dd if=/dev/zero bs=1M count=$count | pv | ./ezcheck-ring calculate sha256 -f -
216-
$ # sha256sum
217-
$ dd if=/dev/zero bs=1M count=$count | pv | sha256sum
218-
```
219-
220-
2. Calculate the average value.
221-
222-
### Result
223-
224-
| Command/Speed(GiB/s)/Test size(MiB) | 1 | 100 | 500 | 1000 | 5000 | 10000 |
225-
|-------------------------------------|------|------|------|------|------|-------|
226-
| Bare (Speed of generating the data) | 2.13 | 3.02 | 4.59 | 5.31 | 5.97 | 6.07 |
227-
| ezcheck-hashes | 0.13 | 0.28 | 0.29 | 0.30 | 0.30 | 0.30 |
228-
| ezcheck-ring | 0.58 | 1.24 | 1.57 | 1.63 | 1.68 | 1.68 |
229-
| sha256sum | 0.73 | 1.26 | 1.63 | 1.69 | 1.75 | 1.81 |
230-
231-
![benchmark](./benchmark.png)
244+
* Steps
245+
246+
1. Run:
247+
```bash
248+
$ algorithm=sha256
249+
$ # ezcheck-hashes
250+
$ dd if=/dev/zero bs=1M count=10000 | pv | ./ezcheck-hashes calculate $algorithm -f -
251+
$ # ezcheck-ring
252+
$ dd if=/dev/zero bs=1M count=10000 | pv | ./ezcheck-ring calculate $algorithm -f -
253+
$ # ezcheck-mix
254+
$ dd if=/dev/zero bs=1M count=10000 | pv | ./ezcheck-mix calculate $algorithm -f -
255+
```
256+
2. Calculate the average value.
257+
258+
#### Result
259+
260+
| Algorithms / Speed(GiB/s) / Implementations | ring | hashes | mix |
261+
|---------------------------------------------|--------|----------|----------|
262+
| MD2 | null* | 0.00896 | 0.00896 |
263+
| MD4 | null* | 0.852 | 0.852 |
264+
| MD5 | null* | 0.549 | 0.549 |
265+
| SHA1 | null* | 0.802 | 0.802 |
266+
| SHA224 | null* | 0.299 | 0.299 |
267+
| SHA256 | 1.69 | 0.298 | 1.70 |
268+
| SHA384 | 1.12 | 0.473 | 1.13 |
269+
| SHA512 | 1.13 | 0.473 | 1.13 |
270+
| SHA512/256 | 1.13 | 0.473 | 1.13 |
271+
| XXHASH32 | null* | null* | 2.45 |
272+
| XXHASH64 | null* | null* | 3.27 |
273+
| XXHASH3_64 | null* | null* | 3.65 |
274+
275+
_null*: The algorithm is not implemented in this implementation._
276+
277+
![benchmark](./benchmark-algorithms-implementations.png)
232278

233279
## License
234280

‎README_zh.md‎

Lines changed: 77 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -5,17 +5,17 @@
55
[![Crates.io Version](https://img.shields.io/crates/v/ezcheck?logo=rust)](https://crates.io/crates/ezcheck)
66
[![Mirror Gitee](https://img.shields.io/badge/Mirror-Gitee-blue?logo=gitee)](https://gitee.com/metaphorme/ezcheck)
77

8-
ezcheck（或 easy check）是一个易于使用、轻量、跨平台、高性能的用于计算、对比和验证字符串或文件哈希值的工具。
8+
ezcheck（或 easy check）是一个轻量、高性能、跨平台、易于使用的用于计算、对比和验证字符串或文件哈希值的工具，用于防止内容篡改和确保文件的完整性。
99

1010
ezcheck 有三个后端：[ring](https://docs.rs/ring)，[hashes](https://docs.rs/hashes)
1111
和混合后端（mix backend，同时使用[ring](https://docs.rs/ring)，[hashes](https://docs.rs/hashes)），并且您只能选择其中一个。这些后端的主要差异在于：
1212

13-
| 特点 | ring | hashes | 混合后端（mix） |
14-
|-------|------------------------------------|-----------------------------------------------------------------|-----------------------------------------------------------------|
15-
| 速度 | 非常快。 | 大约比 ring 慢五倍。 | 使用支持此算法的最快后端。 |
16-
| 支持的算法 | SHA256, SHA384, SHA512, SHA512/256 | MD2, MD4, MD5, SHA1, SHA224, SHA256, SHA384, SHA512, SHA512/256 | MD2, MD4, MD5, SHA1, SHA224, SHA256, SHA384, SHA512, SHA512/256 |
17-
| 实现语言 | Assembly, Rust, C 等。 | Rust | Assembly, Rust, C 等。 |
18-
| 兼容性 | 可能无法在一些系统和架构上工作。 | 和 Rust 兼容性一致。 | 与 ring 相同。 |
13+
| 特点 | ring | hashes | mix（混合后端，推荐使用） |
14+
|-------|------------------------------------|-----------------------------------------------------------------|----------------------------------------------------------------------------------------|
15+
| 速度 | 非常快。 | 大约比 ring 慢五倍。 | 使用支持此算法的最快后端。 |
16+
| 支持的算法 | SHA256, SHA384, SHA512, SHA512/256 | MD2, MD4, MD5, SHA1, SHA224, SHA256, SHA384, SHA512, SHA512/256 | MD2, MD4, MD5, SHA1, SHA224, SHA256, SHA384, SHA512, SHA512/256, XXH32, XXH64, XXH3_64 |
17+
| 实现语言 | Assembly, Rust, C 等。 | Rust | Assembly, Rust, C 等。 |
18+
| 兼容性 | 可能无法在一些系统和架构上工作。 | 和 Rust 兼容性一致。 | 与 ring 相同。 |
1919

2020
❗️ 为了兼顾最快的速度和最大的算法兼容性，默认后端是混合后端（mix backend）。
2121

@@ -93,6 +93,9 @@ $ cargo test --no-default-features --features hashes_backend # hashes backend
9393
| SHA384 | SHA384 | SHA384 (ring 后端) |
9494
| SHA512 | SHA512 | SHA512 (ring 后端) |
9595
| SHA512/256 | SHA512/256 | SHA512/256 (ring 后端) |
96+
| | | XXH32 |
97+
| | | XXH64 |
98+
| | | XXH3_64 |
9699

97100
### 计算
98101

@@ -193,37 +196,80 @@ image.jpg: SHA256 OK
193196

194197
## 基准测试
195198

196-
### 实验方法
199+
### SHA256 基准测试
197200

198-
* 测试设备：MacBook Air M1 8GB
201+
#### 实验方法
199202

200-
* 步骤：
203+
* 设备：MacBook Air M1 8GB
201204

202-
1. 运行并重复 3 次：
203-
```bash
204-
$ count=10000 # Test size = 1MiB * $count
205-
$ # Bare（数据生成速度）
206-
$ dd if=/dev/zero bs=1M count=$count | pv > /dev/null
207-
$ # ezcheck-hashes
208-
$ dd if=/dev/zero bs=1M count=$count | pv | ./ezcheck-hashes calculate sha256 -f -
209-
$ # ezcheck-ring
210-
$ dd if=/dev/zero bs=1M count=$count | pv | ./ezcheck-ring calculate sha256 -f -
211-
$ # sha256sum
212-
$ dd if=/dev/zero bs=1M count=$count | pv | sha256sum
213-
```
205+
* 步骤
214206

215-
2. 计算平均值。
207+
1. 运行并重复 3 次：
208+
```bash
209+
$ count=10000 # Test size = 1MiB * $count
210+
$ # Bare, Speed of generating the data
211+
$ dd if=/dev/zero bs=1M count=$count | pv > /dev/null
212+
$ # ezcheck-hashes
213+
$ dd if=/dev/zero bs=1M count=$count | pv | ./ezcheck-hashes calculate sha256 -f -
214+
$ # ezcheck-ring
215+
$ dd if=/dev/zero bs=1M count=$count | pv | ./ezcheck-ring calculate sha256 -f -
216+
$ # sha256sum
217+
$ dd if=/dev/zero bs=1M count=$count | pv | sha256sum
218+
```
216219

217-
### 结果
220+
2. 计算平均值。
218221

219-
| 工具 / 速度 (GiB/s) / 测试数据大小 (MiB) | 1 | 100 | 500 | 1000 | 5000 | 10000 |
220-
|--------------------------------|------|------|------|------|------|-------|
221-
| Bare（数据生成速度） | 2.13 | 3.02 | 4.59 | 5.31 | 5.97 | 6.07 |
222-
| ezcheck-hashes | 0.13 | 0.28 | 0.29 | 0.30 | 0.30 | 0.30 |
223-
| ezcheck-ring | 0.58 | 1.24 | 1.57 | 1.63 | 1.68 | 1.68 |
224-
| sha256sum | 0.73 | 1.26 | 1.63 | 1.69 | 1.75 | 1.81 |
222+
#### 实验结果
225223

226-
![benchmark](./benchmark.png)
224+
| 实现 / 速度(GiB/s) / 数据量(MiB) | 1 | 100 | 500 | 1000 | 5000 | 10000 |
225+
|---------------------------|------|------|------|------|------|-------|
226+
| Bare（数据生成速度） | 2.13 | 3.02 | 4.59 | 5.31 | 5.97 | 6.07 |
227+
| ezcheck-hashes | 0.13 | 0.28 | 0.29 | 0.30 | 0.30 | 0.30 |
228+
| ezcheck-ring | 0.58 | 1.24 | 1.57 | 1.63 | 1.68 | 1.68 |
229+
| sha256sum | 0.73 | 1.26 | 1.63 | 1.69 | 1.75 | 1.81 |
230+
231+
![benchmark](./benchmark-sha256.png)
232+
233+
### 速度 vs. 算法，实现
234+
235+
#### 实验方法
236+
237+
* 设备：MacBook Air M1 8GB
238+
239+
* 步骤
240+
241+
1. 执行：
242+
```bash
243+
$ algorithm=sha256
244+
$ # ezcheck-hashes
245+
$ dd if=/dev/zero bs=1M count=10000 | pv | ./ezcheck-hashes calculate $algorithm -f -
246+
$ # ezcheck-ring
247+
$ dd if=/dev/zero bs=1M count=10000 | pv | ./ezcheck-ring calculate $algorithm -f -
248+
$ # ezcheck-mix
249+
$ dd if=/dev/zero bs=1M count=10000 | pv | ./ezcheck-mix calculate $algorithm -f -
250+
```
251+
2. 计算平均值
252+
253+
#### 实验结果
254+
255+
| 算法 / 速度(GiB/s) / 实现 | ring | hashes | mix |
256+
|---------------------|--------|----------|----------|
257+
| MD2 | null* | 0.00896 | 0.00896 |
258+
| MD4 | null* | 0.852 | 0.852 |
259+
| MD5 | null* | 0.549 | 0.549 |
260+
| SHA1 | null* | 0.802 | 0.802 |
261+
| SHA224 | null* | 0.299 | 0.299 |
262+
| SHA256 | 1.69 | 0.298 | 1.70 |
263+
| SHA384 | 1.12 | 0.473 | 1.13 |
264+
| SHA512 | 1.13 | 0.473 | 1.13 |
265+
| SHA512/256 | 1.13 | 0.473 | 1.13 |
266+
| XXHASH32 | null* | null* | 2.45 |
267+
| XXHASH64 | null* | null* | 3.27 |
268+
| XXHASH3_64 | null* | null* | 3.65 |
269+
270+
_null*: 此实现并未实现此算法。_
271+
272+
![benchmark](./benchmark-algorithms-implementations.png)
227273

228274
## 许可证
229275

‎benchmark-algorithms-implementations.png‎

240 KB
Loading

‎benchmark-sha256.png‎

158 KB
Loading

‎benchmark.png‎

-236 KB
Binary file not shown.

0 commit comments

Comments
 (0)