Skip to content

Commit 11bd20b

Browse files
authored
Add Amazon S3's parallel SHA2-256 (#343)
When data is uploaded to S3 in can be chunked. Each chunk is hashed, those hashes are then concatenated and hashes again into a single hash. Quilt is using 8MiB chunks, but increases the size in case the input is large, as the maximum size of chunks is 10k. For all details about the hash algorithm see: https://github.com/quiltdata/quilt/blob/fb36269def0f8b811e0c1293fcf3e853732a7b71/docs/sha2-256-chunked.md
1 parent d06fc61 commit 11bd20b

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

table.csv

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -510,6 +510,7 @@ poseidon-bls12_381-a2-fc1-sc, multihash, 0xb402, draft, Pose
510510
rdfc-1, ipld, 0xb403, draft, The result of canonicalizing an input according to RDFC-1.0 and then expressing its hash value as a multihash value.
511511
ssz, serialization, 0xb501, draft, SimpleSerialize (SSZ) serialization
512512
ssz-sha2-256-bmt, multihash, 0xb502, draft, SSZ Merkle tree root using SHA2-256 as the hashing function and SSZ serialization for the block binary
513+
sha2-256-chunked, multihash, 0xb510, draft, Hash of concatenated SHA2-256 digests of 8*2^n MiB source chunks; n = ceil(log2(source_size/(10^4 * 8MiB)))
513514
json-jcs, ipld, 0xb601, draft, The result of canonicalizing an input according to JCS - JSON Canonicalisation Scheme (RFC 8785)
514515
iscc, softhash, 0xcc01, draft, ISCC (International Standard Content Code) - similarity preserving hash
515516
zeroxcert-imprint-256, zeroxcert, 0xce11, draft, 0xcert Asset Imprint (root hash)

0 commit comments

Comments
 (0)