py-multihash Implementation Comparison: Gap Analysis with Go, Rust, and JavaScript #1061

acul71 · 2025-11-25T01:19:32Z

acul71
Nov 25, 2025
Maintainer

Multihash Implementation Comparison

Executive Summary

This document provides a comprehensive comparison between the Python py-multihash implementation and equivalent implementations in Go, Rust, and JavaScript. The analysis identifies feature gaps, API differences, and areas where the Python implementation could be enhanced to achieve feature parity with other implementations.

Key Findings:

Python provides basic encode/decode functionality but lacks actual hash computation capabilities
Go offers the most complete feature set including hash computation, streaming, and extensibility
Rust uses const generics for compile-time size validation and provides type-safe multihash operations
JavaScript offers modern async/sync APIs with first-class digest types

Feature Comparison Table

Feature	Python (py-multihash)	Go (go-multihash)	Rust (rust-multihash)	JavaScript (js-multiformats)
Core Operations
Encode multihash	✅ `encode()`	✅ `Encode()`, `EncodeName()`	✅ `Multihash::wrap()`	✅ `Digest.create()`
Decode multihash	✅ `decode()`	✅ `Decode()`, `Cast()`	✅ `Multihash::from_bytes()`	✅ `Digest.decode()`
Validate multihash	✅ `is_valid()`	✅ `Cast()`	✅ `from_bytes()` (validates)	✅ `decode()` (validates)
Hash Computation
Compute hash	❌	✅ `Sum()`, `SumStream()`	✅ Via codetable	✅ `Hasher.digest()`
Streaming hash	❌	✅ `SumStream()`	✅ Via `io::Read`	✅ Async `digest()`
Variable-size hashes	❌	✅ `RegisterVariableSize()`	✅ Via codetable	✅ Truncate option
String Encoding
Hex encoding	✅ `to_hex_string()`, `from_hex_string()`	✅ `HexString()`, `FromHexString()`	❌	❌ (via multibase)
Base58 encoding	✅ `to_b58_string()`, `from_b58_string()`	✅ `B58String()`, `FromB58String()`	❌	❌ (via multibase)
Data Structures
Multihash type	✅ `Multihash` (namedtuple)	✅ `Multihash` ([]byte), `DecodedMultihash`	✅ `Multihash<const S>`	✅ `MultihashDigest<Code>`
Set/Collection	❌	✅ `Set` type	❌	❌
Extensibility
Register custom hasher	❌	✅ `Register()`, `RegisterVariableSize()`	✅ Via derive macro	✅ Via `Hasher.from()`
Get hasher by code	❌	✅ `GetHasher()`	✅ Via codetable	✅ Via hasher instances
Type Safety
Compile-time size checks	❌	❌	✅ Const generics	✅ TypeScript generics
Type-safe codes	❌	❌	✅ Enum-based codes	✅ Generic code types
Serialization
Serde support	❌	❌	✅ `serde` feature	❌
SCALE codec	❌	❌	✅ `scale-codec` feature	❌
JSON	❌	❌	✅ Via serde	❌
Advanced Features
Truncate digest	❌	✅ Via `Sum()` length param	✅ `truncate()` method	✅ `truncate` option
Resize multihash	❌	❌	✅ `resize()` method	❌
Get prefix	✅ `get_prefix()`	❌	❌	❌
Code validation	✅ `is_valid_code()`, `coerce_code()`	✅ Via `Names`, `Codes` maps	✅ Via codetable	✅ Type checking
App code support	✅ `is_app_code()`	✅ Via code ranges	✅ Via codetable	✅
Error Handling
Custom error types	❌ (ValueError, TypeError)	✅ `ErrInconsistentLen`, etc.	✅ `Error` enum	✅ Error classes
Streaming I/O
Read from stream	❌	✅ `Reader.ReadMultihash()`	✅ `Multihash::read()`	❌
Write to stream	❌	✅ `Writer.WriteMultihash()`	✅ `Multihash::write()`	❌
No_std Support	❌	❌	✅	❌
Async Support	❌	❌	❌	✅ Async/sync hashing

API Comparison

Core Encoding/Decoding

Python

# Encode
multihash = encode(digest, code, length=None)
multihash = encode(digest, "sha2-256", length=None)

# Decode
decoded = decode(multihash)  # Returns Multihash namedtuple
# decoded.code, decoded.name, decoded.length, decoded.digest

Go

// Encode
mh, _ := multihash.Encode(digest, code)
mh, _ := multihash.EncodeName(digest, "sha2-256")

// Decode
dm, _ := multihash.Decode(mh)  // Returns *DecodedMultihash
// dm.Code, dm.Name, dm.Length, dm.Digest

Rust

// Encode
let mh = Multihash::<64>::wrap(code, &digest)?;

// Decode
let mh = Multihash::<64>::from_bytes(&bytes)?;
// mh.code(), mh.size(), mh.digest()

JavaScript

// Encode
const digest = Digest.create(code, digestBytes);

// Decode
const digest = Digest.decode(multihashBytes);
// digest.code, digest.size, digest.digest, digest.bytes

Hash Computation

Python

# NOT SUPPORTED - No hash computation
# Must use external libraries (hashlib, etc.) and then encode
import hashlib
digest = hashlib.sha256(data).digest()
mh = encode(digest, "sha2-256")

Go

// Compute hash directly
mh, _ := multihash.Sum(data, multihash.SHA2_256, -1)

// Streaming
mh, _ := multihash.SumStream(reader, multihash.SHA2_256, -1)

Rust

// Via codetable
use multihash_codetable::Code;
let hash = Code::Sha2_256.digest(&data);

JavaScript

// Sync
const hash = sha256.digest(data);

// Async
const hash = await sha256.digest(data);

// With truncation
const hash = sha256.digest(data, { truncate: 24 });

String Encoding

Python

# Hex
hex_str = to_hex_string(multihash)
mh = from_hex_string(hex_str)

# Base58
b58_str = to_b58_string(multihash)
mh = from_b58_string(b58_str)

Go

// Hex
hexStr := mh.HexString()
mh, _ := multihash.FromHexString(hexStr)

// Base58
b58Str := mh.B58String()
mh, _ := multihash.FromB58String(b58Str)

Rust

// Not built-in - use external crates for base encoding

JavaScript

// Not built-in - use multibase for encoding

Validation and Utilities

Python

# Validation
is_valid(multihash)  # Returns bool
is_valid_code(code)  # Returns bool
is_app_code(code)    # Returns bool

# Code coercion
code = coerce_code("sha2-256")  # Returns int
code = coerce_code(0x12)        # Returns int

# Prefix
prefix = get_prefix(multihash)  # Returns bytes

Go

// Validation via Cast
mh, err := multihash.Cast(bytes)  // Validates

// Code lookup
code, ok := multihash.Names["sha2-256"]
name, ok := multihash.Codes[0x12]

Rust

// Validation via from_bytes
let mh = Multihash::<64>::from_bytes(&bytes)?;  // Validates

// Truncate
let truncated = mh.truncate(16);

// Resize
let resized: Multihash<32> = mh.resize()?;

JavaScript

// Validation via decode
const digest = Digest.decode(bytes);  // Throws if invalid

// Type checking
if (digestHasCode(digest, sha256.code)) {
  // TypeScript knows digest is MultihashDigest<0x12>
}

Hash Function Support

Python (py-multihash)

Status: Constants defined, but no actual hash computation

The Python implementation defines hash codes in constants.py but does not provide hash computation. Users must use external libraries like hashlib and then encode the result.

Defined Hash Functions:

Identity (0x00)
SHA1 (0x11)
SHA2-256 (0x12)
SHA2-512 (0x13)
SHA3 variants (0x14-0x17)
SHAKE (0x18-0x19)
Keccak (0x1a-0x1d)
Murmur3 (0x22-0x23)
Blake2b (0xb201-0xb240) - all sizes
Blake2s (0xb241-0xb260) - all sizes
Skein256/512/1024 (0xb301-0xb3e0) - all sizes
Double SHA2-256 (0x56)

Go (go-multihash)

Status: Full hash computation support with extensible registry

Pre-registered Hash Functions:

Identity (0x00)
MD5 (0xd5)
SHA1 (0x11)
SHA2-224, SHA2-256, SHA2-384, SHA2-512
SHA2-512/224, SHA2-512/256
SHA3-224, SHA3-256, SHA3-384, SHA3-512
SHAKE-128, SHAKE-256
Keccak-224, Keccak-256, Keccak-384, Keccak-512
Blake2b (all sizes via register/blake2)
Blake2s (all sizes via register/blake2)
Blake3 (0x1e) - variable size
Murmur3-x64-64 (0x22)
Double SHA2-256 (0x56)
SHA2-256-trunc254-padded (0x1012)
X11 (0x1100)
Poseidon-BLS12-381 (0xb401)

Extensibility: Can register custom hashers via Register() or RegisterVariableSize()

Rust (rust-multihash)

Status: Core crate is hash-agnostic; codetable crate provides implementations

Core Crate: Only defines data structure, no hash computation

Codetable Crate (via features):

SHA1 (feature: sha1)
SHA2-256, SHA2-512 (feature: sha2)
SHA3-224, SHA3-256, SHA3-384, SHA3-512 (feature: sha3)
Keccak-224, Keccak-256, Keccak-384, Keccak-512 (feature: sha3)
Blake2b-256, Blake2b-512 (feature: blake2b)
Blake2s-128, Blake2s-256 (feature: blake2s)
Blake3-256 (feature: blake3)
Strobe-256, Strobe-512 (feature: strobe)
RIPEMD-160, RIPEMD-256, RIPEMD-320 (feature: ripemd)

Extensibility: Can define custom code tables via multihash-derive macro

JavaScript (js-multiformats)

Status: Provides hasher implementations with sync/async support

Implemented Hashers:

Identity (0x00) - sync
SHA1 (0x11) - sync/async
SHA2-256 (0x12) - sync/async
SHA2-512 (0x13) - sync/async

Extensibility: Can create custom hashers via Hasher.from() factory

Missing Hash Functions in Python

The following hash functions are supported in other implementations but missing from Python's constants table:

Missing from Go Implementation

Hash Function	Code	Notes
MD5	`0xd5`	Available in Go standard library
SHA2-224	`0x1013`	SHA-224 variant
SHA2-384	`0x20`	SHA-384 variant
SHA2-512/224	`0x1014`	Truncated SHA-512 variant
SHA2-512/256	`0x1015`	Truncated SHA-512 variant
Blake3	`0x1e`	Modern cryptographic hash (variable size)
SHA2-256-trunc254-padded	`0x1012`	Specialized variant for certain protocols
X11	`0x1100`	Multi-algorithm hash (used in cryptocurrencies)
Poseidon-BLS12-381-A2-FC1	`0xb401`	Zero-knowledge proof friendly hash

Missing from Rust Implementation

Hash Function	Code	Notes
RIPEMD-160	`0x1053`	Legacy hash function (feature: `ripemd`)
RIPEMD-256	`0x1054`	RIPEMD variant (feature: `ripemd`)
RIPEMD-320	`0x1055`	RIPEMD variant (feature: `ripemd`)
Strobe-256	`0x3312e7`	STROBE protocol hash (feature: `strobe`)
Strobe-512	`0x3312e8`	STROBE protocol hash (feature: `strobe`)

Summary

Total missing hash functions: 14

By category:

SHA2 variants: 4 (SHA2-224, SHA2-384, SHA2-512/224, SHA2-512/256)
Modern hashes: 1 (Blake3)
Legacy hashes: 4 (MD5, RIPEMD-160/256/320)
Specialized hashes: 4 (SHA2-256-trunc254-padded, X11, Poseidon, Strobe variants)
Cryptographic protocols: 1 (X11, Poseidon)

Note: Python has some hash functions that others don't include by default:

Skein256/512/1024 (codes 0xb301-0xb3e0) - Available in Python but not in Go/Rust standard implementations

Gap Analysis

Summary of Missing Features in py-multihash

The following features are available in other implementations but missing in Python:

Feature	Available In	Missing In Python
Hash computation (`Sum()`)	Go, Rust (via codetable), JavaScript	❌
Streaming hash computation (`SumStream()`)	Go	❌
Extensibility/Registry system	Go, Rust, JavaScript	❌
Multihash Set/Collection type	Go	❌
Truncation support	Go, Rust, JavaScript	❌
Serialization (Serde/JSON)	Rust	❌
Type-safe operations	Rust, JavaScript	❌
Custom error types	Go, Rust, JavaScript	❌
Async hash computation	JavaScript	❌
Stream I/O (Read/Write)	Go, Rust	❌

Critical Gaps in py-multihash

1. No Hash Computation

Impact: High - Users must manually compute hashes using external libraries

Missing:

Sum() equivalent to compute hashes directly
SumStream() for streaming hash computation
Integration with Python's hashlib or other hash libraries

Workaround:

import hashlib
from multihash import encode

digest = hashlib.sha256(data).digest()
mh = encode(digest, "sha2-256")

Recommendation: Add sum() and sum_stream() functions that integrate with hashlib and other common Python hash libraries.

2. No Extensibility Mechanism

Impact: Medium - Cannot register custom hash functions

Missing:

Register() equivalent to add custom hashers
GetHasher() to retrieve hasher by code
Registry system for managing available hash functions

Recommendation: Add a registry system similar to Go's implementation, allowing users to register custom hash functions.

3. No Streaming Support

Impact: Medium - Cannot efficiently hash large files/streams

Missing:

SumStream() for hashing from file-like objects
Integration with Python's streaming APIs

Recommendation: Add sum_stream() that accepts file-like objects and uses incremental hashing.

4. Limited Data Structures

Impact: Low - Missing utility collections

Missing:

Set type for multihash collections (like Go)
More sophisticated multihash container types

Recommendation: Add optional MultihashSet class for managing collections of multihashes.

5. No Truncation Support

Impact: Low - Cannot truncate digests during encoding

Missing:

Truncate option in encode() or separate truncate function
Validation of truncation bounds

Recommendation: Add truncate() method or truncate parameter to encode().

6. No Serialization Support

Impact: Low - Cannot serialize multihashes to JSON/other formats

Missing:

Serde-like serialization support
JSON encoding/decoding

Recommendation: Add optional serialization support using standard Python serialization libraries.

Minor Gaps

7. No Type Safety

Python's dynamic typing means no compile-time guarantees
Could add type hints and runtime validation

8. Limited Error Types

Currently uses generic ValueError and TypeError
Could add custom exception hierarchy

9. No Async Support

All operations are synchronous
Could add async variants for hash computation

Recommendations

Priority 1: Critical Features

Add Hash Computation
- Implement sum(data, code, length=-1) function
- Integrate with Python's hashlib standard library
- Support common hash functions: SHA1, SHA2-256, SHA2-512, SHA3 variants
- Example:
```
mh = multihash.sum(b"hello", "sha2-256")
```
Add Streaming Support
- Implement sum_stream(stream, code, length=-1) function
- Accept file-like objects (file handles, BytesIO, etc.)
- Use incremental hashing for memory efficiency
- Example:
```
with open("large_file.bin", "rb") as f:
    mh = multihash.sum_stream(f, "sha2-256")
```

Priority 2: Important Features

Add Extensibility
- Implement registry system for custom hash functions
- Add register(code, hasher_factory) function
- Add get_hasher(code) function
- Example:
```
def my_hasher():
    return hashlib.new("sha256")

multihash.register(0x12, my_hasher)
```
Add Truncation Support
- Add truncate() method or parameter
- Validate truncation bounds
- Example:
```
mh = multihash.sum(data, "sha2-256", length=16)  # Truncate to 16 bytes
```

Priority 3: Nice-to-Have Features

Add Utility Collections
- Implement MultihashSet class
- Support set operations (union, intersection, etc.)

Improve Error Handling

Create custom exception hierarchy
More descriptive error messages

Example:

class MultihashError(Exception): pass
class InvalidMultihashError(MultihashError): pass
class UnsupportedCodeError(MultihashError): pass

Add Type Hints
- Add comprehensive type hints for better IDE support
- Use typing module for generic types
Add Serialization Support
- Optional JSON serialization
- Support for other formats (MessagePack, etc.)

Implementation Examples

Example: Adding Hash Computation

import hashlib
from typing import Union, Optional

def sum(data: bytes, code: Union[int, str], length: Optional[int] = None) -> bytes:
    """
    Compute a multihash for the given data.
    
    Args:
        data: Input data to hash
        code: Hash function code (int) or name (str)
        length: Optional length to truncate to (default: full digest)
    
    Returns:
        Encoded multihash bytes
    """
    hash_code = coerce_code(code)
    
    # Map codes to hashlib functions
    hasher_map = {
        0x11: lambda: hashlib.sha1(),
        0x12: lambda: hashlib.sha256(),
        0x13: lambda: hashlib.sha512(),
        # Add more mappings...
    }
    
    if hash_code not in hasher_map:
        raise ValueError(f"Hash computation not supported for code {hash_code}")
    
    hasher = hasher_map[hash_code]()
    hasher.update(data)
    digest = hasher.digest()
    
    if length is not None:
        if length > len(digest):
            raise ValueError(f"Length {length} exceeds digest size {len(digest)}")
        digest = digest[:length]
    
    return encode(digest, hash_code, len(digest))

Example: Adding Streaming Support

from typing import BinaryIO, Union, Optional

def sum_stream(stream: BinaryIO, code: Union[int, str], length: Optional[int] = None) -> bytes:
    """
    Compute a multihash from a stream.
    
    Args:
        stream: File-like object to read from
        code: Hash function code (int) or name (str)
        length: Optional length to truncate to
    
    Returns:
        Encoded multihash bytes
    """
    hash_code = coerce_code(code)
    
    hasher_map = {
        0x11: lambda: hashlib.sha1(),
        0x12: lambda: hashlib.sha256(),
        0x13: lambda: hashlib.sha512(),
    }
    
    if hash_code not in hasher_map:
        raise ValueError(f"Hash computation not supported for code {hash_code}")
    
    hasher = hasher_map[hash_code]()
    
    # Read in chunks for memory efficiency
    chunk_size = 8192
    while True:
        chunk = stream.read(chunk_size)
        if not chunk:
            break
        hasher.update(chunk)
    
    digest = hasher.digest()
    
    if length is not None:
        if length > len(digest):
            raise ValueError(f"Length {length} exceeds digest size {len(digest)}")
        digest = digest[:length]
    
    return encode(digest, hash_code, len(digest))

Conclusion

The Python py-multihash implementation provides solid foundational functionality for encoding and decoding multihashes, but lacks several key features present in other implementations:

Hash computation - The most critical gap
Streaming support - Important for large files
Extensibility - Needed for custom hash functions
Advanced features - Truncation, collections, serialization

By implementing the Priority 1 and Priority 2 recommendations, py-multihash would achieve feature parity with the Go implementation for most use cases. The Priority 3 features would bring it closer to the Rust and JavaScript implementations in terms of developer experience and type safety.

The current implementation is well-suited for applications that already compute hashes externally and only need multihash encoding/decoding. However, adding hash computation capabilities would make it a more complete and user-friendly library.

acul71 · 2025-12-13T22:28:46Z

acul71
Dec 13, 2025
Maintainer Author

Hello @seetadev @sumanjeet0012 @pacrob
The py-multihash has been improved I think me can make a release with current features, and make this features in future releases:

Add missing hash function implementations:
- Blake3 (requires external library like blake3 package)
- Murmur3 (requires external library like mmh3 package)
- Double SHA2-256 (can be implemented using hashlib)

Medium Priority

Add multihash read/write stream methods:
- Multihash.read(stream) - read multihash from stream
- Multihash.write(stream) - write multihash to stream
Consider async support:
- Add async versions: async_digest(), async_sum(), async_sum_stream()
- Useful for async Python applications

Low Priority

Document hash function availability:
- Clearly document which functions are computable vs. encodable-only
- Add helper to check if function is computable

#################

Full Review

py-multihash Implementation Status Report

Analysis Date: 2025-01-27
Reference: GitHub Discussion #1061
Codebase Version: Current (as of analysis date)

Executive Summary

This report compares the current py-multihash implementation against the feature comparison table from GitHub discussion #1061. The analysis reveals that many features previously marked as "NOT SUPPORTED" have since been implemented, significantly closing the gap with other implementations (Go, Rust, JavaScript).

Key Findings

✅ Hash computation is now fully implemented (previously marked as NOT SUPPORTED)
✅ Streaming hash computation is implemented (previously marked as NOT SUPPORTED)
✅ MultihashSet collection type is implemented (previously marked as NOT SUPPORTED)
✅ Custom hasher registration is implemented (previously marked as NOT SUPPORTED)
✅ Truncation support is implemented (previously marked as NOT SUPPORTED)
✅ JSON serialization is implemented (not mentioned in original discussion)
✅ Custom exception hierarchy is implemented (previously marked as basic ValueError/TypeError)
❌ Resize multihash is NOT implemented (Rust-specific feature)
❌ Multihash read/write from streams is NOT implemented (only hash computation from streams)
⚠️ Some hash functions are defined in constants but not computable (Blake3, Murmur3, dbl-sha2-256)

Detailed Feature Comparison

1. Core Operations

Feature	Status	Location	Notes
Encode multihash	✅ Implemented	`multihash/multihash.py:675`	`encode(digest, code, length=None)`
Decode multihash	✅ Implemented	`multihash/multihash.py:634`	`decode(multihash)` returns `Multihash`
Validate multihash	✅ Implemented	`multihash/multihash.py:701`	`is_valid(multihash)`

Status: ✅ Fully Implemented

2. Hash Computation

Feature	Status	Location	Notes
Compute hash	✅ IMPLEMENTED	`multihash/multihash.py:498, 731`	`digest()` and `sum()` functions
Streaming hash	✅ IMPLEMENTED	`multihash/multihash.py:762`	`sum_stream()` function
Variable-size hashes	✅ Implemented	`multihash/funcs.py:111`	SHAKE via `ShakeHash` wrapper

Status: ✅ Fully Implemented (Previously marked as NOT SUPPORTED in discussion)

Implementation Details:

digest(data, func, length=None) - Pythonic API
sum(data, code, length=None) - Go-compatible API
sum_stream(stream, code, length=None, chunk_size=8192) - Streaming with chunked reading
SHAKE-128 and SHAKE-256 supported with variable length via ShakeHash wrapper
Truncation support via length parameter (with validation)

3. String Encoding

Feature	Status	Location	Notes
Hex encoding	✅ Implemented	`multihash/multihash.py:523, 538`	`to_hex_string()`, `from_hex_string()`
Base58 encoding	✅ Implemented	`multihash/multihash.py:553, 568`	`to_b58_string()`, `from_b58_string()`

Status: ✅ Fully Implemented

4. Data Structures

Feature	Status	Location	Notes
Multihash type	✅ Implemented	`multihash/multihash.py:37`	`Multihash` namedtuple with methods
Set/Collection	✅ IMPLEMENTED	`multihash/multihash.py:203`	`MultihashSet` class

Status: ✅ Fully Implemented (MultihashSet was previously marked as NOT SUPPORTED)

MultihashSet Features:

Go-style API: Add(), Remove(), Has(), All()
Python-style API: add(), remove(), discard(), __contains__()
Set operations: union(), intersection(), difference(), symmetric_difference()
Type-safe: Only accepts Multihash objects

5. Extensibility

Feature	Status	Location	Notes
Register custom hasher	✅ IMPLEMENTED	`multihash/funcs.py:251`	`FuncReg.register()`
Get hasher by code	✅ Implemented	`multihash/funcs.py:223`	`FuncReg.get()`
Get hasher by name	✅ Implemented	`multihash/funcs.py:223`	`FuncReg.get()` accepts str/int/Func
Create hash object	✅ Implemented	`multihash/funcs.py:321`	`FuncReg.hash_from_func()`
App-specific functions	✅ Implemented	`multihash/funcs.py:251`	Codes 0x01-0x0F supported

Status: ✅ Fully Implemented (Previously marked as NOT SUPPORTED)

Implementation Details:

FuncReg.register(code, name, hash_name, hash_new) - Register custom hashers
FuncReg.unregister(code) - Unregister app-specific functions (0x01-0x0F only)
FuncReg.get(func_hint) - Get function by code, name, or Func enum
FuncReg.hash_from_func(func, length=None) - Create hashlib-compatible hash object
FuncReg.func_from_hash(hash_obj) - Get Func from hashlib object

6. Advanced Features

Feature	Status	Location	Notes
Truncate digest	✅ IMPLEMENTED	`multihash/multihash.py:486`	Via `length` parameter in `digest()`/`sum()`
Resize multihash	❌ Not Implemented	N/A	Rust-specific feature (const generics)
Get prefix	✅ Implemented	`multihash/multihash.py:716`	`get_prefix(multihash)`
Code validation	✅ Implemented	`multihash/multihash.py:623, 594`	`is_valid_code()`, `coerce_code()`
App code support	✅ Implemented	`multihash/multihash.py:583`	`is_app_code()`

Status: ⚠️ Mostly Implemented (Resize is Rust-specific and not applicable to Python)

Truncation Implementation:

Supported in digest(), sum(), and sum_stream()
length=-1 means full digest
length=None means auto (full digest for fixed-size, default for SHAKE)
Validation: raises TruncationError for invalid lengths

7. Error Handling

Feature	Status	Location	Notes
Custom error types	✅ IMPLEMENTED	`multihash/exceptions.py`	Full exception hierarchy

Status: ✅ Fully Implemented (Previously marked as basic ValueError/TypeError)

Exception Hierarchy:

MultihashError (base exception)
- InvalidMultihashError
- UnsupportedCodeError
- HashComputationError
- TruncationError

8. Streaming I/O

Feature	Status	Location	Notes
Compute hash from stream	✅ Implemented	`multihash/multihash.py:762`	`sum_stream()`
Read multihash from stream	❌ Not Implemented	N/A	Rust `Multihash::read()` equivalent
Write multihash to stream	❌ Not Implemented	N/A	Rust `Multihash::write()` equivalent

Status: ⚠️ Partially Implemented

Notes:

sum_stream() computes hash from stream (for hashing data)
No read()/write() methods for serializing/deserializing multihash from streams
Can use decode() with BytesIO for reading, encode() with file write for writing

9. Serialization

Feature	Status	Location	Notes
JSON	✅ IMPLEMENTED	`multihash/multihash.py:165, 400`	`to_json()`, `from_json()`
Serde (Rust)	N/A	N/A	Rust-specific
SCALE codec	N/A	N/A	Rust-specific

Status: ✅ JSON Implemented (Not mentioned in original discussion)

JSON Implementation:

Multihash.to_json(verbose=False) - Serialize to JSON
from_json(json_str) - Deserialize from JSON
Compact format: {"code": int, "length": int, "digest": "base64..."}
Verbose format: includes "name" field

10. Hash Function Support

Category	Status	Details
Defined in Func enum	✅	20 functions in `Func` enum
Actually computable	⚠️	Most functions computable, some missing
Defined in constants only	⚠️	Many functions in `constants.py` but not in `Func` enum

Status: ⚠️ Partially Implemented

Functions in Func Enum (Computable):

✅ Identity (0x00)
✅ SHA1 (0x11)
✅ SHA2-256, SHA2-512 (0x12, 0x13)
✅ SHA2-224, SHA2-384 (0x1013, 0x20) - conditionally if available
✅ SHA3-224, SHA3-256, SHA3-384, SHA3-512 (0x17, 0x16, 0x15, 0x14)
✅ SHAKE-128, SHAKE-256 (0x18, 0x19) - variable length
✅ Keccak-224, Keccak-256, Keccak-384, Keccak-512 (0x1A-0x1D)
✅ Blake2b-256, Blake2b-512 (0xB220, 0xB240)
✅ Blake2s-256 (0xB260)
✅ MD5 (0xD5)

Functions in Constants but NOT in Func Enum (Not Computable):

❌ Blake3 (0x1E) - in constants, not in Func enum
❌ Murmur3-128 (0x22) - in constants, not in Func enum
❌ Murmur3-32 (0x23) - in constants, not in Func enum
❌ Double SHA2-256 (0x56) - in constants, not in Func enum
❌ Many Blake2b/Blake2s variants (0xB201-0xB260) - only 3 in Func enum
❌ Many Skein variants (0xB301-0xB3E0) - in constants, not in Func enum
❌ RIPEMD variants (0x1052-0x1055) - in constants, not in Func enum
❌ Other specialized hashes (X11, SM3, etc.)

Gap Analysis:

Functions can be encoded/decoded if defined in constants
Functions can only be computed if registered in FuncReg
Missing implementations for: Blake3, Murmur3, Double SHA2-256, many Blake2/Skein variants

11. Type Safety Features

Feature	Status	Notes
Compile-time size checks	❌ Not Applicable	Python doesn't have const generics (Rust feature)
Type-safe codes	⚠️ Partial	`Func` enum provides some type safety, but not compile-time

Status: ❌ Language Limitation (Python doesn't support compile-time type checking like Rust/TypeScript)

Current Type Safety:

Func IntEnum provides runtime type safety
Type annotations in code (for static type checkers)
No compile-time guarantees (Python limitation)

12. Language-Specific Features

Feature	Status	Notes
Async support	❌ Not Implemented	JavaScript has async/sync APIs
No_std support	N/A	Rust-specific, not applicable to Python

Status: ❌ Not Implemented (Async could be added but currently not present)

Notes:

Python has async/await support, but multihash functions are synchronous
Could add async versions of digest(), sum(), sum_stream() if needed

Summary Table

Category	Python Status	Discussion Status	Gap
Core Operations	✅ Implemented	✅ Implemented	None
Hash Computation	✅ IMPLEMENTED	❌ NOT SUPPORTED	CLOSED
Streaming Hash	✅ IMPLEMENTED	❌ NOT SUPPORTED	CLOSED
String Encoding	✅ Implemented	✅ Implemented	None
Data Structures	✅ IMPLEMENTED	⚠️ MultihashSet NOT SUPPORTED	CLOSED
Extensibility	✅ IMPLEMENTED	❌ NOT SUPPORTED	CLOSED
Truncation	✅ IMPLEMENTED	❌ NOT SUPPORTED	CLOSED
Error Handling	✅ IMPLEMENTED	⚠️ Basic only	IMPROVED
Streaming I/O	⚠️ Partial	❌ NOT SUPPORTED	Partial
Serialization	✅ JSON	Not mentioned	ADDED
Hash Functions	⚠️ Partial	⚠️ Partial	Some missing
Type Safety	❌ N/A	❌ N/A	Language limitation
Async	❌ Not Implemented	❌ Not Implemented	None

Major Improvements Since Discussion

Hash Computation: Fully implemented with digest(), sum(), and sum_stream()
Streaming: sum_stream() supports streaming hash computation
MultihashSet: Complete collection type with Go and Python APIs
Extensibility: Full registry system with FuncReg.register()/unregister()
Truncation: Full support with validation
Error Handling: Custom exception hierarchy
JSON Serialization: Added to_json() and from_json()

Remaining Gaps

Missing Features (Compared to Other Implementations)

Resize Multihash (Rust-specific)
- Rust has resize() method for changing multihash size
- Not applicable to Python (no const generics)
Multihash Read/Write from Streams
- Rust has Multihash::read() and Multihash::write()
- Python can use decode() with BytesIO and encode() with file writes, but no dedicated methods
Missing Hash Function Implementations
- Blake3 (0x1E) - defined but not computable
- Murmur3 variants (0x22, 0x23) - defined but not computable
- Double SHA2-256 (0x56) - defined but not computable
- Many Blake2/Skein variants - only subset implemented
Async Support
- JavaScript has async/sync APIs
- Python implementation is synchronous only

Recommendations

High Priority

Add missing hash function implementations:
- Blake3 (requires external library like blake3 package)
- Murmur3 (requires external library like mmh3 package)
- Double SHA2-256 (can be implemented using hashlib)

Medium Priority

Add multihash read/write stream methods:
- Multihash.read(stream) - read multihash from stream
- Multihash.write(stream) - write multihash to stream
Consider async support:
- Add async versions: async_digest(), async_sum(), async_sum_stream()
- Useful for async Python applications

Low Priority

Document hash function availability:
- Clearly document which functions are computable vs. encodable-only
- Add helper to check if function is computable

Conclusion

The py-multihash implementation has significantly improved since the original discussion. Most features previously marked as "NOT SUPPORTED" have been implemented, bringing it much closer to feature parity with Go, Rust, and JavaScript implementations.

Key Achievements:

✅ Hash computation fully implemented
✅ Streaming support added
✅ Extensibility system complete
✅ MultihashSet collection added
✅ Custom exception hierarchy
✅ JSON serialization added

Remaining Work:

⚠️ Some hash functions defined but not computable
⚠️ No async support
⚠️ No dedicated stream read/write methods

The implementation is now production-ready and provides most features needed for multihash operations in Python.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

py-multihash Implementation Comparison: Gap Analysis with Go, Rust, and JavaScript #1061

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

py-multihash Implementation Comparison: Gap Analysis with Go, Rust, and JavaScript #1061

Uh oh!

Uh oh!

acul71 Nov 25, 2025 Maintainer

Multihash Implementation Comparison

Executive Summary

Feature Comparison Table

API Comparison

Core Encoding/Decoding

Python

Go

Rust

JavaScript

Hash Computation

Python

Go

Rust

JavaScript

String Encoding

Python

Go

Rust

JavaScript

Validation and Utilities

Python

Go

Rust

JavaScript

Hash Function Support

Python (py-multihash)

Go (go-multihash)

Rust (rust-multihash)

JavaScript (js-multiformats)

Missing Hash Functions in Python

Missing from Go Implementation

Missing from Rust Implementation

Summary

Gap Analysis

Summary of Missing Features in py-multihash

Critical Gaps in py-multihash

1. No Hash Computation

2. No Extensibility Mechanism

3. No Streaming Support

4. Limited Data Structures

5. No Truncation Support

6. No Serialization Support

Minor Gaps

7. No Type Safety

8. Limited Error Types

9. No Async Support

Recommendations

Priority 1: Critical Features

Priority 2: Important Features

Priority 3: Nice-to-Have Features

Implementation Examples

Example: Adding Hash Computation

Example: Adding Streaming Support

Conclusion

Replies: 1 comment

Uh oh!

acul71 Dec 13, 2025 Maintainer Author

Medium Priority

Low Priority

Full Review

py-multihash Implementation Status Report

Executive Summary

Key Findings

Detailed Feature Comparison

1. Core Operations

2. Hash Computation

3. String Encoding

4. Data Structures

5. Extensibility

6. Advanced Features

7. Error Handling

8. Streaming I/O

9. Serialization

10. Hash Function Support

Functions in Func Enum (Computable):

Functions in Constants but NOT in Func Enum (Not Computable):

acul71
Nov 25, 2025
Maintainer

acul71
Dec 13, 2025
Maintainer Author