`hashlib` add Protocol for hash-objects

[randolf-scholz]

Feature or enhancement

hashlib describes a Protocol for hash objects without implementing it.
The proposal is simply to create a concreate Hash or HashObject protocol that makes it explicit.

Pitch

Using the Buffer Protocol (PEP 688) this should look something like

@runtime_checkable
class Hash(Protocol):
    name: str

    def digest_size(self) -> int:
        """Return the size of the hash in bytes."""

    def block_size(self) -> int:
        """Return the internal block size of the hash in bytes."""

    def update(self, data: Buffer) -> None:
        """Update this hash object's state with the provided string."""

    def digest(self) -> bytes:
        """Return the digest value as a string of binary data."""

    def hexdigest(self) -> str:
        """Return the digest value as a string of hexadecimal digits."""

    def copy(self) -> Self:
        """Return a copy ("clone") of the hash object."""

Having a Protocol class instead of prose makes it clear how exactly a hash-function needs to look like.

A second suggestion would be to change hashlib.algorithms_guaranteed and hashlib.algorithms_available from set[str] to dict[str, type[Hash]] objects.

[JelleZijlstra]

I'm generally in support of this; it's useful for typing and more self-documenting.

The current protocol in typeshed looks a little different: https://github.com/python/typeshed/blob/main/stdlib/hashlib.pyi#L51. We should probably follow it.

We can't quite use PEP 688 yet because it has not been accepted and implemented. Hopefully it'll be accepted in time for Python 3.12.

It may break some users if we change the type for algorithms_guaranteed. It would probably be better to add new module members of dict types.

[randolf-scholz]

It may break some users if we change the type for algorithms_guaranteed. It would probably be better to add new module members of dict types.

Another option would be to introduce it as ALGORITHMS_GUARANTEED: Final[dict[str, type[Hash]]], since this is essentially a module-constant.

[hauntsaninja]

Thanks for the suggestion! I wouldn't necessarily make the Protocol runtime_checkable. Half the interface here is properties and the way runtime_checkable works with properties can be surprising. runtime_checkable is also slow, especially so as Protocol size increases, and performance cliffs in isinstance are pretty tricky to detect.

[fochoao]

Add protocol for hash objects.

[sobolevn]

I think that an abstract base class Hash with .register of each individual hash type would be a better fit here.

I also don't quite like the idea to add annotations to this base abstract class. Because both Buffer and Self are quite advanced. Annotations are stored in typeshed for now.

But, don't forget that there's also https://github.com/python/typeshed/blob/cc674b5dcef0c14612e79be255d0b2a8b0e11f81/stdlib/hashlib.pyi#L126 what should we do with it?

[gpshead]

Shouldn't something like this just live in typeshed first?

Also be aware that one size doesn't fit all for hash functions, some require additional arguments and have additional attributes. Your common SHA1, 2, 3, and MD5 series will all look the same. Others, including future standards... may not.

As for changing haslib.algorithms_* to be dict rather than set: no. They must stay sets to avoid breaking code. It isn't clear to me what use such a mapping would even be. If you want to propose adding such a map, demonstrate how and when and why it'd be used.

[picnixz]

There are two interfaces that we currently support. "Normal" Hash Functions and Extendable-Output Hash Functions (XOF) which are SHAKE-128 and SHAKE-256. All hash functions have the same API that is defined by PEP-247 and PEP-452. We need to have the following protocols:

class Hash(Protocol):
    name: str
    digest_size: int
    block_size: int

    def update(self, data: Buffer) -> None: ...
    def digest(self) -> bytes: ...
    def hexdigest(self) -> str: ...
    def copy(self) -> Self: ...

class HashXOF(Protocol):
    name: str
    digest_size: int  # should be 0 as it varies
    block_size: int

    def update(self, data: Buffer) -> None: ...
    def digest(self, length: int) -> bytes: ...
    def hexdigest(self, length: int) -> str: ...
    def copy(self) -> Self: ...

Unfortunately, we can't make length positional-only for compatibility reasons. Now, we don't need to suggest a protocol for __init__.

For future hashes, copy() won't change (it should not) but update, digest and hexdigest may need to take additional parameters. If this happens, we will likely need to create a new protocol.

Overall, I'm fine with adding those two protocol classes but they are not compatible with each others and we already have something well-defined in the docs: https://docs.python.org/3/library/hashlib.html#hash-objects.

[gpshead]

I'd still suggest just going this in typeshed given the use case for these is likely entirely in the typing realm. Believed correct and accurate or not, it is very hard to meaningfully iterate on the shapes of non-trivial typing shapes within the standard library. typeshed allows such rapid iteration and updates without a Python upgrade when a shape has been deemed inaccurate for some purpose.