hash() method not returning same value across python versions

[Jisan09]

Bug report

Bug description:

Value from hash isn't matching over python versions, turned off Randomization with PYTHONHASHSEED. Till python 3.10 it was showing 3672830208859661989 for hash of str(3) , but from python 3.11 it became -6250727467951016618.

Any way to get same 3672830208859661989 on higher python versions?

export PYTHONHASHSEED=0

Python 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> hash("3")
3672830208859661989

export PYTHONHASHSEED=0

Python 3.13.1 (v3.13.1:06714517797, Dec  3 2024, 14:00:22) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> hash("3")
-6250727467951016618

CPython versions tested on:

3.9, 3.10, 3.11, 3.13

Operating systems tested on:

Linux, macOS

[dolfinus]

Why this is considered a bug?

[Jisan09]

Why this is considered a bug?

@dolfinus for me its a bug because i was using this to generate key & check data duplicate or not in database. Till 3.10 its fine, but with newer python the hash is getting changed even with same hashseed. Now it will make same data again with new key.

import ctypes

def gen_key(key_string):
    return str(ctypes.c_size_t(hash(key_string)).value)

[skirpichev]

i was using this to generate key & check data duplicate or not in database.

Could you explain why you assumed, that hashing algorithms are part of the public CPython API? Did you read that somewhere in official docs?

If not, probably it's not a CPython issue, isn't?

Till 3.10 its fine, but with newer python the hash is getting change

That's due to #73596, I think. And that's fine. Hashing of the float('nan') was changed recently in the CPython, for example.

[Jisan09]

@skirpichev i checked cpython changelog it said

siphash13 is added as a new internal hashing algorithm. It has similar security properties as siphash24, but it is slightly faster for long inputs. str, bytes, and some other types now use it as the default algorithm for hash().

anyhow my bad if i marked it wrongly as bug, but my doubt remains same can we make same hash as older python with newer version?

[skirpichev]

You can use some other function, instead of calling hash() builtin. Your function may implement same hashing algorithm for strings as in CPython versions <= 3.10.

[ARTHURFLECK1828]

While using a different function will solve this issue , while upgrading legacy systems from 3.10 to 3.11, hashes previously generated using 3.10 cannot be used along with 3.11 especially in case of large sets of hashes, we will be forced to regenerate the entire hash set which is not feasible would be great if there was a backward compatible function for the same

[tim-one]

The language doesn't guarantee anything about specific hash values, not across implementations, not across releases of the CPython implementation, and not even across multiple runs under a single release.

It's very unlikely that will change. __hash__() exists primarily to help implement dicts and sets. Changes have been made more than once over the years to get better performance from those collection types.

But you have the source code, and can patch it in any way like, or pay someone to do that for you. That's your best hope to reintroduce an older release's hash results.

Some version of the older algorithm is documented in the PEP that introduced the current algorithm, or you could copy the code from the older version of CPython you care about.

[skirpichev]

would be great if there was a backward compatible function for the same

If you think it's a common problem, perhaps a package on PyPI could be a good idea. I don't think that stdlib is the right place for this stuff. But feel free to propose such idea on https://discuss.python.org/c/ideas/6

The language doesn't guarantee a lot of stuff, hashing - just a little example. Say, sum() changed internal algorithm for summation of floating-point numbers in the CPython 3.8. Such changes are unavoidable for a live language, sorry about that. Don't rely on undocumented behavior.

[Jisan09]

@ARTHURFLECK1828 This should help, for generate same hash for string. lucky that found a package :)

import logging
import siphash24


def gen_key(data: str, old_method=True) -> int:
    if data and isinstance(data, str):
        seed = int(os.environ.get("PYTHONHASHSEED", "0"))
        key = seed.to_bytes(16, byteorder='little')
        hash_class = siphash24.siphash24 if old_method else siphash24.siphash13
        int_hash = hash_class(bytes(data.encode('utf-8')), key=key).intdigest()
        return int_hash
    logging.warn("Give string value to generate hash...")
    return 0