Skip to content

Commit 52888ae

Browse files
authored
Use a fixed hash seed in perf_compare script (#18285)
With a random hash seed the measurements can vary a lot even for different builds based on the same commit. Some builds were consistently faster/slower than others, even though there were no code changes. This makes the measurements more predictable. It looks like mypyc output has some randomness, though I haven't looked into the root cause in detail.
1 parent bec5cad commit 52888ae

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

misc/perf_compare.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ def build_mypy(target_dir: str) -> None:
3939
env = os.environ.copy()
4040
env["CC"] = "clang"
4141
env["MYPYC_OPT_LEVEL"] = "2"
42+
env["PYTHONHASHSEED"] = "1"
4243
cmd = [sys.executable, "setup.py", "--use-mypyc", "build_ext", "--inplace"]
4344
subprocess.run(cmd, env=env, check=True, cwd=target_dir)
4445

@@ -60,6 +61,7 @@ def run_benchmark(compiled_dir: str, check_dir: str) -> float:
6061
shutil.rmtree(cache_dir)
6162
env = os.environ.copy()
6263
env["PYTHONPATH"] = os.path.abspath(compiled_dir)
64+
env["PYTHONHASHSEED"] = "1"
6365
abschk = os.path.abspath(check_dir)
6466
cmd = [
6567
sys.executable,

0 commit comments

Comments
 (0)