Skip to content

Commit

Permalink
Use a fixed hash seed in perf_compare script (python#18285)
Browse files Browse the repository at this point in the history
With a random hash seed the measurements can vary a lot even for
different builds based on the same commit. Some builds were consistently
faster/slower than others, even though there were no code changes. This
makes the measurements more predictable.

It looks like mypyc output has some randomness, though I haven't looked
into the root cause in detail.
  • Loading branch information
JukkaL authored Dec 12, 2024
1 parent bec5cad commit 52888ae
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions misc/perf_compare.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ def build_mypy(target_dir: str) -> None:
env = os.environ.copy()
env["CC"] = "clang"
env["MYPYC_OPT_LEVEL"] = "2"
env["PYTHONHASHSEED"] = "1"
cmd = [sys.executable, "setup.py", "--use-mypyc", "build_ext", "--inplace"]
subprocess.run(cmd, env=env, check=True, cwd=target_dir)

Expand All @@ -60,6 +61,7 @@ def run_benchmark(compiled_dir: str, check_dir: str) -> float:
shutil.rmtree(cache_dir)
env = os.environ.copy()
env["PYTHONPATH"] = os.path.abspath(compiled_dir)
env["PYTHONHASHSEED"] = "1"
abschk = os.path.abspath(check_dir)
cmd = [
sys.executable,
Expand Down

0 comments on commit 52888ae

Please sign in to comment.