Skip to content

Conversation

EdmundGoodman
Copy link
Owner

@EdmundGoodman EdmundGoodman commented Jul 23, 2025

As discussed in the xDSL Zulip, this PR adds a benchmark for constant folding with xDSL.

The new benchmark appears to run as expected locally:

$ pyperformance run --benchmark xdsl                                                                                         
Python benchmark suite 1.11.0


==================================================
( 1/1) creating venv for benchmark (xdsl)

(trying common venv first)
Installing requirements into the virtual environment /Users/edjg/.../pyperformance/venv/cpython3.12-30629e8c4b43-compat-64aaec271e6b
# /Users/edjg/.../pyperformance/venv/cpython3.12-30629e8c4b43-compat-64aaec271e6b/bin/python -m pip install xdsl==0.46.0 pyperf==2.9.0
Requirement already satisfied: xdsl==0.46.0 in ./venv/cpython3.12-30629e8c4b43-compat-64aaec271e6b/lib/python3.12/site-packages (0.46.0)
Requirement already satisfied: pyperf==2.9.0 in ./venv/cpython3.12-30629e8c4b43-compat-64aaec271e6b/lib/python3.12/site-packages (2.9.0)
Requirement already satisfied: immutabledict<4.2.2 in ./venv/cpython3.12-30629e8c4b43-compat-64aaec271e6b/lib/python3.12/site-packages (from xdsl==0.46.0) (4.2.1)
Requirement already satisfied: typing-extensions<5,>=4.7 in ./venv/cpython3.12-30629e8c4b43-compat-64aaec271e6b/lib/python3.12/site-packages (from xdsl==0.46.0) (4.14.1)
Requirement already satisfied: ordered-set==4.1.0 in ./venv/cpython3.12-30629e8c4b43-compat-64aaec271e6b/lib/python3.12/site-packages (from xdsl==0.46.0) (4.1.0)
Requirement already satisfied: psutil>=5.9.0 in ./venv/cpython3.12-30629e8c4b43-compat-64aaec271e6b/lib/python3.12/site-packages (from pyperf==2.9.0) (7.0.0)

[notice] A new release of pip is available: 24.3.1 -> 25.1.1
[notice] To update, run: python -m pip install --upgrade pip
Installing requirements into the virtual environment /Users/edjg/.../pyperformance/venv/cpython3.12-30629e8c4b43-compat-64aaec271e6b
# /Users/edjg/.../pyperformance/venv/cpython3.12-30629e8c4b43-compat-64aaec271e6b/bin/python -m pip install psutil
Requirement already satisfied: psutil in ./venv/cpython3.12-30629e8c4b43-compat-64aaec271e6b/lib/python3.12/site-packages (7.0.0)

[notice] A new release of pip is available: 24.3.1 -> 25.1.1
[notice] To update, run: python -m pip install --upgrade pip
# /Users/edjg/.../pyperformance/venv/cpython3.12-30629e8c4b43-compat-64aaec271e6b/bin/python -m pip freeze
immutabledict==4.2.1
ordered-set==4.1.0
psutil==7.0.0
pyperf==2.9.0
setuptools==80.9.0
typing_extensions==4.14.1
wheel==0.45.1
xdsl==0.46.0
# /Users/edjg/.../pyperformance/venv/cpython3.12-30629e8c4b43-compat-64aaec271e6b/bin/python -m pip freeze
immutabledict==4.2.1
ordered-set==4.1.0
psutil==7.0.0
pyperf==2.9.0
setuptools==80.9.0
typing_extensions==4.14.1
wheel==0.45.1
xdsl==0.46.0

[1/1] xdsl...
# /Users/edjg/.../pyperformance/venv/cpython3.12-30629e8c4b43-compat-64aaec271e6b/bin/python -u /Users/edjg/.../pyperformance/pyperformance/data-files/benchmarks/bm_xdsl/run_benchmark.py --output /var/folders/dw/qzkqk1s50cd9stxcw22_ymj80000gn/T/tmp4lrfn0vn --inherit-environ PYPERFORMANCE_RUNID
.....................
WARNING: the benchmark result may be unstable
* Not enough samples to get a stable result (95% certainly of less than 1% variation)

Try to rerun the benchmark with more runs, values and/or loops.
Run 'python -m pyperf system tune' command to reduce the system jitter.
Use pyperf stats, pyperf dump and pyperf hist to analyze results.
Use --quiet option to hide these warnings.

xdsl_constant_fold: Mean +- std dev: 24.3 ms +- 1.0 ms

Performance version: 1.11.0
Python version: 3.12.9 (64-bit)
Report on macOS-15.5-arm64-arm-64bit
Number of logical CPUs: 8
Start date: 2025-07-23 12:32:57.663942
End date: 2025-07-23 12:33:18.527806

### xdsl_constant_fold ###
Mean +- std dev: 24.3 ms +- 1.0 ms

[project]
name = "pyperformance_bm_xdsl"
requires-python = ">=3.10"
dependencies = ["pyperf", "xdsl"]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to specify an xdsl version here? I forgot how pyproject.toml specifications work. Presumably yes?

@EdmundGoodman
Copy link
Owner Author

Moved to PR main repo here: python#403

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants