Skip to content

Commit 414c9e6

Browse files
MatthewSZhangMatthewSZhang
authored
MNT add asv benchmarks (#137)
* MNT add asv benchmarks
1 parent 1987a1a commit 414c9e6

File tree

16 files changed

+4086
-1091
lines changed

16 files changed

+4086
-1091
lines changed

‎.github/workflows/asv.yml‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
name: ASV benchmarks
2+
3+
on:
4+
push:
5+
branches: ["main"]
6+
pull_request:
7+
branches: ["main"]
8+
9+
jobs:
10+
run-benchmarks:
11+
name: Benchmark on ${{ matrix.os }}
12+
runs-on: ${{ matrix.os }}
13+
permissions:
14+
contents: read
15+
strategy:
16+
fail-fast: false
17+
matrix:
18+
os: [ubuntu-latest, windows-latest, macos-latest, ubuntu-24.04-arm]
19+
20+
steps:
21+
- uses: actions/checkout@v5
22+
with:
23+
fetch-depth: 0
24+
25+
- name: Set up main branch
26+
run: |
27+
git branch main origin/main
28+
29+
- uses: prefix-dev/[email protected]
30+
with:
31+
environments: dev
32+
cache: true
33+
34+
- name: Run benchmarks
35+
shell: bash
36+
run: |
37+
MACHINE=${{ matrix.os }} pixi run asv-build
38+
39+
- name: Upload benchmark results
40+
uses: actions/upload-artifact@v4
41+
with:
42+
name: asv-results-${{ matrix.os }}
43+
path: asv_benchmarks/results
44+
45+
publish-report:
46+
name: Build HTML report
47+
runs-on: ubuntu-latest
48+
needs: run-benchmarks
49+
permissions:
50+
contents: write
51+
52+
steps:
53+
- uses: actions/checkout@v5
54+
with:
55+
fetch-depth: 0
56+
57+
- name: Set up main branch
58+
run: |
59+
git branch main origin/main
60+
61+
- name: Prepare previous ASV results
62+
uses: actions/checkout@v5
63+
continue-on-error: true
64+
with:
65+
ref: gh-pages
66+
path: gh-pages
67+
68+
- name: Copy previous results
69+
run: |
70+
mkdir -p asv_benchmarks/results
71+
cp -r gh-pages/results/* asv_benchmarks/results/ 2>/dev/null || true
72+
73+
- name: Download all benchmark results
74+
uses: actions/download-artifact@v5
75+
with:
76+
pattern: asv-results-*
77+
78+
- name: Merge new benchmark results
79+
run: |
80+
for d in asv-results-*; do
81+
[ -d "$d" ] || continue
82+
cp -r "$d"/* "asv_benchmarks/results/"
83+
done
84+
85+
- uses: prefix-dev/[email protected]
86+
with:
87+
environments: dev
88+
cache: true
89+
90+
- name: Generate HTML report
91+
run: |
92+
pixi run asv-publish
93+
94+
- name: Copy results to publish directory
95+
run: |
96+
cp -r asv_benchmarks/results html/
97+
98+
- name: Deploy to GitHub Pages
99+
uses: peaceiris/actions-gh-pages@v4
100+
if: github.event_name == 'push'
101+
with:
102+
github_token: ${{ secrets.GITHUB_TOKEN }}
103+
publish_dir: ./html
104+
keep_files: true
105+
user_name: 'github-actions[bot]'
106+
user_email: 'github-actions[bot]@users.noreply.github.com'
107+
commit_message: ${{ github.event.head_commit.message }}

‎.github/workflows/emscripten.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ jobs:
77
build-wasm-wheel:
88
runs-on: ubuntu-latest
99
steps:
10-
- uses: actions/checkout@v4
10+
- uses: actions/checkout@v5
1111
- name: Build WASM wheel
1212
uses: pypa/[email protected]
1313
env:

‎.github/workflows/publish-pypi.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ jobs:
1818
id-token: write
1919
steps:
2020
- name: Download artifacts
21-
uses: actions/download-artifact@v4
21+
uses: actions/download-artifact@v5
2222
with:
2323
path: dist/
2424
merge-multiple: true
@@ -27,9 +27,9 @@ jobs:
2727
- name: Publish distribution to PyPI
2828
if: github.event.release.prerelease == false
2929
uses: pypa/gh-action-pypi-publish@release/v1
30-
30+
3131
- name: get wasm dist artifacts
32-
uses: actions/download-artifact@v4
32+
uses: actions/download-artifact@v5
3333
with:
3434
name: wasm_wheel
3535
path: wasm/

‎.github/workflows/static.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,8 @@ jobs:
88
runs-on: ubuntu-latest
99

1010
steps:
11-
- uses: actions/checkout@v4
12-
- uses: prefix-dev/setup-pixi@v0.8.14
11+
- uses: actions/checkout@v5
12+
- uses: prefix-dev/setup-pixi@v0.9.0
1313
with:
1414
environments: static
1515
cache: true

‎.github/workflows/test.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,8 @@ jobs:
1515
runs-on: ${{ matrix.os }}
1616

1717
steps:
18-
- uses: actions/checkout@v4
19-
- uses: prefix-dev/setup-pixi@v0.8.14
18+
- uses: actions/checkout@v5
19+
- uses: prefix-dev/setup-pixi@v0.9.0
2020
with:
2121
environments: >-
2222
dev

‎.github/workflows/wheel.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,8 @@ jobs:
77
build-sdist:
88
runs-on: ubuntu-latest
99
steps:
10-
- uses: actions/checkout@v4
11-
- uses: prefix-dev/setup-pixi@v0.8.14
10+
- uses: actions/checkout@v5
11+
- uses: prefix-dev/setup-pixi@v0.9.0
1212
with:
1313
environments: dev
1414
cache: true
@@ -31,7 +31,7 @@ jobs:
3131
os: [ubuntu-latest, windows-latest, macos-latest, ubuntu-24.04-arm]
3232
runs-on: ${{ matrix.os }}
3333
steps:
34-
- uses: actions/checkout@v4
34+
- uses: actions/checkout@v5
3535
- name: Build wheels
3636
uses: pypa/[email protected]
3737
env:

‎asv_benchmarks/.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
*__pycache__*
2+
env/
3+
html/
4+
results/
5+
benchmarks/cache/

‎asv_benchmarks/asv.conf.json‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
"version": 1,
3+
"project": "fastcan",
4+
"project_url": "https://github.com/scikit-learn-contrib/fastcan",
5+
"show_commit_url": "https://github.com/scikit-learn-contrib/fastcan/commit/",
6+
"repo": "..",
7+
"branches": ["main"],
8+
"environment_type": "conda",
9+
"conda_channels": ["conda-forge"],
10+
"build_command": ["python -m build --wheel -o {build_cache_dir} {build_dir}"],
11+
"install_command": ["python -mpip install {wheel_file}"],
12+
"uninstall_command": ["return-code=any python -mpip uninstall -y {project}"],
13+
"pythons": ["3.13"],
14+
"matrix": {
15+
"scikit-learn": [""],
16+
"pandas": [""]
17+
}
18+
}

‎asv_benchmarks/benchmarks/__init__.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
"""Benchmark suite for fastcan using ASV"""

‎asv_benchmarks/benchmarks/common.py‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
import pickle
2+
import timeit
3+
from abc import ABC, abstractmethod
4+
from pathlib import Path
5+
6+
7+
def get_estimator_path(benchmark, params):
8+
"""Get path of pickled fitted estimator"""
9+
path = Path(__file__).resolve().parent / "cache" / "estimators"
10+
11+
filename = (
12+
benchmark.__class__.__name__
13+
+ "_estimator_"
14+
+ "_".join(list(map(str, params)))
15+
+ ".pkl"
16+
)
17+
18+
return path / filename
19+
20+
21+
class Benchmark(ABC):
22+
"""Abstract base class for all the benchmarks"""
23+
24+
timer = timeit.default_timer # wall time
25+
timeout = 500
26+
27+
# save estimators
28+
current_path = Path(__file__).resolve().parent
29+
cache_path = current_path / "cache"
30+
cache_path.mkdir(exist_ok=True)
31+
(cache_path / "estimators").mkdir(exist_ok=True)
32+
33+
def setup(self, *params):
34+
"""Generate dataset and load the fitted estimator"""
35+
# This is run once per combination of parameters and per repeat so we
36+
# need to avoid doing expensive operations there.
37+
38+
self.X, self.X_val, self.y, self.y_val = self.make_data(params)
39+
40+
est_path = get_estimator_path(self, params)
41+
with est_path.open(mode="rb") as f:
42+
self.estimator = pickle.load(f)
43+
44+
@abstractmethod
45+
def make_data(self, params):
46+
"""Return the dataset for a combination of parameters"""
47+
# The datasets are cached using joblib.Memory so it's fast and can be
48+
# called for each repeat
49+
50+
@property
51+
@abstractmethod
52+
def params(self):
53+
pass

0 commit comments

Comments
 (0)