Skip to content

Commit 771f676

Browse files
williamsyywilliamsyy
authored
Updates for macOS Stability with NN Backend Options and Pytest
2 parents bd20dfc + 2bce40c commit 771f676

20 files changed

+4485
-408
lines changed

‎.github/workflows/test.yml‎

Lines changed: 29 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ jobs:
1212
runs-on: ubuntu-22.04
1313
strategy:
1414
matrix:
15-
python-version: ["3.8", "3.10", "3.13"]
15+
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"] # Skip 3.13 for now
1616

1717
steps:
1818
- name: Checkout code
@@ -23,6 +23,18 @@ jobs:
2323
with:
2424
python-version: ${{ matrix.python-version }}
2525

26+
- name: Install uv
27+
uses: astral-sh/setup-uv@v5
28+
29+
- name: Cache uv
30+
uses: actions/cache@v4
31+
with:
32+
path: ~/.cache/uv
33+
key: ${{ runner.os }}-uv-${{ matrix.python-version }}-${{ hashFiles('uv.lock') }}
34+
restore-keys: |
35+
${{ runner.os }}-uv-${{ matrix.python-version }}-
36+
${{ runner.os }}-uv-
37+
2638
- name: Cache pip packages
2739
uses: actions/cache@v4
2840
with:
@@ -45,3 +57,19 @@ jobs:
4557

4658
- name: Run tests
4759
run: make test
60+
61+
test-python313:
62+
name: Test Python 3.13 (without UV)
63+
runs-on: ubuntu-latest
64+
steps:
65+
- uses: actions/checkout@v4
66+
- uses: actions/setup-python@v5
67+
with:
68+
python-version: "3.13"
69+
- name: Install dependencies
70+
run: |
71+
python -m pip install --upgrade pip
72+
python -m pip install -r requirements-test-py313.txt
73+
python -m pip install -e ".[all-git]"
74+
- name: Run tests
75+
run: pytest test/

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,7 @@
22
/test/output/
33
.DS_store
44
.vscode
5+
.venv
6+
build/
7+
58
__pycache__/

‎.python-version‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
3.12

‎Makefile‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,19 @@
11
.PHONY: install test clean
22

3+
requirements-test.txt:
4+
uv pip compile --group dev pyproject.toml -o requirements-test.txt
5+
6+
requirements.txt:
7+
uv pip compile pyproject.toml -o requirements.txt
8+
39
install-dev:
4-
pip install -e .
5-
pip install -r requirements-test.txt
10+
uv sync --locked --all-extras --dev
11+
12+
install:
13+
uv sync --locked
614

715
test:
8-
pytest \
16+
uv run pytest \
917
test/test_general.py \
1018
test/test_transform_iris.py \
1119
test/test_randomness.py \
@@ -14,9 +22,11 @@ test:
1422
test/test_metric.py
1523

1624
clean:
17-
pip uninstall -y pacmap
1825
rm -rf build/
1926
rm -rf dist/
2027
rm -rf *.egg-info/
2128
rm -rf source/*.egg-info/
2229
rm -rf test/output/
30+
31+
build:
32+
uv build

‎README.md‎

Lines changed: 41 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -55,22 +55,33 @@ mamba install pacmap -c conda-forge
5555
### <a name='InstallfromPyPIviapip'></a>Install from PyPI via pip
5656

5757
You can use [pip](https://pip.pypa.io/en/stable/) to install pacmap from PyPI.
58-
It will automatically install the dependencies for you:
5958

59+
**Basic installation** (includes FAISS as the default KNN backend):
6060
```bash
6161
pip install pacmap
6262
```
6363

64-
If you have any problems during the installation of dependencies, such as
65-
`Failed building wheel for annoy`, you can try to install these dependencies
66-
with `conda` or `mamba`. Users have also reported that in some cases, you may
67-
wish to use `numba >= 0.57`.
64+
**Optional KNN backends:**
65+
66+
PaCMAP supports multiple KNN backends. Install optional backends as needed:
6867

6968
```bash
70-
conda install -c conda-forge python-annoy
71-
pip install pacmap
69+
# Install with Annoy backend
70+
pip install pacmap[annoy]
71+
72+
# Install with Voyager backend
73+
pip install pacmap[voyager]
74+
75+
# Install all optional backends
76+
pip install pacmap[all]
7277
```
7378

79+
> **Note:** The original PaCMAP paper used Annoy as the default KNN backend. Since the Annoy package is no longer actively maintained, we have switched to FAISS as the default backend for better long-term stability and performance. Annoy remains available as an optional backend for compatibility.
80+
81+
If you have any problems during the installation, you can try installing dependencies
82+
with `conda` or `mamba`. Users have also reported that in some cases, you may
83+
wish to use `numba >= 0.57`.
84+
7485
## <a name='Usage'></a>Usage
7586

7687
### <a name='UsingPaCMAPinPython'></a>Using PaCMAP in Python
@@ -146,6 +157,8 @@ Other parameters include:
146157
- `lr`: learning rate of the AdaGrad optimizer. Default to 1.
147158
- `apply_pca`: whether pacmap should apply PCA to the data before constructing the k-Nearest Neighbor graph. Using PCA to preprocess the data can largely accelerate the DR process without losing too much accuracy. Notice that this option does not affect the initialization of the optimization process.
148159
- `intermediate`: whether pacmap should also output the intermediate stages of the optimization process of the lower dimension embedding. If `True`, then the output will be a numpy array of the size (n, `n_components`, 13), where each slice is a "screenshot" of the output embedding at a particular number of steps, from [0, 10, 30, 60, 100, 120, 140, 170, 200, 250, 300, 350, 450].
160+
- `nn_backend`: the backend used to construct the k-Nearest Neighbor graph. One of `"annoy"`, `"faiss"`, or `"voyager"`. Default to `"faiss"`. **Note:** The original paper used Annoy, but FAISS is now the default since Annoy is no longer actively maintained.
161+
- `low_dist_thres`[Only Used in `LocalMAP`]: The Proximal Cluster Distance Commons, the acceptance distance threshold for selecting local FP with distance no larger than the average low-dimension distance among all nearest clusters pair. Default to 10.
149162

150163
## <a name='Methods'></a>Methods
151164

@@ -173,7 +186,27 @@ We test against several version of Python. (See [`.github/workflows/test.yml`][w
173186

174187
If you are contributing code, please confirm our tests pass, and consider adding your own for any new functionality.
175188

176-
You may run the test suite like so:
189+
### Development with UV (Recommended)
190+
191+
We use [UV](https://docs.astral.sh/uv/) for fast, reproducible dependency management. To set up your development environment:
192+
193+
```bash
194+
# Install UV (if not already installed)
195+
curl -LsSf https://astral.sh/uv/install.sh | sh
196+
197+
# Sync dependencies (creates .venv automatically)
198+
uv sync --dev
199+
200+
# Run tests
201+
uv run pytest
202+
203+
# Or use make
204+
make test
205+
```
206+
207+
### Development with pip (Alternative)
208+
209+
You may also use traditional pip-based workflow:
177210

178211
```sh
179212
# Clean up past outputs (plots will be DELETED),

‎pyproject.toml‎

Lines changed: 74 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,69 @@
11
[build-system]
2-
requires = [
3-
"setuptools>=42",
4-
"wheel"
2+
requires = ["hatchling"]
3+
build-backend = "hatchling.build"
4+
5+
[project]
6+
name = "pacmap"
7+
version = "0.9.0"
8+
description = "The official implementation for PaCMAP: Pairwise Controlled Manifold Approximation Projection"
9+
readme = "README.md"
10+
license = {text = "Apache-2.0"}
11+
authors = [
12+
{name = "Yingfan Wang", email = "yingfan.wang@duke.edu"},
13+
{name = "Haiyang Huang"},
14+
{name = "Cynthia Rudin"},
15+
{name = "Yaron Shaposhnik"},
16+
]
17+
maintainers = [
18+
{name = "Yingfan Wang", email = "yingfan.wang@duke.edu"},
19+
]
20+
keywords = ["dimensionality reduction", "manifold learning", "visualization", "embedding"]
21+
classifiers = [
22+
"Programming Language :: Python :: 3",
23+
"Programming Language :: Python :: 3.8",
24+
"Programming Language :: Python :: 3.9",
25+
"Programming Language :: Python :: 3.10",
26+
"Programming Language :: Python :: 3.11",
27+
"Programming Language :: Python :: 3.12",
28+
"License :: OSI Approved :: Apache Software License",
29+
"License :: OSI Approved :: MIT License",
30+
"Operating System :: OS Independent",
31+
"Intended Audience :: Science/Research",
32+
"Intended Audience :: Developers",
33+
"Topic :: Scientific/Engineering",
34+
"Topic :: Scientific/Engineering :: Artificial Intelligence",
35+
"Topic :: Scientific/Engineering :: Visualization",
536
]
6-
build-backend = "setuptools.build_meta"
37+
requires-python = ">=3.8"
38+
dependencies = [
39+
"scikit-learn>=0.20",
40+
"numba>=0.57",
41+
"numpy>=1.20",
42+
"faiss-cpu",
43+
]
44+
45+
[project.optional-dependencies]
46+
annoy = ["annoy>=1.17,<2"]
47+
voyager-pypi = ["voyager>=2.0"] # For Python 3.8-3.12
48+
voyager-git = ["voyager @ git+https://github.com/spotify/voyager.git#subdirectory=python"] # For Python 3.13+
49+
voyager = ["voyager>=2.0"] # Default to PyPI
50+
all = ["annoy>=1.17,<2", "voyager>=2.0"]
51+
all-git = ["annoy>=1.17,<2", "voyager @ git+https://github.com/spotify/voyager.git#subdirectory=python"]
52+
53+
[project.urls]
54+
Homepage = "https://github.com/YingfanWang/PaCMAP"
55+
Repository = "https://github.com/YingfanWang/PaCMAP"
56+
Documentation = "https://github.com/YingfanWang/PaCMAP"
57+
"Bug Tracker" = "https://github.com/YingfanWang/PaCMAP/issues"
58+
59+
[tool.hatch.metadata]
60+
allow-direct-references = true
61+
62+
[tool.hatch.build.targets.sdist]
63+
include = ["source/pacmap"]
64+
65+
[tool.hatch.build.targets.wheel]
66+
packages = ["source/pacmap"]
767

868
[tool.pytest.ini_options]
969
testpaths = ["test"]
@@ -16,3 +76,13 @@ addopts = [
1676
# Throw errors for typos in test markers.
1777
"--strict-markers",
1878
]
79+
80+
[dependency-groups]
81+
dev = [
82+
"pytest>=6.0",
83+
"pytest-cov",
84+
"matplotlib>=3.0",
85+
"annoy>=1.17,<2", # For testing annoy backend
86+
"voyager>=2.0", # For testing voyager backend
87+
"pandas>=1.0", # For testing openml datasets
88+
]

‎release_notes.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,13 @@
11
# Release Notes
22

3+
- 0.9.0
4+
5+
Add `Voyager` and `FAISS` backend within the package ([136](https://github.com/YingfanWang/PaCMAP/issues/136) and [94](https://github.com/YingfanWang/PaCMAP/issues/94)) based on the pull request from @Tyrannas in https://github.com/YingfanWang/PaCMAP/pull/137.
6+
7+
Fixed https://github.com/YingfanWang/PaCMAP/issues/127, https://github.com/YingfanWang/PaCMAP/issues/128 for PaCMAP hangs
8+
9+
Improve the normalization to avoid small xmax
10+
311
- 0.8.0
412
Add `LocalMAP` algorithm to release.
513

‎requirements-test-py313.txt‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# This file is for Python 3.13 testing where Annoy and Voyager are installed separately via [all-git]
2+
# It includes all dev dependencies except the optional backends
3+
4+
coverage==7.13.3
5+
# via pytest-cov
6+
contourpy==1.3.3
7+
# via matplotlib
8+
cycler==0.12.1
9+
# via matplotlib
10+
faiss-cpu==1.13.2
11+
# via pacmap (required dependency)
12+
fonttools==4.61.1
13+
# via matplotlib
14+
iniconfig==2.3.0
15+
# via pytest
16+
joblib==1.5.3
17+
# via scikit-learn
18+
kiwisolver==1.4.9
19+
# via matplotlib
20+
llvmlite==0.46.0
21+
# via numba
22+
matplotlib==3.10.8
23+
# via testing
24+
numba==0.63.1
25+
# via pacmap
26+
numpy==2.3.5
27+
# via pacmap and various dependencies
28+
packaging==26.0
29+
# via faiss-cpu, matplotlib, pytest
30+
pandas>=1.0
31+
# via testing openml datasets
32+
pillow==12.1.0
33+
# via matplotlib
34+
pluggy==1.6.0
35+
# via pytest, pytest-cov
36+
pygments==2.19.2
37+
# via pytest
38+
pyparsing==3.3.2
39+
# via matplotlib
40+
pytest==9.0.2
41+
# via testing
42+
pytest-cov==7.0.0
43+
# via testing
44+
python-dateutil==2.9.0.post0
45+
# via matplotlib
46+
scikit-learn==1.8.0
47+
# via pacmap
48+
scipy==1.17.0
49+
# via scikit-learn
50+
six==1.17.0
51+
# via python-dateutil
52+
threadpoolctl==3.6.0
53+
# via scikit-learn

0 commit comments

Comments
 (0)