Skip to content

Commit 388eb21

Browse files
ruben-artslucascolley
ruben-arts
and
lucascolley
authored
docs: explain the matchspec and source spec (#5224)
Co-authored-by: Lucas Colley <[email protected]>
1 parent c22166f commit 388eb21

File tree

7 files changed

+505
-12
lines changed

7 files changed

+505
-12
lines changed

‎docs/concepts/package_specifications.md‎

Lines changed: 311 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,311 @@
1+
# Package Specifications
2+
3+
When adding packages to your Pixi workspace or global environment, you can use various specifications to control exactly which package version and build you want.
4+
This is particularly important when packages have multiple builds for different hardware configurations (like CPU vs GPU).
5+
For the conda packages Pixi uses the [MatchSpec](https://rattler.prefix.dev/py-rattler/match_spec#matchspec) format to specify package requirements.
6+
For PyPI packages, Pixi uses the standard [PEP440 version specifiers](https://peps.python.org/pep-0440/).
7+
8+
## Quick Examples
9+
10+
=== "Add"
11+
```bash
12+
--8<-- "docs/source_files/shell/package_specifications.sh:quick-add-examples"
13+
```
14+
=== "Global"
15+
```bash
16+
--8<-- "docs/source_files/shell/package_specifications.sh:quick-global-examples"
17+
```
18+
=== "Exec"
19+
```bash
20+
--8<-- "docs/source_files/shell/package_specifications.sh:quick-exec-examples"
21+
```
22+
23+
## Basic Version Specifications
24+
Pixi uses the [**conda MatchSpec**](https://rattler.prefix.dev/py-rattler/match_spec#matchspec) format for specifying package requirements.
25+
A MatchSpec allows you to precisely define which package version, build, and channel you want.
26+
27+
The simplest way to specify a package is by name and [optional version operators](#version-operators):
28+
29+
```toml title="pixi.toml"
30+
--8<-- "docs/source_files/pixi_tomls/package_specifications.toml:basic-version"
31+
```
32+
33+
## Full MatchSpec Syntax
34+
35+
Beyond simple version specifications, you can use the full [MatchSpec](https://rattler.prefix.dev/py-rattler/match_spec#matchspec) syntax to precisely control which package variant you want.
36+
37+
### Command Line Syntax
38+
39+
Pixi supports two syntaxes on the command line:
40+
41+
**1. Equals syntax (compact):**
42+
```shell
43+
# Format: package=version=build
44+
pixi add "pytorch=2.0.*=cuda*"
45+
# Only build string (any version)
46+
pixi add "numpy=*=py311*"
47+
```
48+
49+
**2. Bracket syntax (explicit):**
50+
```shell
51+
# Format: package [key='value', ...]
52+
pixi add "pytorch [version='2.0.*', build='cuda*']"
53+
# Multiple constraints
54+
pixi add "numpy [version='>=1.21', build='py311*', channel='conda-forge']"
55+
# Build number constraint
56+
pixi add "python [version='3.11.0', build_number='>=1']"
57+
```
58+
59+
Both syntaxes are equivalent and can be used interchangeably.
60+
61+
### TOML Mapping Syntax
62+
63+
In your `pixi.toml`, use the mapping syntax for complete control:
64+
65+
```toml title="pixi.toml"
66+
--8<-- "docs/source_files/pixi_tomls/package_specifications.toml:mapping-syntax-full"
67+
```
68+
69+
This syntax allows you to specify:
70+
71+
- **version**: Version constraint using [operators](#version-operators)
72+
- **build**: Build string pattern (see [build strings](#build-strings))
73+
- **build-number**: Build number constraint (e.g., `">=1"`, `"0"`) (see [build number](#build-number))
74+
- **channel**: Specific channel name or full URL (see [channel](#channel))
75+
- **sha256/md5**: Package checksums for verification (see [checksums](#checksums-sha256md5))
76+
- **license**: Expected license type (see [license](#license))
77+
- **file-name**: Specific package file name (see [file name](#file-name))
78+
79+
### Version Operators
80+
81+
Pixi supports various version operators:
82+
83+
| Operator | Meaning | Example |
84+
|----------|---------|---------|
85+
| `==` | Exact match | `==3.11.0` |
86+
| `!=` | Not equal | `!=3.8` |
87+
| `<` | Less than | `<3.12` |
88+
| `<=` | Less than or equal | `<=3.11` |
89+
| `>` | Greater than | `>3.9` |
90+
| `>=` | Greater than or equal | `>=3.9` |
91+
| `~=` | Compatible release | `~=3.11.0` (>= 3.11.0, < 3.12.0) |
92+
| `*` | Wildcard | `3.11.*` (any 3.11.x) |
93+
| `,` | AND | `">=3.9,<3.12"` |
94+
| `|` | OR | `"3.10|3.11"` |
95+
96+
### Build Strings
97+
98+
Build strings identify specific builds of the same package version.
99+
They're especially important for packages that have different builds for:
100+
101+
- **Hardware acceleration**: CPU, GPU/CUDA builds
102+
- **Python versions**: Packages built for different Python interpreters
103+
- **Compiler variants**: Different compiler versions or configurations
104+
105+
A build string typically looks like: `py311h43a39b2_0`
106+
107+
Breaking it down:
108+
109+
- `py311`: Python version indicator
110+
- `h43a39b2`: Hash of the build configuration
111+
- `_0`: Build number
112+
113+
You can use wildcards in build patterns to match multiple builds:
114+
115+
```shell
116+
# Match any CUDA build
117+
pixi add "pytorch=*=cuda*"
118+
119+
# Match Python 3.11 builds
120+
pixi add "numpy=*=py311*"
121+
122+
# Using bracket syntax
123+
pixi add "pytorch [build='cuda*']"
124+
```
125+
126+
### Build Number
127+
128+
The build number is an integer that increments each time a package is rebuilt with the same version.
129+
Use build number constraints when you need a specific rebuild of a package:
130+
131+
```shell
132+
# Specific build number
133+
pixi add "python [version='3.11.0', build_number='1']"
134+
135+
# Build number constraint
136+
pixi add "numpy [build_number='>=5']"
137+
```
138+
139+
```toml title="pixi.toml"
140+
[dependencies.python]
141+
version = "3.11.0"
142+
build-number = ">=1"
143+
```
144+
145+
Build numbers are useful when:
146+
147+
- A package was rebuilt to fix a compilation issue
148+
- You need to ensure you have a specific rebuild with bug fixes
149+
- Working with reproducible environments that require exact builds
150+
151+
### Channel
152+
153+
Channels are repositories where conda packages are hosted.
154+
You can specify which channel to fetch a package from:
155+
156+
```shell
157+
# Specific channel by name
158+
pixi add "pytorch [channel='pytorch']"
159+
160+
# Channel URL
161+
pixi add "custom-package [channel='https://prefix.dev/my-channel']"
162+
163+
# Or use the shorter `::` syntax
164+
pixi add pytorch::pytorch
165+
pixi add https://prefix.dev/my-channel::custom-package
166+
```
167+
168+
```toml title="pixi.toml"
169+
[dependencies.pytorch]
170+
channel = "pytorch"
171+
172+
[dependencies.custom-package]
173+
channel = "https://prefix.dev/my-channel"
174+
```
175+
176+
Note that for `pixi add`, channels must also be listed in your workspace configuration:
177+
178+
```toml title="pixi.toml"
179+
[workspace]
180+
channels = ["conda-forge", "pytorch", "nvidia"]
181+
```
182+
183+
You can also add these using the command line:
184+
185+
```shell
186+
pixi workspace channel add conda-forge
187+
```
188+
189+
### Checksums (SHA256/MD5)
190+
191+
Checksums verify package integrity and authenticity.
192+
Use them for reproducibility and security:
193+
194+
```toml title="pixi.toml"
195+
[dependencies.numpy]
196+
version = "1.21.0"
197+
sha256 = "1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef"
198+
md5 = "abcdef1234567890abcdef1234567890"
199+
```
200+
201+
When specified, Pixi will:
202+
203+
- Verify the downloaded package matches the checksum
204+
- Fail the installation if checksums don't match
205+
- Ensure you get the exact package you expect
206+
207+
SHA256 is preferred over MD5 for security reasons.
208+
209+
### License
210+
211+
Specify the expected license of a package.
212+
This is useful for:
213+
214+
- Ensuring compliance with your organization's policies
215+
- Filtering packages by license type
216+
- Documentation purposes
217+
218+
```toml title="pixi.toml"
219+
[dependencies.pytorch]
220+
version = "2.0.*"
221+
license = "BSD-3-Clause"
222+
```
223+
224+
### File Name
225+
226+
Specify the exact package file name to download.
227+
This is rarely needed but useful for:
228+
229+
- Debugging package resolution issues
230+
- Ensuring a specific package artifact
231+
- Advanced use cases with custom package builds
232+
233+
```toml title="pixi.toml"
234+
[dependencies.pytorch]
235+
file-name = "pytorch-2.0.0-cuda.tar.bz2"
236+
```
237+
238+
## Source Packages
239+
240+
!!! warning
241+
`pixi-build` is a preview feature, and will change until it is stabilized.
242+
Please keep that in mind when you use it for your workspaces.
243+
244+
For these packages to be recognized they need to be understood by Pixi as source packages.
245+
Look at the [Pixi Manifest Reference](../reference/pixi_manifest.md#the-package-section) for more information on how to declare source packages in your `pixi.toml`.
246+
247+
### Path-based Source Packages
248+
```toml title="pixi.toml"
249+
--8<-- "docs/source_files/pixi_tomls/package_specifications.toml:path-fields"
250+
```
251+
252+
The path should be relative to the workspace root or an absolute path, but absolute paths are discouraged for portability.
253+
254+
255+
### Git-based Source Packages
256+
```toml title="pixi.toml"
257+
--8<-- "docs/source_files/pixi_tomls/package_specifications.toml:git-fields"
258+
```
259+
260+
For git repositories, you can specify:
261+
262+
- **git**: Repository URL
263+
- **branch**: Git branch name
264+
- **tag**: Git tag
265+
- **rev**: Specific git revision/commit SHA
266+
- **subdirectory**: Path within the repository
267+
268+
## PyPI package specifications
269+
270+
Pixi also supports installing package dependencies from PyPI using `pixi add --pypi` and in your `pixi.toml` and `pyproject.toml`.
271+
Pixi implements the standard [PEP440 version specifiers](https://peps.python.org/pep-0440/) for specifying package versions.
272+
273+
### Command Line Syntax
274+
275+
When using `pixi add --pypi`, you can [specify packages similarly to `pip`](https://pip.pypa.io/en/stable/user_guide/#installing-packages):
276+
277+
```shell
278+
# Simple package
279+
pixi add --pypi requests
280+
281+
# Specific version
282+
pixi add --pypi "requests==2.25.1"
283+
284+
# Version range
285+
pixi add --pypi "requests>=2.20,<3.0"
286+
287+
# Extras
288+
pixi add --pypi "requests[security]==2.25.1"
289+
290+
# URL
291+
pixi add --pypi "requests @ https://files.pythonhosted.org/packages/1e/db/4254e3eabe8020b458f1a747140d32277ec7a271daf1d235b70dc0b4e6e3/requests-2.32.5-py3-none-any.whl#sha256=2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6"
292+
293+
# Git repository
294+
pixi add --pypi "requests @ git+https://github.com/psf/[email protected]"
295+
pixi add --pypi requests --git https://github.com/psf/requests.git --tag v2.25.1
296+
pixi add --pypi requests --git https://github.com/psf/requests.git --branch main
297+
pixi add --pypi requests --git https://github.com/psf/requests.git --rev 70298332899f25826e35e42f8d83425124f755a
298+
```
299+
300+
### TOML Mapping Syntax
301+
In your `pixi.toml` or `pyproject.toml` (under `[tool.pixi.pypi-dependencies]`), you can specify PyPI packages like this:
302+
303+
```toml title="pixi.toml"
304+
--8<-- "docs/source_files/pixi_tomls/package_specifications.toml:pypi-fields"
305+
```
306+
307+
## Further Reading
308+
309+
- [Pixi Manifest Reference](../reference/pixi_manifest.md#dependencies) - Complete dependency specification options
310+
- [Multi-Platform Configuration](../workspace/multi_platform_configuration.md) - Platform-specific dependencies
311+
- [Conda Package Specification](https://conda.io/projects/conda/en/latest/user-guide/concepts/pkg-specs.html) - Conda's package specification docs

‎docs/reference/cli/pixi/add_extender‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,21 @@ pixi add --git https://github.com/mahmoud/boltons.git boltons --tag v0.1.0 --pyp
6161
25. This will add the `boltons` package with the given `git` url and `v0.1.0` tag as `pypi` dependency.
6262
26. This will add the `boltons` package with the given `git` url, `v0.1.0` tag and the `boltons` folder in the repository as `pypi` dependency.
6363

64+
!!! tip "Need to specify build strings or hardware-specific packages?"
65+
For advanced package specifications including build strings, see the [Package Specifications](../../../concepts/package_specifications.md) guide.
66+
67+
Examples (both syntaxes work):
68+
```shell
69+
# Equals syntax (compact)
70+
pixi add "pytorch=*=*cuda*"
71+
pixi add "numpy=*=py311*"
72+
73+
# Bracket syntax (explicit)
74+
pixi add "pytorch [build='*cuda*']"
75+
pixi add "numpy [build='py311*']"
76+
```
77+
78+
6479
!!! tip
6580
If you want to use a non default pinning strategy, you can set it using [pixi's configuration](../../pixi_configuration.md#pinning-strategy).
6681
```

‎docs/reference/cli/pixi/global/install_extender‎

Lines changed: 20 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,6 @@ pixi global install ruff
88
pixi global install starship rattler-build
99
# Specify the channel(s)
1010
pixi global install --channel conda-forge --channel bioconda trackplot
11-
# Or in a more concise form
12-
pixi global install -c conda-forge -c bioconda trackplot
1311

1412
# Support full conda matchspec
1513
pixi global install python=3.9.*
@@ -33,14 +31,29 @@ pixi global install --expose "py39=python3.9" "python=3.9.*"
3331

3432
--8<-- [start:description]
3533

34+
!!! tip "Need to specify build strings or hardware-specific packages?"
35+
For advanced package specifications including build strings, see the [Package Specifications](../../../../concepts/package_specifications.md) guide.
36+
37+
Examples (both syntaxes work):
38+
```shell
39+
# Equals syntax (compact)
40+
pixi global install "pytorch=*=*cuda*" --channel pytorch
41+
pixi global install "jax=*=*cuda*"
42+
43+
# Bracket syntax (explicit)
44+
pixi global install "pytorch [build='*cuda*']" --channel pytorch
45+
pixi global install "jax [build='*cuda*']"
46+
```
47+
3648
!!! tip
37-
Running `osx-64` on Apple Silicon will install the Intel binary but run it using [Rosetta](https://developer.apple.com/documentation/apple-silicon/about-the-rosetta-translation-environment)
38-
```
39-
pixi global install --platform osx-64 ruff
40-
```
49+
Running `osx-64` on Apple Silicon will install the Intel binary but run it using [Rosetta](https://developer.apple.com/documentation/apple-silicon/about-the-rosetta-translation-environment)
50+
```
51+
pixi global install --platform osx-64 ruff
52+
```
4153

4254
!!! note
43-
When you pass `--path` with a local `.conda` archive, Pixi copies the file into `PIXI_HOME/conda-files` and installs from that managed copy. Supplying any other kind of path keeps the original location unchanged.
55+
When you pass `--path` with a local `.conda` archive, Pixi copies the file into `PIXI_HOME/conda-files` and installs from that managed copy.
56+
Supplying any other kind of path keeps the original location unchanged.
4457

4558
After using global install, you can use the package you installed anywhere on your system.
4659
--8<-- [end:description]

0 commit comments

Comments
 (0)