Set environment variables before installing dependencies

[0x1997]

Problem description

I have a python & rust mixed project. My pyproject.toml is basically like this

[build-system]
requires = ["maturin>=1.8.6,<2.0"]
build-backend = "maturin"

[project]
name = "mypkg"
version = "1.0.0"
description = "mypkg"
readme = "README.md"
requires-python = ">=3.13"
dependencies = []

[project.scripts]
mypkg = "mypkg.main:run_module"

[tool.pixi.project]
channels = ["conda-forge"]
platforms = ["win-64"]

[tool.pixi.dependencies]
python = "3.13.*"

[tool.pixi.pypi-dependencies]
mypkg = { path = ".", editable = true }      # so I can use it like `pixi run mypkg --args`

# [tool.pixi.target.win.activation.env]
# ASSET_DIR = "web\\dist"
# PROTOC = "%CONDA_PREFIX%\\Library\\bin\\protoc.exe"
# SQLITE3_INCLUDE_DIR = "%CONDA_PREFIX%\\Library\\include"
# SQLITE3_LIB_DIR = "%CONDA_PREFIX%\\Library\\lib"

[tool.maturin]
module-name = "mypkg._mypkg"
features = ["pyo3/extension-module"]
manifest-path = "crates/mypkg/Cargo.toml"

The commented out sections of environment variables are needed by the build script of the rust crate. But they are not available when maturin is run. Updated PATH is also not available at that stage, so I need PROTOC instead of locating protoc from PATH.
Currently I have to manage them with another tool like mise which adds another layer of complexity.

[lucascolley]

check out https://github.com/prefix-dev/pixi-build-testsuite/tree/main/tests/data/pixi_build/maturin !

[0x1997]

@lucascolley Thanks, I tried to update my project accordingly but failed to pin python to 3.13.

This is a sample repo.

$ git clone https://github.com/0x1997/pixi-maturin.git
$ pixi i
Error:   × failed to solve requirements of environment 'default' for platform 'win-64'
  ├─▶ failed to solve the environment
  ╰─▶ Cannot solve the request because of: python 3.13.* cannot be installed because there are no viable options:
      └─ python 3.13.0 | 3.13.0 | 3.13.0 | 3.13.0 | 3.13.1 | 3.13.1 | 3.13.1 | 3.13.1 | 3.13.1 | 3.13.1 | 3.13.1 |
      3.13.1 | 3.13.1 | 3.13.1 | 3.13.2 | 3.13.2 | 3.13.2 | 3.13.2 | 3.13.3 | 3.13.3 | 3.13.3 | 3.13.3 | 3.13.5 |
      3.13.5 | 3.13.5 | 3.13.5 | 3.13.5 | 3.13.5 would require
         └─ python_abi 3.13.* *_cp313t, for which no candidates were found.

[lucascolley]

try adding python = "3.13.*" to package.host-dependencies?

I think this is something we need to fix though, having a package available for multiple Python versions and automatically choosing one at the workspace level with a Python dependency. I will look into it.

[lucascolley]

if you have python = "*" in package.host-dependencies, I think this might work already

[0x1997]

python = "3.13." fixed it. It kept resolving to python 3.12 if python="" was specified everywhere.

[lucascolley]

just going to move this to a discussion as there isn't a clear bug/feature request yet

[0x1997]

It works. But I got this warning with pixi run start.
WARN failed to determine modification time of input files: failed to parse glob expression. Assuming the package is out-of-date.

With verbose output enabled:

$ pixi run -v start
 INFO pixi::lock_file::update: the lock-file is up-to-date
✨ Pixi task (start): python -m fast_math                                                                                      INFO pixi::task::executable_task: Checking if task can be skipped
 WARN query{package=win-64/fast_math=0.1.0=hbf21a9e_0 source=.}: pixi_command_dispatcher::source_build_cache_status: failed to determine modification time of input files: failed to parse glob expression. Assuming the package is out-of-date.

 ╭─ Running build for recipe: fast_math-0.1.0-hbf21a9e_0
 │
 │ ╭─ Fetching source code
 │ │ No sources to fetch
 │ │
 │ ╰─────────────────── (took 0 seconds)
 │
 │ ╭─ Running build script
 │ │ %SRC_DIR%>IF "" == "" (
 │ │
 │ │  call "%SRC_DIR%\build_env.bat"
 │ │ )
 │ │ %SRC_DIR%>%PREFIX%\python.exe -m pip install --ignore-installed -vv --no-deps --no-build-isolation --editable D:\q\pixi-maturin
 │ │ Using pip 25.2 from %PREFIX%\Lib\site-packages\pip (python 3.13)
 │ │ Non-user install because user site-packages disabled
 │ │ Ignoring indexes: https://pypi.org/simple
 │ │ Created temporary directory: C:\Users\zhe\AppData\Local\Temp\pip-build-tracker-d67uhfz5
 │ │ Initialized build tracking at C:\Users\zhe\AppData\Local\Temp\pip-build-tracker-d67uhfz5
 │ │ Created build tracker: C:\Users\zhe\AppData\Local\Temp\pip-build-tracker-d67uhfz5
 │ │ Entered build tracker: C:\Users\zhe\AppData\Local\Temp\pip-build-tracker-d67uhfz5
 │ │ Created temporary directory: C:\Users\zhe\AppData\Local\Temp\pip-install-j8q_vuiy
 │ │ Created temporary directory: C:\Users\zhe\AppData\Local\Temp\pip-ephem-wheel-cache-lvw4s3bw
 │ │ Obtaining file:///D:/q/pixi-maturin
 │ │   Added file:///D:/q/pixi-maturin to build tracker 'C:\\Users\\zhe\\AppData\\Local\\Temp\\pip-build-tracker-d67uhfz5'
 │ │   Running command Checking if build backend supports build_editable
 │ │   Checking if build backend supports build_editable: started
 │ │   Checking if build backend supports build_editable: finished with status 'done'
 │ │   Created temporary directory: C:\Users\zhe\AppData\Local\Temp\pip-modern-metadata-nxc0exnn
 │ │   Preparing editable metadata (pyproject.toml): started
 │ │   Running command Preparing editable metadata (pyproject.toml)
 │ │   🍹 Building a mixed python/rust project
 │ │   🔗 Found pyo3 bindings
 │ │   🐍 Found CPython 3.13 at %PREFIX%\python.exe
 │ │   📡 Using build options features from pyproject.toml
 │ │   fast_math-0.1.0.dist-info
 │ │   Checking for Rust toolchain....
 │ │   Running `maturin pep517 write-dist-info --metadata-directory C:\Users\zhe\AppData\Local\Temp\pip-modern-metadata-nxc0exnn --interpreter %PREFIX%\python.e
 │ │ xe`
 │ │   Preparing editable metadata (pyproject.toml): finished with status 'done'
 │ │   Source in d:\q\pixi-maturin has version 0.1.0, which satisfies requirement fast_math==0.1.0 from file:///D:/q/pixi-maturin
 │ │   Removed fast_math==0.1.0 from file:///D:/q/pixi-maturin from build tracker 'C:\\Users\\zhe\\AppData\\Local\\Temp\\pip-build-tracker-d67uhfz5'
 │ │ Created temporary directory: C:\Users\zhe\AppData\Local\Temp\pip-unpack-s4o4fu15
 │ │ Building wheels for collected packages: fast_math
 │ │   Created temporary directory: C:\Users\zhe\AppData\Local\Temp\pip-wheel-pibw4nb2
 │ │   Destination directory: C:\Users\zhe\AppData\Local\Temp\pip-wheel-pibw4nb2
 │ │   Running command Building editable for fast_math (pyproject.toml)
 │ │   Building editable for fast_math (pyproject.toml): started
 │ │   Running `maturin pep517 build-wheel -i %PREFIX%\python.exe --compatibility off --editable`
 │ │   🍹 Building a mixed python/rust project
 │ │   🔗 Found pyo3 bindings
 │ │   🐍 Found CPython 3.13 at %PREFIX%\python.exe
 │ │   📡 Using build options features from pyproject.toml
 │ │   warning: unused variable: `a`
 │ │    --> src\lib.rs:6:18
 │ │     |
 │ │   6 | fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
 │ │     |                  ^ help: if this is intentional, prefix it with an underscore: `_a`
 │ │     |
 │ │     = note: `#[warn(unused_variables)]` on by default
 │ │   warning: unused variable: `b`
 │ │    --> src\lib.rs:6:28
 │ │     |
 │ │   6 | fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
 │ │     |                            ^ help: if this is intentional, prefix it with an underscore: `_b`
 │ │   warning: `fast_math` (lib) generated 2 warnings
 │ │       Finished `release` profile [optimized] target(s) in 0.06s
 │ │   📦 Built wheel for CPython 3.13 to D:\q\pixi-maturin\target\wheels\fast_math-0.1.0-cp313-cp313-win_amd64.whl
 │ │   D:\q\pixi-maturin\target\wheels\fast_math-0.1.0-cp313-cp313-win_amd64.whl
 │ │   Building editable for fast_math (pyproject.toml): finished with status 'done'
 │ │   Created wheel for fast_math: filename=fast_math-0.1.0-cp313-cp313-win_amd64.whl size=1001 sha256=43c7d69b6adc73a94d2c7f7ec84298944bf8ef83d2dd366f3a0d9bb9
 │ │ 29ce6678
 │ │   Stored in directory: C:\Users\zhe\AppData\Local\Temp\pip-ephem-wheel-cache-lvw4s3bw\wheels\ea\45\a7\35fe069aeb0670ba116064c18d6a2d7005c54c8153978324a3
 │ │ Successfully built fast_math
 │ │ Installing collected packages: fast_math
 │ │ Successfully installed fast_math-0.1.0
 │ │ Removed build tracker: 'C:\\Users\\zhe\\AppData\\Local\\Temp\\pip-build-tracker-d67uhfz5'
 │ │ %SRC_DIR%>if errorlevel 1 exit 1
 │ │
 │ ╰─────────────────── (took 3 seconds)
 │
 │ ╭─ Packaging new files
 │ │ Copying done!
 │ │ Post-processing done!
 │ │ Writing test files
 │ │ Writing metadata for package
 │ │ Copying license files
 │ │ Copying recipe files
 │ │ Creating entry points
 │ │
 │ │ Files in package:
 │ │   - Lib/site-packages/fast_math-0.1.0.dist-info/INSTALLER
 │ │   - Lib/site-packages/fast_math-0.1.0.dist-info/METADATA
 │ │   - Lib/site-packages/fast_math-0.1.0.dist-info/RECORD
 │ │   - Lib/site-packages/fast_math-0.1.0.dist-info/REQUESTED
 │ │   - Lib/site-packages/fast_math-0.1.0.dist-info/WHEEL
 │ │   - Lib/site-packages/fast_math-0.1.0.dist-info/direct_url.json
 │ │   - Lib/site-packages/fast_math.pth
 │ │   - info/about.json
 │ │   - info/hash_input.json
 │ │   - info/index.json
 │ │   - info/paths.json
 │ │   - info/tests/tests.yaml
 │ │ Creating target folder 'D:\q\pixi-maturin\.pixi\build\work\fast_math-fYZaA0CCDtQ\output\win-64'
 │ │ Compressing archive...
 │ │ Archive written to 'D:\q\pixi-maturin\.pixi\build\work\fast_math-fYZaA0CCDtQ\output\win-64\fast_math-0.1.0-hbf21a9e_0.conda'
 │ │
 │ ╰─────────────────── (took 0 seconds)
 │ Checking for symlinks ...
 │
 ╰─────────────────── (took 3 seconds)
 WARN rattler_cache::package_cache: hash mismatch, wanted a package with hash 462e07874be771e092dbb105697d1381ba47721e2396ad034e79248d78aa5cd0 but the cached package has hash 1bc6b68d3f61349230d6b0d0bf9897a0229757267109dc3bb142ebe456924cd2, fetching package
 INFO pixi::environment::conda_metadata: No update needed for the prefix file.
 INFO pixi::lock_file::update: Installed environment 'default' in 15.6071832s
3 + 5 = main

[lucascolley]

cc @Hofer-Julian maybe you have seen this warning before?

[0x1997]

Hi @lucascolley, problem solved, thanks for your help.
I did some rearangement to my pixi.toml, pyproject.toml and .cargo/config.toml, now everything works as expected.

[HernandoR]

similar issue, I need $CUDA_HOME set during (at least) the install of certain package. How can i achive this?