Possible to force Python package install over conda package?

[matthewfeickert]

By default Pixi will work to prefer conda packages in an environment over Python (index) packages. However, there might be a time in an environment in your workspace that you want to get dependency as a Python package to test it at its current development state.

Example:

[workspace]
authors = ["Matthew Feickert"]
channels = ["conda-forge"]
name = "reproducer"
platforms = ["linux-64", "osx-arm64"]
version = "0.1.0"

[tasks]

[dependencies]
atlas-schema = ">=0.2.4,<0.3"

[feature.af.dependencies]
python = "==3.9.18"

[feature.af.pypi-dependencies]
awkward = { git = "git+https://github.com/scikit-hep/awkward.git", rev = "5e3849f385fcd1eab46358ab0ed5f0883bcd113b" }

[environments]
af = {features = ["af"], solve-group = "af"}

rm -rf pixi.lock && pixi list -vv --environment af awkward:

$ rm -rf pixi.lock && pixi list -vv --environment af awkward
 INFO pixi_manifest::discovery: Found manifest in directory: "/tmp/reproducer", continuing further.
 INFO pixi::lock_file::outdated: environment 'default' is out of date because it does not exist in the lock-file.
 INFO pixi::lock_file::outdated: environment 'af' is out of date because it does not exist in the lock-file.
 INFO rattler_networking::s3_middleware: Creating S3 middleware using {}
 INFO pixi::lock_file::resolve::uv_resolution_context: uv keyring provider is disabled
 INFO resolve_conda{group=default platform=osx-arm64}:get_or_create_subdir{platform=OsxArm64 channel=https://conda.anaconda.org/conda-forge/}: rattler_repodata_gateway::gateway::subdir_builder: sharded repodata seems to be missing for https://conda.anaconda.org/conda-forge/osx-arm64/, falling back to repodata.json files
 INFO resolve_conda{group=default platform=linux-64}:get_or_create_subdir{platform=NoArch channel=https://conda.anaconda.org/conda-forge/}: rattler_repodata_gateway::gateway::subdir_builder: sharded repodata seems to be missing for https://conda.anaconda.org/conda-forge/noarch/, falling back to repodata.json files
 INFO resolve_conda{group=default platform=linux-64}:get_or_create_subdir{platform=Linux64 channel=https://conda.anaconda.org/conda-forge/}: rattler_repodata_gateway::gateway::subdir_builder: sharded repodata seems to be missing for https://conda.anaconda.org/conda-forge/linux-64/, falling back to repodata.json files
 INFO resolve_conda{group=af platform=osx-arm64}: pixi::lock_file::update: fetched 27029 records in 525.46771ms
 INFO resolve_conda{group=default platform=osx-arm64}: pixi::lock_file::update: fetched 27217 records in 531.456523ms
 INFO pixi::lock_file::update: resolved conda environment for environment 'default' 'osx-arm64' in 936ms 891us 824ns
 INFO resolve_conda{group=default platform=linux-64}: pixi::lock_file::update: fetched 67721 records in 1.006514875s
 INFO resolve_conda{group=af platform=linux-64}: pixi::lock_file::update: fetched 67313 records in 1.023724922s
 INFO pixi::lock_file::update: resolved conda environment for environment 'default' 'linux-64' in 2s 985ms 192us 606ns
 INFO resolve_conda{group=af platform=osx-arm64}: pypi_mapping: Amended 216 out of 216 records with purls in 18.729s. 184 cache hits and 33 cache misses (84.79%).
 INFO pixi::lock_file::update: resolved conda environment for environment 'af' 'osx-arm64' in 19s 655ms 846us 748ns
 INFO resolve_conda{group=af platform=linux-64}: pypi_mapping: Amended 227 out of 227 records with purls in 16.845s. 132 cache hits and 19 cache misses (87.42%).
 INFO pixi::lock_file::update: resolved conda environment for environment 'af' 'linux-64' in 19s 861ms 541us 89ns
 INFO resolve_pypi{group=af platform=osx-arm64}: pixi::lock_file::resolve::pypi: the following python packages are assumed to be installed by conda: mplhep-data 0.0.4, charset-normalizer 3.4.1, idna 3.10, correctionlib 2.6.4, colorama 0.4.6, requests 2.32.3, aiohappyeyeballs 2.6.1, aiosignal 1.3.2, multidict 6.3.2, brotli 1.1.0, async-timeout 5.0.1, wrapt 1.17.2, tornado 6.4.2, unicodedata2 16.0.0, cffi 1.17.1, particle 0.25.3, awkward-cpp 45, hist 2.8.0, tblib 3.1.0, cachetools 5.5.2, boost-histogram 1.5.2, toolz 1.0.0, certifi 2025.1.31, mdurl 0.1.2, contourpy 1.3.0, pillow 11.1.0, psutil 7.0.0, msgpack 1.1.0, pygments 2.19.1, yarl 1.18.3, fastjsonschema 2.21.1, uproot 5.6.0, packaging 24.2, dask-histogram 2025.2.0, sortedcontainers 2.4.0, annotated-types 0.7.0, hepunits 2.3.5, hyperframe 6.1.0, pysocks 1.7.1, uhi 0.5.0, importlib-metadata 8.6.1, tzdata 2025.2, markupsafe 3.0.2, iminuit 2.31.1, urllib3 2.3.0, zipp 3.21.0, markdown-it-py 3.0.0, click 8.1.8, pandas 2.2.3, distributed 2024.8.0, pyarrow 19.0.1, pyyaml 6.0.2, zict 3.0.0, pytz 2024.1, fsspec-xrootd 0.5.1, bokeh 3.4.2, rich 14.0.0, six 1.17.0, atlas-schema 0.2.4, vector 1.6.1, cycler 0.12.1, munkres 1.1.4, typing-inspection 0.4.0, pydantic-core 2.33.1, h2 4.2.0, hpack 4.1.0, deprecated 1.2.18, aiohttp 3.11.16, zstandard 0.23.0, pyarrow-hotfix 0.6, coffea 2025.3.0, kiwisolver 1.4.7, lz4 4.3.3, histoprint 2.6.0, python-dateutil 2.9.0.post0, importlib-resources 6.5.2, xrootd 5.8.0, xyzservices 2025.1.0, pydantic 2.11.2, tqdm 4.67.1, partd 1.4.2, llvmlite 0.43.0, cloudpickle 3.1.1, dask-expr 1.1.10, dask-awkward 2025.3.0, cramjam 2.8.4rc3, mplhep 0.3.59, frozenlist 1.5.0, propcache 0.2.1, numba 0.60.0, jinja2 3.1.6, pyparsing 3.2.3, matplotlib 3.9.4, locket 1.0.0, awkward 2.8.1, dask 2024.8.0, typing-extensions 4.13.1, cytoolz 1.0.1, attrs 25.3.0, fsspec 2025.3.2, scipy 1.13.1, toml 0.10.2, xxhash 3.5.0, fonttools 4.57.0, numpy 2.0.2
 INFO resolve_pypi{group=af platform=osx-arm64}: pixi::lock_file::resolve::pypi: using requires python specifier (this may differ from the above): ==3.9.*
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(0) @ 0a0.dev0 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(1) @ 2.8.1 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(2) @ 45 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(3) @ 8.6.1 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(8) @ 8.6.1 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(9) @ 8.6.1 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(4) @ 2.0.2 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(5) @ 24.2 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(6) @ 4.13.1 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(10) @ 4.13.1 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(11) @ 4.13.1 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(7) @ 2025.3.2 without checking dependencies    
 INFO pixi::lock_file::update: resolved pypi packages for environment 'af' 'osx-arm64' in 342ms 933us 356ns
 INFO resolve_pypi{group=af platform=linux-64}: pixi::lock_file::resolve::pypi: the following python packages are assumed to be installed by conda: mplhep-data 0.0.4, aiohappyeyeballs 2.6.1, fonttools 4.57.0, async-timeout 5.0.1, hpack 4.1.0, pysocks 1.7.1, scipy 1.13.1, dask-awkward 2025.3.0, toml 0.10.2, click 8.1.8, locket 1.0.0, pillow 11.1.0, tornado 6.4.2, cytoolz 1.0.1, propcache 0.2.1, tblib 3.1.0, atlas-schema 0.2.4, matplotlib 3.9.4, lz4 4.3.3, tqdm 4.67.1, hist 2.8.0, aiohttp 3.11.16, typing-extensions 4.13.1, annotated-types 0.7.0, psutil 7.0.0, llvmlite 0.43.0, urllib3 2.3.0, toolz 1.0.0, mplhep 0.3.59, tzdata 2025.2, zstandard 0.23.0, pyarrow-hotfix 0.6, histoprint 2.6.0, pytz 2024.1, distributed 2024.8.0, awkward 2.8.1, attrs 25.3.0, vector 1.6.1, deprecated 1.2.18, importlib-resources 6.5.2, python-dateutil 2.9.0.post0, coffea 2025.3.0, cloudpickle 3.1.1, packaging 24.2, correctionlib 2.6.4, requests 2.32.3, zict 3.0.0, brotli 1.1.0, xyzservices 2025.1.0, cachetools 5.5.2, msgpack 1.1.0, dask-expr 1.1.10, sortedcontainers 2.4.0, hepunits 2.3.5, frozenlist 1.5.0, wrapt 1.17.2, partd 1.4.2, pyarrow 19.0.1, xxhash 3.5.0, yarl 1.18.3, zipp 3.21.0, uproot 5.6.0, munkres 1.1.4, awkward-cpp 45, contourpy 1.3.0, cramjam 2.8.4rc3, typing-inspection 0.4.0, xrootd 5.8.0, uhi 0.5.0, aiosignal 1.3.2, multidict 6.3.2, cffi 1.17.1, markdown-it-py 3.0.0, mdurl 0.1.2, pygments 2.19.1, dask-histogram 2025.2.0, importlib-metadata 8.6.1, h2 4.2.0, pandas 2.2.3, pyparsing 3.2.3, bokeh 3.4.2, iminuit 2.31.1, idna 3.10, unicodedata2 16.0.0, fastjsonschema 2.21.1, pydantic-core 2.33.1, cycler 0.12.1, boost-histogram 1.5.2, jinja2 3.1.6, markupsafe 3.0.2, six 1.17.0, colorama 0.4.6, rich 14.0.0, kiwisolver 1.4.7, particle 0.25.3, pyyaml 6.0.2, fsspec-xrootd 0.5.1, pydantic 2.11.2, charset-normalizer 3.4.1, dask 2024.8.0, hyperframe 6.1.0, numpy 2.0.2, numba 0.60.0, fsspec 2025.3.2, certifi 2025.1.31
 INFO resolve_pypi{group=af platform=linux-64}: pixi::lock_file::resolve::pypi: using requires python specifier (this may differ from the above): ==3.9.*
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(0) @ 0a0.dev0 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(1) @ 2.8.1 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(2) @ 45 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(3) @ 8.6.1 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(8) @ 8.6.1 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(9) @ 8.6.1 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(4) @ 2.0.2 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(5) @ 24.2 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(6) @ 4.13.1 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(10) @ 4.13.1 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(11) @ 4.13.1 without checking dependencies    
 INFO solve: pubgrub::internal::partial_solution: add_decision: Id::<PubGrubPackage>(7) @ 2025.3.2 without checking dependencies    
 INFO pixi::lock_file::update: resolved pypi packages for environment 'af' 'linux-64' in 95ms 627us 757ns
 INFO pixi::lock_file::resolve::uv_resolution_context: uv keyring provider is disabled
Environment: af
Package       Version   Build             Size       Kind   Source
awkward       2.8.1     pyhd8ed1ab_0      419.5 KiB  conda  https://conda.anaconda.org/conda-forge/
awkward-cpp   45        py39hd3a5d65_100  512.1 KiB  conda  https://conda.anaconda.org/conda-forge/
dask-awkward  2025.3.0  pyh29332c3_0      78.2 KiB   conda  https://conda.anaconda.org/conda-forge/

results in awkward being installed as a conda package from conda-forge even though we specified that we wanted it to be installed from Git as a Python package

Environment: af
Package       Version   Build             Size       Kind   Source
awkward       2.8.1     pyhd8ed1ab_0      419.5 KiB  conda  https://conda.anaconda.org/conda-forge/
awkward-cpp   45        py39hd3a5d65_100  512.1 KiB  conda  https://conda.anaconda.org/conda-forge/
dask-awkward  2025.3.0  pyh29332c3_0      78.2 KiB   conda  https://conda.anaconda.org/conda-forge/

this is happening because of the dependency structure of atlas-schema

$ pixi tree --invert --environment af awkward

Environment: af

awkward 2.8.1 
├── uproot 5.6.0 
│   └── coffea 2025.3.0 
│       └── atlas-schema 0.2.4 
├── coffea 2025.3.0 (*)
└── dask-awkward 2025.3.0 
    ├── coffea 2025.3.0 (*)
    └── dask-histogram 2025.2.0 
        └── coffea 2025.3.0 (*)

dask-awkward 2025.3.0 (*)

awkward-cpp 45 
└── awkward 2.8.1 (*)

Is there a way to indicate that the Python package version of a package should take precedence and overwrite a conda package install?

cc @kratsg

[matthewfeickert]

Currently the only way I know how to get around this is by making all tools that require the Git based Python project also be Python projects (i.e. atlas-schema is in the [feature.af.pypi-dependencies] table) and then setting the target environment to have no-default-feature = true.

[workspace]
authors = ["Matthew Feickert>"]
channels = ["conda-forge"]
name = "reproducer"
platforms = ["linux-64", "osx-arm64"]
version = "0.1.0"

[tasks]

[dependencies]
atlas-schema = ">=0.2.4,<0.3"

[feature.af.dependencies]
python = "==3.9.18"

[feature.af.pypi-dependencies]
atlas-schema = ">=0.2.4,<0.3"
awkward = { git = "git+https://github.com/scikit-hep/awkward.git", rev = "5e3849f385fcd1eab46358ab0ed5f0883bcd113b" }

[environments]
af = {features = ["af"], solve-group = "af", no-default-feature = true}

$ rm -rf pixi.lock && pixi list -e af awkward
Environment: af
Package       Version   Build  Size  Kind  Source
awkward       2.8.1                  pypi  git+https://github.com/scikit-hep/awkward.git?rev=5e3849f385fcd1eab46358ab0ed5f0883bcd113b#5e3849f385fcd1eab46358ab0ed5f0883bcd113b
awkward_cpp   45                     pypi  
dask_awkward  2025.3.0               pypi  

The "problem" here is that this then can potentially require huge amount of software to get installed as Python packages when it would be nice to have everything except the one piece of software you want installed from Git to be installed as conda packages.