pip works - pixi fails to resolve for magic-pdf #3217

geoHeil · 2025-02-26T15:06:05Z

geoHeil
Feb 26, 2025

Using pip like:

[project]
name = "foo"
version = "1.0.0"
description = "bar"
authors = [{ name = "xx", email = "yy" }]
requires-python = "~= 3.13"

[tool.pixi.project]
platforms = ["linux-64", "win-64", "linux-aarch64", "osx-arm64"]
channels = ["conda-forge"]

[build-system]
build-backend = "hatchling.build"
requires = ["hatchling"]

[tool.pixi.environments]
ocr-mineru = { features = [
  "ocr-mineru",
  
], solve-group = "ocr-mineru", no-default-feature = true }

[tool.pixi.feature.ocr-mineru.dependencies]
python = "~=3.10.0"
pip = "*"

Then executing:

pixi update
pixi shell -e ocr-mineru
pip install -U "magic-pdf[full]" --extra-index-url https://wheels.myhloli.com

works fine.

However, a pure pixi appraoch like:

[project]
name = "foo"
version = "1.0.0"
description = "bar"
authors = [{ name = "xx", email = "yy" }]
requires-python = "~= 3.13"

[tool.pixi.project]
platforms = ["linux-64", "win-64", "linux-aarch64", "osx-arm64"]
channels = ["conda-forge"]

[build-system]
build-backend = "hatchling.build"
requires = ["hatchling"]

[tool.pixi.environments]
ocr-mineru = { features = [
  "ocr-mineru",
  
], solve-group = "ocr-mineru", no-default-feature = true }

[tool.pixi.feature.ocr-mineru.dependencies]
python = "~=3.10.0"
pip = "*"

[tool.pixi.feature.ocr-mineru.pypi-dependencies]
magic-pdf = { version = ">1,<2", extras = ["full"] }

given a config ./.pixi/config.toml file like the one below which specifies the extra index:

[pypi-config]
extra-index-urls = ["https://wheels.myhloli.com"]

fails with:

pixi update
Error:
  × failed to solve the pypi requirements of 'ocr-mineru' 'osx-arm64'
  ├─▶ failed to resolve pypi dependencies
  ╰─▶ Because there is no version of doclayout-yolo==0.0.2 and magic-pdf[full]==1.0.1 depends on doclayout-yolo==0.0.2, we can conclude that magic-pdf[full]==1.0.1 cannot be used.
      And because only the following versions of magic-pdf[full] are available:
          magic-pdf[full]<1
          magic-pdf[full]==1.0.1
          magic-pdf[full]==1.1.0
      we can conclude that magic-pdf[full]>1,<1.1.0 cannot be used. (1)

      Because there is no version of doclayout-yolo==0.0.2b1 and magic-pdf[full]==1.1.0 depends on doclayout-yolo==0.0.2b1, we can conclude that magic-pdf[full]==1.1.0 cannot be
      used.
      And because we know from (1) that magic-pdf[full]>1,<1.1.0 cannot be used, we can conclude that magic-pdf[full]>1 cannot be used.
      And because you require magic-pdf[full]>1, we can conclude that your requirements are unsatisfiable.

      hint: `doclayout-yolo` was requested with a pre-release marker (e.g., doclayout-yolo==0.0.2b1), but pre-releases weren't enabled (try: `--prerelease=allow`)

how can this be fixed?

Answered by nichmor

Feb 27, 2025

it is related to selecting pre-release versions during the solve. by default UV allows pre-release versions if all versions of a package are pre-release, or if the package has an explicit pre-release marker in its version requirements.

so in this case we are using an explicit version selector to trigger this behaviour.

it is configurable in UV https://docs.astral.sh/uv/reference/settings/#prerelease

but we don't allow this configuration yet: #1291 (comment)

also cross-related with this: #1357

View full answer

nichmor · 2025-02-26T15:27:30Z

nichmor
Feb 26, 2025
Maintainer

hey @geoHeil !

what is the result if you will use this version specifier magic-pdf = { version = "*", extras = ["full"] }?

could you also please do a pip freeze in first approach?

thanks!

11 replies

geoHeil Feb 26, 2025
Author

Looking at https://pypi.org/project/doclayout-yolo/#history it looks like 0.0.2 is not even available on pip - and pixi refuses - correctly.

Am I understanding this correctly?

Why does pip still resolve ==0.0.2 is not 0.0.2b?

nichmor Feb 27, 2025
Maintainer

could you add doclayout_yolo = { version = ">=0.0.2b1" } in your pypi-dependencies?

I've tried this pixi.toml and I was able to solve it

[project]
name = "foo"
version = "1.0.0"
description = "bar"
# authors = [{ name = "xx", email = "yy" }]
requires-python = "~= 3.13"

[tool.pixi.project]
platforms = ["osx-arm64"]
channels = ["conda-forge"]

[build-system]
build-backend = "hatchling.build"
requires = ["hatchling"]

[tool.pixi.environments]
ocr-mineru = { features = [
  "ocr-mineru",
  
], solve-group = "ocr-mineru", no-default-feature = true }

[tool.pixi.feature.ocr-mineru.dependencies]
python = "~=3.10.0"
pip = "*"

[tool.pixi.feature.ocr-mineru.pypi-dependencies]
magic-pdf = { version = ">1.0.0 ,<2", extras = ["full"] }
doclayout_yolo = { version = ">=0.0.2b1" }

[tool.pixi.feature.ocr-mineru.pypi-options]
extra-index-urls = ["https://wheels.myhloli.com"]

geoHeil Feb 27, 2025
Author

Why does:

doclayout_yolo = { version = ">=0.0.2,<0.1" }

fail but

doclayout_yolo = { version = ">=0.0.2b1,<0.1" }

work?

nichmor Feb 27, 2025
Maintainer

it is related to selecting pre-release versions during the solve. by default UV allows pre-release versions if all versions of a package are pre-release, or if the package has an explicit pre-release marker in its version requirements.

so in this case we are using an explicit version selector to trigger this behaviour.

it is configurable in UV https://docs.astral.sh/uv/reference/settings/#prerelease

but we don't allow this configuration yet: #1291 (comment)

also cross-related with this: #1357

Answer selected by geoHeil

geoHeil Feb 27, 2025
Author

Thanks! This is already a big step forward. May I add a related follow-up question to finalize this discussion: Enabling the GPU support.

The GPU version sits here: https://paddle-whl.bj.bcebos.com/stable/cu123/paddlepaddle-gpu/paddlepaddle_gpu-3.0.0rc1-cp310-cp310-linux_x86_64.whl

A:

pixi shell -e ocr-mineru-gpu
python -m pip install paddlepaddle-gpu==3.0.0rc1 -i https://www.paddlepaddle.org.cn/packages/stable/cu123/

installs nicely and freezes to:

❯ pip freeze
alembic==1.13.1
annotated-types==0.6.0
anyio==4.8.0
astor==0.8.1
awscli==1.29.57
boto3==1.34.41
certifi==2024.12.14
click==8.1.7
colorama==0.4.4
coloredlogs==14.0
croniter==2.0.1
decorator==5.1.1
distro==1.8.0
docstring-parser==0.15
docutils==0.16
exceptiongroup==1.2.2
fsspec==2024.2.0
github3.py==4.0.1
greenlet==3.0.3
grpcio-health-checking==1.60.1
h11==0.14.0
httpcore==1.0.7
httpx==0.28.1
humanfriendly==10.0
idna==3.10
Jinja2==3.1.3
jmespath==1.0.1
Mako==1.3.2
markdown-it-py==3.0.0
MarkupSafe==2.1.5
mdurl==0.1.2
networkx==3.4.2
numpy==2.2.1
nvidia-cublas-cu12==12.3.4.1
nvidia-cuda-cupti-cu12==12.3.101
nvidia-cuda-nvrtc-cu12==12.3.107
nvidia-cuda-runtime-cu12==12.3.101
nvidia-cudnn-cu12==9.1.1.17
nvidia-cufft-cu12==11.2.1.3
nvidia-curand-cu12==10.3.5.147
nvidia-cusolver-cu12==11.6.1.9
nvidia-cusparse-cu12==12.3.1.170
nvidia-nccl-cu12==2.19.3
nvidia-nvjitlink-cu12==12.6.85
nvidia-nvtx-cu12==12.4.127
opt-einsum==3.3.0
paddlepaddle-gpu==3.0.0rc1
pendulum==3.0.0
pex==2.1.163
pillow==11.1.0
protobuf==4.25.2
psycopg2-binary==2.9.9
pyasn1==0.5.0
pydantic_core==2.16.2
Pygments==2.16.1
PyJWT==2.8.0
python-dateutil==2.8.2
python-dotenv==1.0.1
pytz==2024.1
PyYAML==6.0.1
rich==13.6.0
rsa==4.7.2
shell_gpt==0.9.4
shellingham==1.5.4
six==1.16.0
sniffio==1.3.1
SQLAlchemy==2.0.27
structlog==24.1.0
tabulate==0.9.0
time-machine==2.13.0
tomli==2.0.1
toposort==1.10
typer==0.9.0
typing_extensions==4.8.0
tzdata==2024.1
universal_pathlib==0.2.0
uritemplate==4.1.1
urllib3==1.26.16
watchdog==4.0.0
wcwidth==0.2.13

[project]
name = "foo"
version = "1.0.0"
description = "bar"
authors = [{ name = "xx", email = "yy" }]
requires-python = "~= 3.13"

[tool.pixi.project]
platforms = [ "linux-64"]
channels = ["conda-forge"]

[build-system]
build-backend = "hatchling.build"
requires = ["hatchling"]

[tool.pixi.environments]
ocr-mineru-gpu = { features = [
  "ocr-mineru-gpu",
  
], solve-group = "ocr-mineru-gpu", no-default-feature = true }

[tool.pixi.feature.ocr-mineru-gpu.pypi-options]
no-build-isolation = ["magic-pdf"]

[tool.pixi.feature.ocr-mineru-gpu]
platforms = [ "linux-64" ]
[tool.pixi.feature.ocr-mineru-gpu.dependencies]
python = "~=3.10.0"
pip = "*"

when trying via pixi:

[project]
name = "foo"
version = "1.0.0"
description = "bar"
authors = [{ name = "xx", email = "yy" }]
requires-python = "~= 3.13"

[tool.pixi.project]
platforms = [ "linux-64"]
channels = ["conda-forge"]

[build-system]
build-backend = "hatchling.build"
requires = ["hatchling"]

[tool.pixi.environments]
ocr-mineru-gpu = { features = [
  "ocr-mineru-gpu",
  
], solve-group = "ocr-mineru-gpu", no-default-feature = true }

[tool.pixi.feature.ocr-mineru-gpu.pypi-options]
no-build-isolation = ["magic-pdf"]

[tool.pixi.feature.ocr-mineru-gpu]
platforms = [ "linux-64" ]
[tool.pixi.feature.ocr-mineru-gpu.dependencies]
python = "~=3.10.0"
pip = "*"

[tool.pixi.feature.ocr-mineru-gpu.pypi-dependencies]
#magic-pdf = { version = ">=1.2.0 ,<2", extras = ["full"] }
doclayout_yolo = { version = ">=0.0.2b1,<0.1" }
paddlepaddle-gpu = { version = ">=3.0.0rc1", index = "https://www.paddlepaddle.org.cn/packages/stable/cu123/" }

almost works.

However, when enabling:

magic-pdf = { version = ">=1.2.0 ,<2", extras = ["full"] }

the error is:

failed to solve the pypi requirements of 'ocr-mineru-gpu' 'linux-64'
  ├─▶ failed to resolve pypi dependencies
  ╰─▶ Because there is no version of paddlepaddle{sys_platform == 'linux'}==3.0.0b1 and magic-pdf[full]>=1.0.1,<=1.1.0 depends on paddlepaddle{sys_platform == 'linux'}==3.0.0b1, we can conclude that magic-pdf[full]>=1.0.1,<=1.1.0 cannot be used.
      And because only the following versions of magic-pdf[full] are available:
          magic-pdf[full]<1.0.0
          magic-pdf[full]==1.0.1
          magic-pdf[full]==1.1.0
          magic-pdf[full]==1.2.0
      we can conclude that magic-pdf[full]>1.0.0,<1.1.0 cannot be used. (1)
      
      Because there is no version of paddlepaddle{sys_platform == 'linux'}==3.0.0b1 and magic-pdf[full]==1.1.0 depends on paddlepaddle{sys_platform == 'linux'}==3.0.0b1, we can conclude that magic-pdf[full]==1.1.0 cannot be used.
      And because we know from (1) that magic-pdf[full]>1.0.0,<1.1.0 cannot be used, we can conclude that magic-pdf[full]>1.0.0,<1.2.0 cannot be used. (2)
      
      Because there is no version of paddlepaddle{sys_platform == 'darwin' or sys_platform == 'linux'}==3.0.0rc1 and magic-pdf[full]==1.2.0 depends on paddlepaddle{sys_platform == 'darwin' or sys_platform == 'linux'}==3.0.0rc1, we can conclude that magic-pdf[full]==1.2.0 cannot be
      used.
      And because we know from (2) that magic-pdf[full]>1.0.0,<1.2.0 cannot be used, we can conclude that magic-pdf[full]>1.0.0 cannot be used.
      And because you require magic-pdf[full]>1.0.0, we can conclude that your requirements are unsatisfiable.
      
      hint: `paddlepaddle` was requested with a pre-release marker (e.g., paddlepaddle==3.0.0b1), but pre-releases weren't enabled (try: `--prerelease=allow`)

geoHeil Feb 27, 2025
Author

one more observation

I want to use python 3.12

Changes:

python = "~=3.12.0"

and

detectron2 = { git = "https://github.com/facebookresearch/detectron2.git", rev = "9604f5995cc628619f0e4fd913453b4d7d61db3f" }
torch = { version = ">=2.2.0,<2.7", index = "https://download.pytorch.org/whl/cpu" }

and

[tool.pixi.feature.ocr-mineru.pypi-options]
extra-index-urls = ["https://wheels.myhloli.com"]
no-build-isolation = ["detectron2"]

[project]
name = "foo"
version = "1.0.0"
description = "bar"
authors = [{ name = "xx", email = "yy" }]
requires-python = "~= 3.13"

[tool.pixi.project]
platforms = ["osx-arm64", "linux-64", "linux-aarch64" ]
channels = ["conda-forge"]

[build-system]
build-backend = "hatchling.build"
requires = ["hatchling"]

[tool.pixi.environments]
ocr-mineru = { features = [
  "ocr-mineru",
], solve-group = "ocr-mineru", no-default-feature = true }

ocr-mineru-gpu = { features = [
  "ocr-mineru-gpu",
], solve-group = "ocr-mineru-gpu", no-default-feature = true }

[tool.pixi.feature.ocr-mineru.pypi-options]
extra-index-urls = ["https://wheels.myhloli.com"]
no-build-isolation = ["detectron2"]

[tool.pixi.feature.ocr-mineru]
# notice no windows
platforms = ["osx-arm64", "linux-64", "linux-aarch64" ]
[tool.pixi.feature.ocr-mineru.dependencies]
# TODO: get 3.12 to work.
python = "~=3.12.0"
pip = "*"

[tool.pixi.feature.ocr-mineru.pypi-dependencies]
magic-pdf = { version = ">1.0.0 ,<2", extras = ["full"] }
doclayout_yolo = { version = ">=0.0.2b1,<0.1" }
detectron2 = { git = "https://github.com/facebookresearch/detectron2.git", rev = "9604f5995cc628619f0e4fd913453b4d7d61db3f" }
# TODO set up GPU version
# https://download.pytorch.org/whl/cpu
# https://download.pytorch.org/whl/cu126
#torch = { version = ">=2.6.0,<2.7", index = "https://download.pytorch.org/whl/cpu" }
#torchvision = { version = ">=0.21.0,<0.22", index = "https://download.pytorch.org/whl/cpu" }
torch = { version = ">=2.2.0,<2.7", index = "https://download.pytorch.org/whl/cpu" }
paddlepaddle = { version = ">=3.0.0rc1,<3.1", index = "https://www.paddlepaddle.org.cn/packages/stable/cpu/" }


[tool.pixi.feature.ocr-mineru-gpu.pypi-options]
extra-index-urls = ["https://wheels.myhloli.com"]

[tool.pixi.feature.ocr-mineru-gpu]
# just need osx for convenient resolution
platforms = ["osx-arm64", "linux-64" ]
[tool.pixi.feature.ocr-mineru-gpu.dependencies]
python = "~=3.10.0"

[tool.pixi.feature.ocr-mineru-gpu.pypi-dependencies]
# TODO: fix GPUs to work: https://github.com/prefix-dev/pixi/discussions/3217#discussioncomment-12338215
# magic-pdf = { version = ">1.0.0 ,<2", extras = ["full"] }
doclayout_yolo = { version = ">=0.0.2b1,<0.1" }
[feature.research-basics.target.linux-64.dependencies]
paddlepaddle = { version = ">=3.0.0rc1,<3.1", index = "https://www.paddlepaddle.org.cn/packages/stable/cu123/" }

observation

osx resolves just fine
linux fails with:

Error: 
  × failed to solve the pypi requirements of 'ocr-mineru' 'linux-aarch64'
  ├─▶ Failed to build `detectron2 @ git+https://github.com/facebookresearch/detectron2.git@9604f5995cc628619f0e4fd913453b4d7d61db3f`
  ├─▶ The build backend returned an error
  ╰─▶ Call to `setuptools.build_meta:__legacy__.prepare_metadata_for_build_wheel` failed (exit status: 1)
      
      [stderr]
      Traceback (most recent call last):
        File "<string>", line 14, in <module>
        File "/data/raid5/data/heiler/foo/.pixi/envs/ocr-mineru/lib/python3.12/site-packages/setuptools/build_meta.py", line 377, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/data/raid5/data/heiler/foo/.pixi/envs/ocr-mineru/lib/python3.12/site-packages/setuptools/build_meta.py", line 522, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/data/raid5/data/heiler/foo/.pixi/envs/ocr-mineru/lib/python3.12/site-packages/setuptools/build_meta.py", line 320, in run_setup
          exec(code, locals())
        File "<string>", line 10, in <module>
      ModuleNotFoundError: No module named 'torch'
      
      hint: This error likely indicates that `detectron2 @ git+https://github.com/facebookresearch/detectron2.git@9604f5995cc628619f0e4fd913453b4d7d61db3f` depends on `torch`, but
      doesn't declare it as a build dependency. If `detectron2 @ git+https://github.com/facebookresearch/detectron2.git@9604f5995cc628619f0e4fd913453b4d7d61db3f` is a first-party
      package, consider adding `torch` to its `build-system.requires`. Otherwise, `uv pip install torch` into the environment and re-run with `--no-build-isolation`.

is this a bug?

geoHeil Feb 27, 2025
Author

It looks like linux is more correct - as it fails right away. Osx fails when instanciating the env with pixi shell -e ocr-mineru.

This leads me to the following questions:

how to enable the GPU support
how to properly feed the isolation switch to allow using python 3.12
why is there the inconsistency between osx/linux? osx should also fail right away

lucascolley · 2025-07-29T14:49:19Z

lucascolley
Jul 29, 2025
Collaborator

@geoHeil do you still have questions here? If so, perhaps they could be broken out into new discussions and we can close this one?

1 reply

geoHeil Jul 29, 2025
Author

lets close for now

pip works - pixi fails to resolve for magic-pdf #3217

Uh oh!

geoHeil Feb 26, 2025

Replies: 2 comments · 12 replies

Uh oh!

nichmor Feb 26, 2025 Maintainer

Uh oh!

geoHeil Feb 26, 2025 Author

Uh oh!

nichmor Feb 27, 2025 Maintainer

Uh oh!

geoHeil Feb 27, 2025 Author

Uh oh!

nichmor Feb 27, 2025 Maintainer

Uh oh!

Uh oh!

geoHeil Feb 27, 2025 Author

Uh oh!

geoHeil Feb 27, 2025 Author

one more observation

observation

Uh oh!

Uh oh!

geoHeil Feb 27, 2025 Author

Uh oh!

lucascolley Jul 29, 2025 Collaborator

Uh oh!

geoHeil Jul 29, 2025 Author

geoHeil
Feb 26, 2025

Replies: 2 comments 12 replies

nichmor
Feb 26, 2025
Maintainer

geoHeil Feb 26, 2025
Author

nichmor Feb 27, 2025
Maintainer

geoHeil Feb 27, 2025
Author

nichmor Feb 27, 2025
Maintainer

geoHeil Feb 27, 2025
Author

geoHeil Feb 27, 2025
Author

geoHeil Feb 27, 2025
Author

lucascolley
Jul 29, 2025
Collaborator

geoHeil Jul 29, 2025
Author