nanobind: can't build against non-Latest Python Versions

[stellarpower]

Xmake Version

v3.0.6+20260112

Operating System Version and Architecture

OpenSUSE Tumbleweed (Container)

Describe Bug

Hi,

I'm having real problems trying to build an extension using anything but the latest/default version of python.

I have started from scratch in a container environment with a brand new installation of xmake:

FROM opensuse/tumbleweed

RUN zypper refresh
RUN zypper --non-interactive in -y python311 unzip
RUN ln -s /usr/bin/python3.11 /usr/local/bin/python3
RUN curl -fsSL https://xmake.io/shget.text | bash
ENV XMAKE_ROOT=y 
RUN source ~/.xmake/profile ; yes | xrepo install nanobind

xmake sporadically tries to install Python3.13, I haven't managed to log and work out where and when. However byu hitting cancel and see the recipe below, I don;t have it installed:

xrepo scan:

scanning packages ..
ca-certificates-20250131:
  -> e9ba281ae183402a80dc072d4d571c20: linux, x86_64
    -> {debug=false,pic=true,shared=false}
cmake-4.2.1:
  -> bf711153e47044bdaa806a75cf88fe51: linux, x86_64, unused
    -> {debug=false,pic=true,shared=false}
python-3.11.9:
  -> 00e3a7732c1e4c61a825d542de39c544: linux, x86_64
    -> {debug=false,headeronly=false,pic=true,shared=false}
nanobind-v2.10.2:
  -> 083ad9b779c54a3c863e16c569a593b3: linux, x86_64
    -> {debug=false,pic=true,shared=false}
ninja-v1.13.1:
  -> e7cc4735aeb048e6870171a23c4ee800: linux, x86_64, unused
    -> {debug=false,pic=true,shared=false}
openssl-1.1.1-w:
  -> 6c51ab6278e2479b883dffafac69fdaf: linux, x86_64
    -> {debug=false,pic=true,shared=false}
libffi-3.4.8:
  -> 46686cf534074f4ebf90e4ab1f9662dd: linux, x86_64
    -> {debug=false,pic=true,shared=false}
robin-map-v1.4.1:
  -> fc76e5d54201422db0da0db8f9a73e1f: linux, x86_64
    -> {debug=false,pic=true,shared=false}

xmake.lua:

add_rules("mode.debug", "mode.release")

set_languages("c++23")


add_requires("nanobind")
add_requireconfs("nanobind.python", { version = "3.11", override = true }) 


target("Test311")
    --set_policy("build.python", "/usr/bin/python3.11")

    add_rules("python.library")
    
    add_packages("nanobind")

    add_files("pythonEntrypoint.cpp")
    

This builds an extension for 3.11 fine in the container (on the host, I had to start removing newer versions from my path), but it's still broken:

9c55327f1006:/w/newProject/build/linux/x86_64/release # nm -D Test311.cpython-311-x86_64-linux-gnu.so  |& grep -i RaisedExc
                 U PyErr_GetRaisedException
                 U PyErr_SetRaisedException

9c55327f1006:/w/newProject/build/linux/x86_64/release # python3.11
Python 3.11.14 (main, Dec 19 2025, 22:09:22) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from Test311 import *
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /w/newProject/build/linux/x86_64/release/Test311.cpython-311-x86_64-linux-gnu.so: undefined symbol: PyErr_GetRaisedException

This symbol was added in the 3.12 API, soi even though it's building for 3.11, the internals of nanobind are not seeing the correct version and a macro is expanding to something unsupported.

According to this link, the package ought to force the version with CMake for when it calls `find_package

I think I fixed this on my machine only 3-4 months ago with a monkey-patch of the sources, but now can't find any record of how I did it. I remember trying to add variables to the cmake command-line flags and they were ignored.

There's also the question of xmake using multiple different python installations as part of the process - there's the system python, there's the package it downloads, and the installation complained it could not find python in the container unless python3 was in the path. The package depends on python, but that may be different from the version I want to build against.

I'm sure it's a complicated problem and there isn't an easy solution to cover all bases, Python is just the modern version of DLL hell. But for now it seems confusing and to be dancing around with different installations in the same build. I think the simplest solution would be to explicitly point the nanobind package towards a particular path, similar to how PYTHON_HOME works - i.e. I must explicitly tell it I want to use a system/virtualenv/conda/other external interpreter with a given version and path (and if headers are needed and not present, it's a build failure), or I want xmake to go and download a version.

Thinking about it, I'm not sure how useful it is for xmake to be downloading a version - to build an extension we need to target a version because Python's API is so unstable. I'm not sure of a usecase beyond testing where I might want to build an extension but not have a specific version I want/need to target. I guess part of CI maybe, but then I would still have a version I know. So it feels like something similar to

add_requires("nanobind", { config = { version = "3.11", system = true } })
or
add_requires("nanobind", { config = { version = "3.11", path = "..." } })
or
add_requires("nanobind", { config = { version = "3.12", download = true } })

might be a useful way to go, and make it an error if I don't specify something.

Thanks

Expected Behavior

see above

Project Configuration

see above

Additional Information and Error Logs

see above

[star-hengxing]

I've been struggling with the same issue in xmake, as there’s very little community effort to improve its Python packages. Instead, we can use tools like uv or pixi, which have excellent Python support. That's what I use for my own projects at work.
reference: #7711 (comment)

add_requires("nanobind", { config = { version = "3.11", system = true } }) or add_requires("nanobind", { config = { version = "3.11", path = "..." } }) or add_requires("nanobind", { config = { version = "3.12", download = true } })

By the way, all of these approaches are incorrect.

[stellarpower]

By the way, all of these approaches are incorrect.

Thanks, wouldn't surprise me. Without documentation on the right way I'm just trying things people suggested in discord.

uv/pixi would be analogous to conda/mamba, right? This would help with making sure only one version is on the path but even in a container with exactly one version available on the system (outside of installing through xmake itself) and it's failing to build properly. I think first we need to make sure that cmake is explicitly given the version that xmake is using so that those two are in-sync. Then after that could look at specifying the version we want only once.

[star-hengxing]

Personally, I position pixi as a project-level package manager. It can manage C++ toolchains for me, such as cmake and ninja, and can also be used to download formatters—you can refer to the project for more details.
https://github.com/clice-io/clice/blob/main/pixi.toml

By running xmake f -c -v, we can observe that most of the reasons why xmake fails to find the system Python are due to detection failures of these three tools.

xmake-repo/packages/p/python/fetch.lua

Lines 14 to 16 in 229fa1f

	os.vrunv(result.program, { "-c", "import pip" })
	os.vrunv(result.program, { "-c", "import setuptools" })
	os.vrunv(result.program, { "-c", "import wheel" })