CMake: use of `Python_SITELIB`

[nim65s]

Hi,

nanoeigenpy/CMakeLists.txt

Line 242 in 325645b

LIBRARY DESTINATION ${Python_SITELIB}

CMake FindPython seems to populate Python_SITELIB to an absolute system path.

ROS builds are thus failing with eg. https://build.ros2.org/job/Kdev__nanoeigenpy__ubuntu_noble_amd64/4/console:

PermissionError: [Errno 13] Permission denied: '/usr/local/lib/python3.12/dist-packages/nanoeigenpy.pyi'
CMake Error at cmake_install.cmake:89 (file):
  file INSTALL cannot copy file
  "/tmp/ws/build_isolated/nanoeigenpy/lib/nanoeigenpy.cpython-312-x86_64-linux-gnu.so"
  to
  "/usr/local/lib/python3.12/dist-packages/nanoeigenpy.cpython-312-x86_64-linux-gnu.so":
  Permission denied.

Same issue on Nix side, where this is patched: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/python-modules/nanoeigenpy/default.nix#L38

PermissionError: [Errno 13] Permission denied:
'/nix/store/…-python3-3.12.9/lib/python3.12/site-packages/nanoeigenpy.pyi'

That CMake variable is initialized from sysconfig.get_path('purelib'), but in other projects we instead use Path(sysconfig.get_path('purelib')).relative_to(sysconfig.get_path('data')) to fix this issue, ref. https://github.com/jrl-umi3218/jrl-cmakemodules/blob/c815b3ba45fa8b7cf4b7a21b69b26ad0bc04fd5c/python.cmake#L310C62-L310C137

[jorisv]

I like what @ahoarau did here.
By default it will install in Python_SITELIB except if the user give a PYTHON_BINDING_CUSTOM_INSTALL_DIR.
It's simple and can manage all cases.

[nim65s]

Yes, I think me must provide an option for the end user. With the current jrl-cmakemodules, that option is simply -DPYTHON_SITELIB=….

But I think the correct default to install python module is the relative lib/python3.X/site-packages. Or at least an absolute path inside $CMAKE_INSTALL_PREFIX: I think it is very wrong to install anything outside of this prefix.

The absolute Python_SITELIB from CMake FindPython can only work for people installing in venv / conda env. I mean, currently, it doesn't even work on a out-of-the-box ubuntu:

FROM ubuntu:25.04

RUN --mount=type=cache,sharing=locked,target=/var/cache/apt \
    --mount=type=cache,sharing=locked,target=/var/lib/apt \
    apt update \
 && DEBIAN_FRONTEND=noninteractive \
    apt install -qqy --no-install-recommends \
    build-essential \
    cmake \
    git \
    libeigen3-dev \
    libpython3-dev \
    nanobind-dev \
    python-is-python3 \
    python3-numpy \
    python3-scipy

RUN useradd user
USER user
WORKDIR /home/user

ADD --chown=user https://github.com/Simple-Robotics/nanoeigenpy/releases/download/v0.4.0/nanoeigenpy-0.4.0.tar.gz .
RUN tar xf nanoeigenpy-0.4.0.tar.gz
RUN cmake -B build -S nanoeigenpy-0.4.0 -DCMAKE_INSTALL_PREFIX=install
RUN cmake --build build
RUN cmake --install build

fails with:

=> ERROR [stage-0 9/9] RUN cmake --install build                                                                                                                                                               0.4s
------                                                                                                                                                                                                               
 > [stage-0 9/9] RUN cmake --install build:                                                                                                                                                                          
0.191 -- Install configuration: "Release"                                                                                                                                                                            
0.199 Error: could not load cache                                                                                                                                                                                    
0.204 -- Installing: /home/user/install/include/nanoeigenpy/config.hpp                                                                                                                                               
0.204 -- Installing: /home/user/install/include/nanoeigenpy/deprecated.hpp                                                                                                                                           
0.204 -- Installing: /home/user/install/include/nanoeigenpy/warning.hpp
0.342 Module "nanoeigenpy" ..
0.342   - importing ..
0.342   - analyzing ..
0.342   - writing stub "/usr/local/lib/python3.13/dist-packages/nanoeigenpy.pyi" ..
0.343 Traceback (most recent call last):
0.343   File "/usr/share/nanobind/stubgen.py", line 1428, in <module>
0.343     main()
0.343     ~~~~^^
0.343   File "/usr/share/nanobind/stubgen.py", line 1418, in main
0.343     with open(file, "w", encoding='utf-8') as f:
0.343          ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
0.343 PermissionError: [Errno 13] Permission denied: '/usr/local/lib/python3.13/dist-packages/nanoeigenpy.pyi'
0.352 -- Installing: /usr/local/lib/python3.13/dist-packages/nanoeigenpy.cpython-313-x86_64-linux-gnu.so
0.352 CMake Error at build/cmake_install.cmake:89 (file):
0.352   file INSTALL cannot copy file
0.352   "/home/user/build/lib/nanoeigenpy.cpython-313-x86_64-linux-gnu.so" to
0.352   "/usr/local/lib/python3.13/dist-packages/nanoeigenpy.cpython-313-x86_64-linux-gnu.so":
0.352   Permission denied.
0.352 
0.352 
------
Dockerfile:26
--------------------
  24 |     RUN cmake -B build -S nanoeigenpy-0.4.0 -DCMAKE_INSTALL_PREFIX=install
  25 |     RUN cmake --build build
  26 | >>> RUN cmake --install build
  27 |     
--------------------
ERROR: failed to build: failed to solve: process "/bin/sh -c cmake --install build" did not complete successfully: exit code: 1

[jorisv]

That why I'm looking for a way to install python binding through build-system in pyproject.toml.
This remove custom CMake code to handle that. I previously send you a POC about this and this is also working in the proxsuite refactor.

[nim65s]

I don't get it.

On that link I can see a PYTHON_BINDING_CUSTOM_INSTALL_DIR = "proxsuite".

So I say "the current preconfigured cmake thing is wrong".
You say "if we add an additional layer, the user call pip, pip call build-backend, build-backend call cmake with a preconfigured override to the current wrongly preconfigured cmake thing, and it works".

I mean, we agree the current preconfigured thing is wrong, and there is still need preconfiguration. What am I missing ?

(I also don't get why you are "looking for a way to install python binding through build-system in pyproject.toml", because we already have exactly that with cmeel)

Or is it the other way around ? Do you intend to call pip from within cmake ?

[jorisv]

Hello, I sent you a sample project on gitlab with a README explaining my motivation.

We can also make a call to speak about this issue, it will be probably faster :)

[jorisv]

After a talk:

• We will no longer use the nanobind way of installing in ${Python_SITELIB}: we will ensure installation is done inside CMAKE_INSTALL_PREFIX
• We will provide PYTHON_BINDING_CUSTOM_INSTALL_DIR to allow user to provide a custom install path

[ManifoldFR]

Doesn't nanobind offer a facility for installing in a directory you want?

[nim65s]

I don't think so :(

We have OUTPUT ${Python_SITELIB}/nanoeigenpy.pyi, and that output should be relative to CMAKE_INSTALL_PREFIX according to https://nanobind.readthedocs.io/en/latest/api_cmake.html#stub-generation, but it is not nanobind responsibility to decide in which subdir of CMAKE_INSTALL_PREFIX this should go.

[ManifoldFR]

One solution would be to provide a prefix override to nanobind's commands perhaps

We could go without nanobind's tool for now but still make a proposal for such a thing upstream if we explain what we want to solve.