How to install Cython files next to their files inside the repo, instead of in build/ directory

[adam2392]

Hi,

My question is how do you specify to not use a build directory when building Cython files?

I am trying to use Meson to build and install Cython files locally such that their compiled files are next to their pyx/pxd files inside the repo. This is similar to how in scikit-learn, you can do pip install --verbose --no-build-isolation --editable . and the Cython binaries appear next to their file counterparts inside the repo.

I use the following command meson setup build --prefix=$PWD/build-install , which gives me:

The Meson build system
Version: 1.0.0
Source dir: /Users/adam2392/Documents/scikit-tree
Build dir: /Users/adam2392/Documents/scikit-tree/build
Build type: native build
Project name: scikit-tree
Project version: 0.0.0.dev0
C compiler for the host machine: arm64-apple-darwin20.0.0-clang (clang 14.0.6 "clang version 14.0.6")
C linker for the host machine: arm64-apple-darwin20.0.0-clang ld64 609
C++ compiler for the host machine: arm64-apple-darwin20.0.0-clang++ (clang 14.0.6 "clang version 14.0.6")
C++ linker for the host machine: arm64-apple-darwin20.0.0-clang++ ld64 609
Host machine cpu family: aarch64
Host machine cpu: aarch64
Library m found: YES
Program cython found: YES (/Users/adam2392/miniforge3/envs/sktree/bin/cython)
Program python3 found: YES (/Users/adam2392/miniforge3/envs/sktree/bin/python3)
Message: /Users/adam2392/miniforge3/envs/sktree/bin/python3
Message: /Users/adam2392/Documents/scikit-tree/build-install/lib/python3.9/site-packages/
Found pkg-config: /opt/homebrew/bin/pkg-config (0.29.2)
Program _build_utils/cythoner.py found: YES (python /Users/adam2392/Documents/scikit-tree/sktree/_build_utils/cythoner.py)
Build targets in project: 1

scikit-tree 0.0.0.dev0

  User defined options
    prefix: /Users/adam2392/Documents/scikit-tree/build-install

Found ninja-1.11.1 at /opt/homebrew/bin/ninja

When I then run ninja:

>>> ninja -C build
ninja: Entering directory `build'
[1/3] Generating 'sktree/tree/_unsup_criterion.cpython-39-darwin.so.p/_unsup_criterion.c'.
inside cythoner: 
input file:  /Users/adam2392/Documents/scikit-tree/sktree/tree/_unsup_criterion.pyx
output file:  /Users/adam2392/Documents/scikit-tree/build/sktree/tree/_unsup_criterion.cpython-39-darwin.so.p/_unsup_criterion.c
[3/3] Linking target sktree/tree/_unsup_criterion.cpython-39-darwin.so
ld: warning: -pie being ignored. It is only used when linking a main executable

Internally, I use a cythoner.py file to run the Cythonize commands. I copied this from https://github.com/scipy/scipy/blob/main/scipy/_build_utils/cythoner.py. However, how do I specify to not install in the build/ subdirectory?

I want the output file instead to be:

input file:  /Users/adam2392/Documents/scikit-tree/sktree/tree/_unsup_criterion.pyx
output file:  /Users/adam2392/Documents/scikit-tree/sktree/tree/_unsup_criterion.cpython-39-darwin.so.p/_unsup_criterion.c

[bruchar1]

It is in meson design philosophy to build all projects out-of-source. Therefore, I think you must think the problem in a different way.

To be able to load the generated .so files, one thing you could do it to add the build directory the the PYTHONPATH.

If this doesn't work for you, another approach would be to copy the source files to the build directory. using fs.copyfile.

[eli-schwartz]

I think the only reason this matters is for the sake of editable installs, right? (To allow other projects to then detect scikit-tree and make use of the cython headers in their own builds.)

So perhaps overall it's a better idea to make meson-python (the PEP 517 backend) support this transparently.

[adam2392]

Yeah pretty much want it for editable installs, but upon further reading, it seems that editable installs is not supported yet(?)