Rust extension given invalid name by meson-python

[medley56]

Link to reproducible example: https://github.com/medley56/python_rust

When using meson-python to package a Rust extension, I get the following error when running pip install .

../meson.build:10:14: ERROR: Rust crate python_rust_lib.cpython-312-darwin type dylib does not allow spaces, periods or dashes in the library name due to a limitation of rustc. Replace them with underscores, for example

The project works when using maturin as a build backend so I'm reasonably sure it's not a problem with the actual extension. I believe the problem is in meson-python or in my configuration of meson-python, causing the backend to build the rust extension with an invalid name.

Platform: MacOS Sequioia 15.3

Documentation I've consulted:
Tutorial: https://mesonbuild.com/meson-python/tutorials/introduction.html#tutorial
meson docs for extension_module: https://mesonbuild.com/Python-module.html#extension_module
PyO3 docs (uses maturin): https://pyo3.rs/v0.23.4/module.html

[rgommers]

Thanks for the bug report and reproducer @medley56! We should make that work - I think the reason you're running into it is that you're the first one to try. Rust support in Meson is pretty recent, and none of the maintainers of meson-python are regular Rust users I believe.

One thing that jumps out when browsing the code is that this line in meson.build:

py = import('python').find_installation()

should be:

py = import('python').find_installation(pure: false)

That may not be the problem though, I'll try to test as soon as possible.

[rgommers]

Let's call this a feature request rather than a bug. Maturin does the following (from https://www.maturin.rs/bindings#pyo3): "maturin automatically detects pyo3 bindings when it's added as a dependency in Cargo.toml."

Neither Meson nor meson-python does anything like this. Using py.extension() on a Rust source file needs to do either something like what Maturin does (adding a pyo3 dependency and invoking the right machinery), or the user must do this explicitly in meson.build instead of feeding a single .rs file to py.extension.

I did a bit of digging in the Meson issue tracker, but can't find any relevant discussions - it may be that no one has worked on this yet.

[rgommers]

In meson-python itself we should probably start by adding some test cases for packaging projects that are known to work with Meson as wheels, just to make sure that that all works (it should).

[medley56]

Thanks for the attention to this! Looking forward to being able to use meson to build a Rust extension. It sounds like you have an idea for a workaround:

or the user must do this explicitly in meson.build instead of feeding a single .rs file to py.extension

Is that something simple/useful or would you rather just call this unsupported until a better solution can be implemented? For my part, I'm happy to just use maturin for now.

[greglucas]

It looks like if the crate is declared as cdylib, then we need to specify the rust_abi in the extension module.

rust_ext = py.extension_module(
    'python_rust_lib',  # Output name
    'src/lib.rs',  # Rust source
    install: true,
    rust_abi: 'c'
)

However, it appears that building the extension module still doesn't follow the crate dependencies and install those (it can't find pyo3 when compiling). Do you need to manually declare a dependency on other crates?

[eli-schwartz]

Meson can have a rust target depend on a crate: https://mesonbuild.com/Wrap-dependency-system-manual.html#cargo-wraps

dependencies: [dependency('foo-rs')]

and a cargo subproject dependency will automatically parse Cargo.toml and depend on its dependencies. But you still need to define your direct dependencies, just like you do with cargo (but with cargo, you define them in Cargo.toml instead of meson.build).

FTR: I don't really have experience with meson's rust support, that is being primarily driven by maintainers from the Gnome and Mesa ecosystems. I don't know offhand about anyone doing python module experiments with it, the experiments I've seen have involved replacing C/C++ libraries and executables. In theory, pyo3 extensions are "just" libraries that happen to utilize libpython, so it should be more or less the same modulo usage of the pyo3 crate itself.

[dnicolodi]

I've tried to put together an example project building with Meson a Python module using PyO3. The issue with the cdylib being given a Rust invalid name can be worked around compiling a Rust static lib and then linking it into a python module whose sources are an empty C file (at least Meson does not complain about this, I'm not sure if it actually works when linking). The second ingredient is a Cargo.lock that lists PyO3 and all transitive dependencies. Then Meson is able to resolve dependency('pyo3-0.25-rs'), generate wrap files for all required crates and start building. However, the build does not go very far: one of the PyO3 dependencies uses some proc_macro dependency and AFAICT Meson is not yet able to make sense of that. I don't know much about Rust (yet) and I don't know where to go from here.

I think refusing the name for the cdylib is a Meson bug and someone more familiar with Rust should probably have a look at this. I had a quick look at how maturin compiles Python extensions and AFAICT it directly instructs cargo to compile a cdylib with a name equivalent to the one the Meson's python module uses.

Paging @xclaesse, @dcbaker, and @bonzini for a consult.