Skip to content

ENH: add tool.meson-python.wheel.exclude setting #766

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

dnicolodi
Copy link
Member

@dnicolodi dnicolodi commented Jun 14, 2025

I am not entirely sure we want to include this facility: it may potentially be abused when better solutions exist. However, I think it is better than the solution we currently recommend https://mesonbuild.com/meson-python/how-to-guides/shared-libraries.html#shared-library-from-subproject of adding specific options to subprojects to exclude this or that parts from the installation.

@dnicolodi dnicolodi force-pushed the wheel-exclude branch 6 times, most recently from 6b98023 to d68c896 Compare June 14, 2025 14:02
@dnicolodi
Copy link
Member Author

Another idea I was toying with is to extend the filtering by meson install tags to allow to specify different tags per subproject. meson-python does not rely on the meson implementation of the filtering anyway, thus extending the facility is possible.

@dnicolodi dnicolodi added the enhancement New feature or request label Jun 24, 2025
@rgommers rgommers changed the title EHN: add tool.meson-python.wheel.exclude setting ENH: add tool.meson-python.wheel.exclude setting Jul 15, 2025
Copy link
Contributor

@rgommers rgommers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @dnicolodi! I like this. Agreed that these unmappable install locations/targets are a pain point, and this is a nice and ergonomic solution I think.

The wheel.exclude name is good, and it matches the name in scikit-build-core which is helpful.

One question about using introspection metadata locations. I can see that working, but there's also an alternative of using wheel locations - I'm not quite sure which is better.

@rgommers
Copy link
Contributor

Another idea I was toying with is to extend the filtering by meson install tags to allow to specify different tags per subproject. meson-python does not rely on the meson implementation of the filtering anyway, thus extending the facility is possible.

That may still require changes to the subproject itself (e.g., there are multiple shared libraries and you need only one of them), so is less general. I prefer wheel.exclude because of that.

@dnicolodi
Copy link
Member Author

We could also consider implementing both wheel.exclude and wheel.include so that the first could be used to exclude a whole subtree and the latter to include selected files in the same subtree.

@rgommers
Copy link
Contributor

We could also consider implementing both wheel.exclude and wheel.include so that the first could be used to exclude a whole subtree and the latter to include selected files in the same subtree.

I can see the appeal of that, yes. Especially on projects with lots of install targets, that seems nice. Although the cases where you can't make things concise with excluding install tags + wheel.exclude may be a small minority.

This could be done immediately, or later if there's demand. Given this PR is very nice and clean, may be okay to add it immediately?

@dnicolodi
Copy link
Member Author

I've implemented tool.meson-python.wheel.include too. I think this should be ready now.

Working on this it came to my mind that we could also have a tool.meson-python.wheel.map that allows to map files installed in one location to another. I'm just worried that it may be misused.

@rgommers
Copy link
Contributor

Working on this it came to my mind that we could also have a tool.meson-python.wheel.map that allows to map files installed in one location to another. I'm just worried that it may be misused.

Yeah potentially useful, but also potentially easy to misuse. This thread from yesterday is a good example: https://discuss.python.org/t/include-header-above-project-root/102221. Could potentially be a nicer way to include a C library that's one folder up in the repo, instead of symlinking it in - we've had examples of this, like nanoarrow.

On the other hand, this seems quite bad: https://hatch.pypa.io/latest/config/build/#forced-inclusion. It gives an explicit example of pulling in a header file from outside the repo, which seems like a good way to create non-reproducible builds/issues.

@eli-schwartz
Copy link
Member

Hatchling has had some really quite inspirational bugs if I recall correctly, related to trying to match inclusion patterns against parent directories of the project root. Very fun, in the sense of not being fun at all, when it incorrectly operates on the standard system directory structure used for ALL package builds.

Could potentially be a nicer way to include a C library that's one folder up in the repo, instead of symlinking it in - we've had examples of this, like nanoarrow.

Not really sure I understand the problem. If you're building a C library one folder up in the repo, I'm guessing this is a C library and its optional bindings. You really want the entire project to be a single build including both C and python bits. Autotools / Meson / CMake can all do that already, with varying abilities to install the python code in the right site-packages directory, and with meson-python you can even support both building a system C library and its bindings, OR a PyPI wheel that statically links the C library for standalone distribution.

You really don't want to require "one folder up" because it's not possible to produce an sdist like that, and you can't guarantee the C library is built when needed if it's two separate builds.

@dnicolodi
Copy link
Member Author

Yeah potentially useful, but also potentially easy to misuse. This thread from yesterday is a good example: https://discuss.python.org/t/include-header-above-project-root/102221. Could potentially be a nicer way to include a C library that's one folder up in the repo, instead of symlinking it in - we've had examples of this, like nanoarrow.

What I had in mind would not solve this issue. What I had in mind is a facility that maps a Meson installation path, for example to another, for example {py_platlib}/package/foolib.so to {py_platlib}/package/foolib.so. However, just while writing this I realized that a simple mapping like this is not going to be easy to use as the filename to be mapped is platform dependent.

@rgommers
Copy link
Contributor

Not really sure I understand the problem. If you're building a C library one folder up in the repo, I'm guessing this is a C library and its optional bindings. You really want the entire project to be a single build including both C and python bits. Autotools / Meson / CMake can all do that already,

The point is that you can't do it through a standard build frontend interface if the C library code isn't inside the part of the repo containing pyproject.toml. gh-674 is the example I was thinking about, there have been others. The problem to solve is how to invoke python -m build --wheel or similar and pull in that C library.

What I had in mind would not solve this issue.. [...] I realized that a simple mapping like this is not going to be easy to use as the filename to be mapped is platform dependent.

Good point. Okay, leaving that problem for another day then:)

The added include part also looks good on a first read. Let me see if I can test and hit the green button on this one after dinner.

@rgommers
Copy link
Contributor

However, I think it is better than the solution we currently recommend https://mesonbuild.com/meson-python/how-to-guides/shared-libraries.html#shared-library-from-subproject of adding specific options to subprojects to exclude this or that parts from the installation.

I assume you want to leave an update to those docs out of this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants