Add way to include OUT_DIR assets in Maturin builds

[orlp]

If you generate some Python file or other assets in build.rs there is (as far as I'm aware) currently no way to include this in a Maturin build through the OUT_DIR. Your only option is to either store it in the source directory so Maturin will pick it up, but Rust recommends against storing files in the source directory.

[orlp]

Note that for my use-case it's impossible to store this asset inside the Rust binary. The data stored contains the CPU flags the Rust binary was compiled with, so that they can be checked for compatibility before the C extension gets loaded into Python.

[messense]

Could you have a git-ignored directory in source directory, put generated files in there and use [tool.maturin.include] to include them in sdist/wheel?

[orlp]

I can and we do, but it's explicitly not recommended by Rust that you don't:

Build scripts may save any output files or intermediate artifacts in the directory specified in the OUT_DIR environment variable. Scripts should not modify any files outside of that directory.

[messense]

I guess we could extend [tool.maturin.include] to support something like:

[tool.maturin]
incude = { from = "$OUT_DIR/<artifact path>", to = "<source dir>/<artifact path>" }

but I'm not sure if it's easy to get $OUT_DIR since it's dynamic. And any crate can put anything in its $OUT_DIR so we probably need more information here.

[konstin]

Is this about SIMD and similar CPU extensions? In that case, this is better served by wheel variants tagging CPU extension: python/peps#4740. Instead of a runtime error, avoid installing incompatible wheels in the first place.

In the meantime, is it possible to inspect the binary itself from the Python side, and e.g. put extension info in a symbol in the binary?