Build succeeds but wheel is broken on cross compile. #2160
Description
Bug Description
The build succeeds both when specifying a target and not. The natively built wheel installs and imports fine. The wheel with a --target aarch64-unknown-linux-musl specified installs fine on the target architecture but the import fails ModuleNotFoundError: No module named 'rust_module.rust_module'.
I believe that if this is an issue with my linker or installation, maturin should fail on build. If this is a bug with the build please let me know what other information it would be helpful to gather.
Your maturin version (maturin --version)
maturin 1.7.0
Your Python version (python -V)
python 3.8.18, python 3.8.3
Your pip version (pip -V)
pip 23.2.1
What bindings you're using
pyo3
Does cargo build work?
- Yes, it works
If on windows, have you checked that you aren't accidentally using unix path (those with the forward slash /)?
- Yes
Steps to Reproduce
The Builds
Me@Computer:~/projects/rust_module# make clean
cargo clean
Removed 529 files, 123.9MiB total
rm -rf ./.tmp
rm -rf ./target
Me@Computer:~/projects/rust_module# make maturin-build
maturin build --release --target aarch64-unknown-linux-musl --zig -i python3.8 --strip
🔗 Found pyo3 bindings
🐍 Found CPython 3.8
📡 Using build options features, bindings, compatibility from pyproject.toml
Compiling target-lexicon v0.12.15
Compiling once_cell v1.19.0
Compiling proc-macro2 v1.0.86
Compiling unicode-ident v1.0.12
Compiling libc v0.2.155
Compiling autocfg v1.3.0
Compiling cfg-if v1.0.0
Compiling heck v0.5.0
Compiling bitflags v1.3.2
Compiling byteorder v1.5.0
Compiling unindent v0.2.3
Compiling bitflags v2.6.0
Compiling memoffset v0.9.1
Compiling indoc v2.0.5
Compiling pyo3-build-config v0.22.2
Compiling quote v1.0.36
Compiling syn v2.0.72
Compiling nix v0.26.4
Compiling i2cdev v0.6.1
Compiling pyo3-ffi v0.22.2
Compiling pyo3-macros-backend v0.22.2
Compiling pyo3 v0.22.2
Compiling pyo3-macros v0.22.2
Compiling rust_module v0.1.0 (/home/Me/projects/rust_module)
Finished `release` profile [optimized] target(s) in 12.90s
📖 Found type stub file at rust_module.pyi
📦 Built wheel for CPython 3.8 to /home/Me/projects/rust_module/target/wheels/rust_module-0.1.0-cp38-cp38-linux_aarch64.whl
Me@Computer:~/projects/rust_module# make clean
cargo clean
Removed 290 files, 71.8MiB total
rm -rf ./.tmp
rm -rf ./target
Me@Computer:~/projects/rust_module# make maturin-build-native
maturin build --release --zig -i python3.8 --strip
🔗 Found pyo3 bindings
🐍 Found CPython 3.8 at /home/Me/projects/.venv/bin/python3.8
📡 Using build options features, bindings, compatibility from pyproject.toml
Compiling target-lexicon v0.12.15
Compiling once_cell v1.19.0
Compiling proc-macro2 v1.0.86
Compiling libc v0.2.155
Compiling unicode-ident v1.0.12
Compiling autocfg v1.3.0
Compiling heck v0.5.0
Compiling cfg-if v1.0.0
Compiling bitflags v1.3.2
Compiling indoc v2.0.5
Compiling bitflags v2.6.0
Compiling byteorder v1.5.0
Compiling unindent v0.2.3
Compiling memoffset v0.9.1
Compiling pyo3-build-config v0.22.2
Compiling quote v1.0.36
Compiling syn v2.0.72
Compiling nix v0.26.4
Compiling i2cdev v0.6.1
Compiling pyo3-ffi v0.22.2
Compiling pyo3-macros-backend v0.22.2
Compiling pyo3 v0.22.2
Compiling pyo3-macros v0.22.2
Compiling rust_module v0.1.0 (/home/Me/projects/rust_module)
Finished `release` profile [optimized] target(s) in 13.91s
📖 Found type stub file at rust_module.pyi
📦 Built wheel for CPython 3.8 to /home/Me/projects/rust_module/target/wheels/rust_module-0.1.0-cp38-cp38-linux_x86_64.whl
Me@Computer:~/projects/rust_module#
Both succeed without warnings or errors.
Install and Import
In my local enviroment:
Me@Computer:~/projects/rust_module# python -m pip install --force-reinstall target/wheels/rust_module-0.1.0-cp38-cp38-linux_x86_64.whl
Looking in indexes: https://pypi.python.org/simple
Processing ./target/wheels/rust_module-0.1.0-cp38-cp38-linux_x86_64.whl
Installing collected packages: rust-module
Attempting uninstall: rust-module
Found existing installation: rust_module 0.1.0
Uninstalling rust_module-0.1.0:
Successfully uninstalled rust_module-0.1.0
Successfully installed rust-module-0.1.0
Me@Computer:~/projects/rust_module# python
Python 3.8.18 (default, Oct 5 2023, 14:23:18)
[GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rust_module
>>> rust_module.sum_as_string(1,2)
'3'
>>>
Me@Computer:~/projects/rust_module#
As can be seen the wheel has installed and executed as expected.
In the target architecture:
me@target:~# python -m pip install --force-reinstall rust_module-0.1.0-cp38-cp38-linux_aarch64.whl
Processing ./rust_module-0.1.0-cp38-cp38-linux_aarch64.whl
Installing collected packages: rust-module
Attempting uninstall: rust-module
Found existing installation: rust-module 0.1.0
Uninstalling rust-module-0.1.0:
Successfully uninstalled rust-module-0.1.0
Successfully installed rust-module-0.1.0
me@target:~# python
Python 3.8.3 (default, Sep 27 2021, 16:15:52)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rust_module
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/site-packages/rust_module/__init__.py", line 1, in <module>
from .rust_module import *
ModuleNotFoundError: No module named 'rust_module.rust_module'
>>>
me@target:~#
The wheel installs as expected. As can be seen an import error occurs. If this error is worked around by editing the init.py file the module also has none of the functions that are expected.
Builds with Logs
Me@Computer:~/projects/rust_module# export RUST_LOG=maturin=debug
Me@Computer:~/projects/rust_module# make maturin-build
maturin build --release --target aarch64-unknown-linux-musl --zig -i python3.8 --strip
2024-07-25T10:09:21.141621Z DEBUG into_build_context: maturin::project_layout: Found pyproject.toml in working directory at "/home/Me/projects/rust_module/pyproject.toml"
2024-07-25T10:09:21.152221Z DEBUG into_build_context: maturin::project_layout: Using cargo manifest path from working directory: "/home/Me/projects/rust_module/Cargo.toml"
2024-07-25T10:09:21.152546Z DEBUG into_build_context:resolve_cargo_metadata: maturin::project_layout: Resolving cargo metadata from "/home/Me/projects/rust_module/Cargo.toml"
2024-07-25T10:09:21.246007Z INFO into_build_context:resolve_cargo_metadata: maturin::project_layout: close time.busy=93.5ms time.idle=2.80µs
2024-07-25T10:09:21.246576Z DEBUG into_build_context: maturin::project_layout: Project layout resolved project_home/Me=/home/Me/projects/rust_module python_dir=/home/Me/projects/rust_module rust_module=/home/Me/projects/rust_module/rust_module python_module=/home/Me/projects/rust_module/rust_module extension_name=rust_module module_name=rust_module
🔗 Found pyo3 bindings
2024-07-25T10:09:21.306072Z DEBUG into_build_context: maturin::build_options: Found CPython 3.8 in bundled sysconfig
🐍 Found CPython 3.8
📡 Using build options features, bindings, compatibility from pyproject.toml
2024-07-25T10:09:21.306156Z INFO into_build_context: maturin::build_options: close time.busy=165ms time.idle=4.50µs
2024-07-25T10:09:21.309541Z DEBUG build_wheels: maturin::compile: Setting `-C target-features=-crt-static` for musl dylib
2024-07-25T10:09:22.376940Z DEBUG build_wheels: maturin::compile: Running env -u CARGO AR_aarch64_unknown_linux_musl="/home/Me/.cache/cargo-zigbuild/0.18.4/ar" CARGO_ENCODED_RUSTFLAGS="-C\u{1f}target-feature=-crt-static" CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER="/home/Me/.cache/cargo-zigbuild/0.18.4/zigcc-aarch64-unknown-linux-musl.sh" CC_aarch64_unknown_linux_musl="/home/Me/.cache/cargo-zigbuild/0.18.4/zigcc-aarch64-unknown-linux-musl.sh" CMAKE_TOOLCHAIN_FILE_aarch64_unknown_linux_musl="/home/Me/.cache/cargo-zigbuild/0.18.4/cmake/aarch64-unknown-linux-musl-toolchain.cmake" CXX_aarch64_unknown_linux_musl="/home/Me/.cache/cargo-zigbuild/0.18.4/zigcxx-aarch64-unknown-linux-musl.sh" PYO3_CONFIG_FILE="/home/Me/projects/rust_module/target/maturin/pyo3-config-aarch64-unknown-linux-musl-3.8.txt" RANLIB_aarch64_unknown_linux_musl="/home/Me/.cache/cargo-zigbuild/0.18.4/zigranlib.sh" ZIG_COMMAND="/home/Me/projects/.venv/bin/python3 -m ziglang" "cargo" "rustc" "--features" "pyo3/extension-module" "--target" "aarch64-unknown-linux-musl" "--message-format" "json-render-diagnostics" "--manifest-path" "/home/Me/projects/rust_module/Cargo.toml" "--release" "--lib" "--" "-C" "link-arg=-s"
Compiling libc v0.2.155
Compiling cfg-if v1.0.0
Compiling bitflags v1.3.2
Compiling unindent v0.2.3
Compiling byteorder v1.5.0
Compiling pyo3-build-config v0.22.2
Compiling memoffset v0.9.1
Compiling once_cell v1.19.0
Compiling bitflags v2.6.0
Compiling pyo3-macros-backend v0.22.2
Compiling pyo3-ffi v0.22.2
Compiling pyo3 v0.22.2
Compiling nix v0.26.4
Compiling i2cdev v0.6.1
Compiling pyo3-macros v0.22.2
Compiling rust_module v0.1.0 (/home/Me/projects/rust_module)
Finished `release` profile [optimized] target(s) in 8.72s
2024-07-25T10:09:31.149197Z INFO build_wheels:warn_missing_py_init: maturin::compile: close time.busy=642µs time.idle=3.00µs
2024-07-25T10:09:31.160070Z DEBUG build_wheels: maturin::module_writer: Adding rust_module-0.1.0.dist-info/METADATA
2024-07-25T10:09:31.162226Z DEBUG build_wheels: maturin::module_writer: Adding rust_module-0.1.0.dist-info/WHEEL
2024-07-25T10:09:31.163147Z DEBUG build_wheels:write_bindings_module: maturin::module_writer: Adding rust_module/__init__.py
📖 Found type stub file at rust_module.pyi
2024-07-25T10:09:31.163975Z DEBUG build_wheels:write_bindings_module: maturin::module_writer: Adding rust_module/__init__.pyi from /home/Me/projects/rust_module/rust_module.pyi
2024-07-25T10:09:31.165279Z DEBUG build_wheels:write_bindings_module: maturin::module_writer: Adding rust_module/py.typed
2024-07-25T10:09:31.165977Z DEBUG build_wheels:write_bindings_module: maturin::module_writer: Adding rust_module/rust_module.cpython-38-aarch64-linux-gnu.so from /home/Me/projects/rust_module/target/aarch64-unknown-linux-musl/release/maturin/librust_module.so
2024-07-25T10:09:31.200497Z INFO build_wheels:write_bindings_module: maturin::module_writer: close time.busy=37.4ms time.idle=3.40µs
2024-07-25T10:09:31.200604Z DEBUG build_wheels: maturin::module_writer: Adding rust_module-0.1.0.dist-info/RECORD
📦 Built wheel for CPython 3.8 to /home/Me/projects/rust_module/target/wheels/rust_module-0.1.0-cp38-cp38-linux_aarch64.whl
2024-07-25T10:09:31.202936Z INFO build_wheels: maturin::build_context: close time.busy=9.90s time.idle=1.60µs
Me@Computer:~/projects/rust_module#
Me@Computer:~/projects/rust_module# make maturin-build-native
maturin build --release --zig -i python3.8 --strip
2024-07-25T10:15:05.403827Z DEBUG into_build_context: maturin::project_layout: Found pyproject.toml in working directory at "/home/Me/projects/rust_module/pyproject.toml"
2024-07-25T10:15:05.404088Z DEBUG into_build_context: maturin::project_layout: Using cargo manifest path from working directory: "/home/Me/projects/rust_module/Cargo.toml"
2024-07-25T10:15:05.404296Z DEBUG into_build_context:resolve_cargo_metadata: maturin::project_layout: Resolving cargo metadata from "/home/Me/projects/rust_module/Cargo.toml"
2024-07-25T10:15:05.507164Z INFO into_build_context:resolve_cargo_metadata: maturin::project_layout: close time.busy=103ms time.idle=1.50µs
2024-07-25T10:15:05.507681Z DEBUG into_build_context: maturin::project_layout: Project layout resolved project_home/Me=/home/Me/projects/rust_module python_dir=/home/Me/projects/rust_module rust_module=/home/Me/projects/rust_module/rust_module python_module=/home/Me/projects/rust_module/rust_module extension_name=rust_module module_name=rust_module
🔗 Found pyo3 bindings
2024-07-25T10:15:05.623289Z DEBUG into_build_context:check_executable: maturin::python_interpreter: Found CPython interpreter at /home/Me/projects/.venv/bin/python3.8 executable=python3.8
2024-07-25T10:15:05.623354Z INFO into_build_context:check_executable: maturin::python_interpreter: close time.busy=38.1ms time.idle=2.10µs executable=python3.8
🐍 Found CPython 3.8 at /home/Me/projects/.venv/bin/python3.8
📡 Using build options features, bindings, compatibility from pyproject.toml
2024-07-25T10:15:05.623391Z INFO into_build_context: maturin::build_options: close time.busy=220ms time.idle=2.90µs
2024-07-25T10:15:06.204732Z DEBUG build_wheels: maturin::compile: Setting PYO3_PYTHON to /home/Me/projects/.venv/bin/python3.8
2024-07-25T10:15:06.204790Z DEBUG build_wheels: maturin::compile: Running env -u CARGO AR_x86_64_unknown_linux_gnu="/home/Me/.cache/cargo-zigbuild/0.18.4/ar" CARGO_TARGET_APPLIES_TO_HOST="false" CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER="/home/Me/.cache/cargo-zigbuild/0.18.4/zigcc-x86_64-unknown-linux-gnu.sh" CARGO_UNSTABLE_TARGET_APPLIES_TO_HOST="true" CC_x86_64_unknown_linux_gnu="/home/Me/.cache/cargo-zigbuild/0.18.4/zigcc-x86_64-unknown-linux-gnu.sh" CMAKE_TOOLCHAIN_FILE_x86_64_unknown_linux_gnu="/home/Me/.cache/cargo-zigbuild/0.18.4/cmake/x86_64-unknown-linux-gnu-toolchain.cmake" CXX_x86_64_unknown_linux_gnu="/home/Me/.cache/cargo-zigbuild/0.18.4/zigcxx-x86_64-unknown-linux-gnu.sh" PYO3_ENVIRONMENT_SIGNATURE="cpython-3.8-64bit" PYO3_PYTHON="/home/Me/projects/.venv/bin/python3.8" PYTHON_SYS_EXECUTABLE="/home/Me/projects/.venv/bin/python3.8" RANLIB_x86_64_unknown_linux_gnu="/home/Me/.cache/cargo-zigbuild/0.18.4/zigranlib.sh" ZIG_COMMAND="/home/Me/projects/.venv/bin/python3 -m ziglang" __CARGO_TEST_CHANNEL_OVERRIDE_DO_NOT_USE_THIS="nightly" "cargo" "rustc" "--features" "pyo3/extension-module" "--target" "x86_64-unknown-linux-gnu" "--message-format" "json-render-diagnostics" "--manifest-path" "/home/Me/projects/rust_module/Cargo.toml" "--release" "--lib" "--" "-C" "link-arg=-s"
Compiling target-lexicon v0.12.15
Compiling once_cell v1.19.0
Compiling proc-macro2 v1.0.86
Compiling unicode-ident v1.0.12
Compiling libc v0.2.155
Compiling autocfg v1.3.0
Compiling heck v0.5.0
Compiling cfg-if v1.0.0
Compiling bitflags v1.3.2
Compiling unindent v0.2.3
Compiling indoc v2.0.5
Compiling bitflags v2.6.0
Compiling byteorder v1.5.0
Compiling memoffset v0.9.1
Compiling quote v1.0.36
Compiling pyo3-build-config v0.22.2
Compiling syn v2.0.72
Compiling nix v0.26.4
Compiling i2cdev v0.6.1
Compiling pyo3-ffi v0.22.2
Compiling pyo3-macros-backend v0.22.2
Compiling pyo3 v0.22.2
Compiling pyo3-macros v0.22.2
Compiling rust_module v0.1.0 (/home/Me/projects/rust_module)
Finished `release` profile [optimized] target(s) in 12.05s
2024-07-25T10:15:18.304557Z INFO build_wheels:warn_missing_py_init: maturin::compile: close time.busy=703µs time.idle=2.40µs
2024-07-25T10:15:18.310640Z DEBUG build_wheels: maturin::module_writer: Adding rust_module-0.1.0.dist-info/METADATA
2024-07-25T10:15:18.311228Z DEBUG build_wheels: maturin::module_writer: Adding rust_module-0.1.0.dist-info/WHEEL
2024-07-25T10:15:18.311624Z DEBUG build_wheels:write_bindings_module: maturin::module_writer: Adding rust_module/__init__.py
📖 Found type stub file at rust_module.pyi
2024-07-25T10:15:18.312027Z DEBUG build_wheels:write_bindings_module: maturin::module_writer: Adding rust_module/__init__.pyi from /home/Me/projects/rust_module/rust_module.pyi
2024-07-25T10:15:18.312337Z DEBUG build_wheels:write_bindings_module: maturin::module_writer: Adding rust_module/py.typed
2024-07-25T10:15:18.312661Z DEBUG build_wheels:write_bindings_module: maturin::module_writer: Adding rust_module/rust_module.cpython-38-x86_64-linux-gnu.so from /home/Me/projects/rust_module/target/x86_64-unknown-linux-gnu/release/maturin/librust_module.so
2024-07-25T10:15:18.350877Z INFO build_wheels:write_bindings_module: maturin::module_writer: close time.busy=39.3ms time.idle=2.10µs
2024-07-25T10:15:18.350940Z DEBUG build_wheels: maturin::module_writer: Adding rust_module-0.1.0.dist-info/RECORD
📦 Built wheel for CPython 3.8 to /home/Me/projects/rust_module/target/wheels/rust_module-0.1.0-cp38-cp38-linux_x86_64.whl
2024-07-25T10:15:18.351889Z INFO build_wheels: maturin::build_context: close time.busy=12.7s time.idle=1.60µs
Me@Computer:~/projects/rust_module#