Generate bindings in build.rs to fix cross-compilation

[ivan-aksamentov]

Related to: rust-ndarray/ndarray-linalg#384

The pre-generated bindings are not working on the platforms that is not the same platform these bindings are generated at: /src/lapack.rs

error[E0308]: mismatched types
     --> /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lax-0.16.0/src/cholesky.rs:30:26
      |
30    |                     $trf(uplo.as_ptr(), &n, AsPtr::as_mut_ptr(a), &n, &mut info);
      |                     ---- ^^^^^^^^^^^^^ expected `*const u8`, found `*const i8`
      |                     |
      |                     arguments to this function are incorrect
...
41    | impl_cholesky_!(c64, lapack_sys::zpotrf_);
      | ----------------------------------------- in this macro invocation
      |
      = note: expected raw pointer `*const u8`
                 found raw pointer `*const i8`

In particular, this is failing during pretty much any attempt at cross-compilation. And might cause problems during native compilation on different platforms as well. For example, when cross-compiling to aarch64-unknown-linux-gnu, the defaut C integer types are different from what's on x86_64: build.log.txt

The preferred approach nowadays is to generate bindings at build-time and then programmatically paste them into lib.rs. This way they come out with all the specifics required for the target platform.

Here I:

• add build.rs script which uses bindgen library to generate $OUTPUT_DIR/bindings.rs (in the target/ directory)

• add the functions starting with underscore to the allowlist (.allowlist_function("^.*_$"))

• include!() the $OUTPUT_DIR/bindings.rsinto lib.rs

• remove the generate.sh script (the build.rs is now doing it's job)

• remove pre-generated bindings

[chadmed]

This does not just fail with cross compilation, it fails when compiling natively on non-x86 hosts too. I have had to postpone indefinitely a Rust signal processing project intended to be run on aarch64 because of this issue.

[IvanUkhov]

Hello! Sorry for being slow. Now trying to catch up with what has happened. According to the discussion in the linked PR, it has been resolved. At least, as far as ndarray-linalg is concerned. @ivan-aksamentov, is this the case?

[lmmx]

I just encountered this building for aarch64 with maturin-action with a ndarray-linalg dependency that induces a dependency on lapack-sys. Unless I am missing a simple version upgrade fix then I’d say it’s not resolved

Click to show error

error[E0308]: mismatched types
    --> /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/lax-0.16.0/src/tridiagonal/solve.rs:38:25
     |
37   |                     $trs(
     |                     ---- arguments to this function are incorrect
38   |                         t.as_ptr(),
     |                         ^^^^^^^^^^ expected `*const u8`, found `*const i8`
...
63   | impl_solve_tridiagonal!(f64, lapack_sys::dgttrs_);
     | ------------------------------------------------- in this macro invocation
     |
     = note: expected raw pointer `*const u8`
                found raw pointer `*const i8`
note: function defined here
    --> /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/lapack-sys-0.14.0/src/lapack.rs:6595:12
     |
6595 |     pub fn dgttrs_(
     |            ^^^^^^^
     = note: this error originates in the macro `impl_solve_tridiagonal` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
    --> /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/lax-0.16.0/src/tridiagonal/solve.rs:38:25
     |
37   |                     $trs(
     |                     ---- arguments to this function are incorrect
38   |                         t.as_ptr(),
     |                         ^^^^^^^^^^ expected `*const u8`, found `*const i8`
...
64   | impl_solve_tridiagonal!(f32, lapack_sys::sgttrs_);
     | ------------------------------------------------- in this macro invocation
     |
     = note: expected raw pointer `*const u8`
                found raw pointer `*const i8`
note: function defined here
    --> /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/lapack-sys-0.14.0/src/lapack.rs:6610:12
     |
6610 |     pub fn sgttrs_(
     |            ^^^^^^^
     = note: this error originates in the macro `impl_solve_tridiagonal` (in Nightly builds, run with -Z macro-backtrace for more info)

For more information about this error, try `rustc --explain E0308`.
error: could not compile `lax` (lib) due to 92 previous errors
💥 maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_CONFIG_FILE="/home/runner/work/powell-opt/powell-opt/target/maturin/pyo3-config-aarch64-unknown-linux-gnu-3.9.txt" "cargo" "rustc" "--features" "pyo3/extension-module" "--target" "aarch64-unknown-linux-gnu" "--message-format" "json-render-diagnostics" "--manifest-path" "/home/runner/work/powell-opt/powell-opt/Cargo.toml" "--release" "--lib"`
Error: The process '/usr/bin/docker' failed with exit code 1
    at ExecState._setResult (/home/runner/work/_actions/PyO3/maturin-action/aef21716ff3dcae8a1c301d23ec3e4446972a6e3/dist/index.js:1823:25)
    at ExecState.CheckComplete (/home/runner/work/_actions/PyO3/maturin-action/aef21716ff3dcae8a1c301d23ec3e4446972a6e3/dist/index.js:1806:18)
    at ChildProcess.<anonymous> (/home/runner/work/_actions/PyO3/maturin-action/aef21716ff3dcae8a1c301d23ec3e4446972a6e3/dist/index.js:1700:27)
    at ChildProcess.emit (node:events:524:28)
    at maybeClose (node:internal/child_process:1104:16)
    at ChildProcess._handle.onexit (node:internal/child_process:304:5)

• Logs
• Workflow