Fails to build on Jetson Orin Nano / Internal Compiler Error?

[robo-todd]

System Configuration:
rustc 1.90.0 (1159e78c4 2025-09-14)
Linux ubuntu 5.15.148-tegra #1 SMP PREEMPT Mon Jun 16 08:24:48 PDT 2025 aarch64 aarch64 aarch64 GNU/Linux
Ubuntu 22.04 (aarch64) platform

This is the standard JetPack 6.2 NVIDIA distribution for the platform.

Git branch main: commit 11db0ac

Build command:
cargo build --release --jobs 1

Expected result:
kornia-rs build

Error produced:

   Compiling kornia-3d v0.1.10-rc.6 (/home/team1073/Projects/kornia-rs/crates/kornia-3d)
error: rustc interrupted by SIGSEGV, printing backtrace

/home/team1073/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/librustc_driver-68c3d348eb8c0b4a.so(+0x3470324)[0xffffa1490324]
linux-vdso.so.1(__kernel_rt_sigreturn+0x0)[0xffffa270c7bc]
/home/team1073/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/libLLVM.so.20.1-rust-1.90.0-stable(_ZN4llvm21BranchProbabilityInfo9calculateERKNS_8FunctionERKNS_8LoopInfoEPKNS_17TargetLibraryInfoEPNS_13DominatorTreeEPNS_17PostDominatorTreeE+0x9c4)[0xffff9ae094b0]
/home/team1073/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/libLLVM.so.20.1-rust-1.90.0-stable(+0x46ed86c)[0xffff9adfd86c]
/home/team1073/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/libLLVM.so.20.1-rust-1.90.0-stable(_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE+0x9a8)[0xffff9ab6ec3c]
/home/team1073/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/libLLVM.so.20.1-rust-1.90.0-stable(_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE+0x3c)[0xffff9ab6e268]
/home/team1073/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/libLLVM.so.20.1-rust-1.90.0-stable(_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE+0x288)[0xffff9afbb6ac]
/home/team1073/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/librustc_driver-68c3d348eb8c0b4a.so(+0x2579504)[0xffffa0599504]
/home/team1073/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/librustc_driver-68c3d348eb8c0b4a.so(+0x2578fe0)[0xffffa0598fe0]
/home/team1073/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/librustc_driver-68c3d348eb8c0b4a.so(+0x2a87ec4)[0xffffa0aa7ec4]
/home/team1073/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/librustc_driver-68c3d348eb8c0b4a.so(_RINvNtNtCsdMqm6Kbgqcm_3std3sys9backtrace28___rust_begin_short_backtraceNCINvXs0_Cs3IvziIWkJAJ_18rustc_codegen_llvmNtB1g_18LlvmCodegenBackendNtNtNtCshpzNK4ui6Fj_17rustc_codegen_ssa6traits7backend19ExtraBackendMethods18spawn_named_threadNCINvNtNtB2k_4back5write10spawn_workB1O_E0uE0uEB1g_+0x2c8)[0xffffa0aa5b08]
/home/team1073/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/librustc_driver-68c3d348eb8c0b4a.so(+0x2a84ef0)[0xffffa0aa4ef0]
/home/team1073/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/librustc_driver-68c3d348eb8c0b4a.so(+0x423dbf0)[0xffffa225dbf0]
/lib/aarch64-linux-gnu/libc.so.6(+0x80398)[0xffff9de90398]
/lib/aarch64-linux-gnu/libc.so.6(+0xe9e9c)[0xffff9def9e9c]

note: we would appreciate a report at https://github.com/rust-lang/rust
help: you can increase rustc's stack size by setting RUST_MIN_STACK=16777216
error: could not compile `kornia-3d` (lib)

Caused by:
  process didn't exit successfully: `/home/team1073/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/bin/rustc --crate-name kornia_3d --edition=2021 crates/kornia-3d/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=79 --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --check-cfg 'cfg(docsrs,test)' --check-cfg 'cfg(feature, values())' -C metadata=c0463b6f0341d759 -C extra-filename=-c0b7dd88059638e7 --out-dir /home/team1073/Projects/kornia-rs/target/release/deps -C strip=debuginfo -L dependency=/home/team1073/Projects/kornia-rs/target/release/deps --extern bincode=/home/team1073/Projects/kornia-rs/target/release/deps/libbincode-dc886e762921239b.rmeta --extern faer=/home/team1073/Projects/kornia-rs/target/release/deps/libfaer-11ac4f3f8106e8e8.rmeta --extern serde=/home/team1073/Projects/kornia-rs/target/release/deps/libserde-aac3f8ce33be3178.rmeta --extern thiserror=/home/team1073/Projects/kornia-rs/target/release/deps/libthiserror-d876b909480bc5f8.rmeta -Ctarget-feature=+fp16,+fhm` (signal: 11, SIGSEGV: invalid memory reference)

[edgarriba]

@AS1100K did you find similar issues in the jetson?

[robo-todd]

I didn't know if this would be helpful to report to Rust compiler team, since there is no "minimal reproduction"... I mean the instructions to reproduce are short, but "build all of kornia-rs" seems to go beyond what their guidelines suggest as an acceptable compiler issue.

[AS1100K]

@AS1100K did you find similar issues in the jetson?

No, I didn't faced any issues. Have you tried doing cargo clean according to rust-lang/rust#133269

[robo-todd]

Yes I've run cargo clean and started over multiple times.

This morning I've been repeating the process to reproduce the problem while monitoring top to make sure we're not running out of memory or something. I set RUSTFLAGS="-C target-cpu=native" for this round with same results.

The build process (as far as I can tell from top) is not running out of memory and on the commit on main as of this morning (commit 4e983ee ) it still does not build due to internal compiler error. I noticed that it fails with the ICE in kornia-3d and kornia-icp. Some other packages build fine like kornia-linalg, etc.

Also (in case this is helpful).:
if I run cargo build --release --keep-going -j 1 it works its way through all the other dependencies so ~1200 out of the 1238 packages complete and then the build is blocked after this point.

I took the error message advice (within the possibilities of the system setup) and set:
export RUST_MIN_STACK=8000000 which would consume nearly all memory+swap on the system in the default configuration. Also still fails in kornia-icp and korina-3d. But the memory usage % on top does not point to memory exhaustion (at least not incremental memory exhaustion... perhaps one giant request for a huge amount of memory could still be failing).

Repeated same build on same system on same commit but with cargo +1.88.0 build --release --keep-going
and got the same failures.

error: rustc interrupted by SIGSEGV, printing backtrace

/home/todd/.rustup/toolchains/1.88.0-aarch64-unknown-linux-gnu/lib/librustc_driver-d1bcda88f32c0faa.so(+0x31bfe70)[0xffffa966fe70]
linux-vdso.so.1(__kernel_rt_sigreturn+0x0)[0xffffaa8e17bc]
/home/todd/.rustup/toolchains/1.88.0-aarch64-unknown-linux-gnu/lib/libLLVM.so.20.1-rust-1.88.0-stable(_ZN4llvm21BranchProbabilityInfo9calculateERKNS_8FunctionERKNS_8LoopInfoEPKNS_17TargetLibraryInfoEPNS_13DominatorTreeEPNS_17PostDominatorTreeE+0xbb4)[0xffffa3328638]
/home/todd/.rustup/toolchains/1.88.0-aarch64-unknown-linux-gnu/lib/libLLVM.so.20.1-rust-1.88.0-stable(+0x44dc520)[0xffffa312c520]
/home/todd/.rustup/toolchains/1.88.0-aarch64-unknown-linux-gnu/lib/libLLVM.so.20.1-rust-1.88.0-stable(_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE+0x9a8)[0xffffa30fb85c]
/home/todd/.rustup/toolchains/1.88.0-aarch64-unknown-linux-gnu/lib/libLLVM.so.20.1-rust-1.88.0-stable(_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE+0x3c)[0xffffa30fae88]
/home/todd/.rustup/toolchains/1.88.0-aarch64-unknown-linux-gnu/lib/libLLVM.so.20.1-rust-1.88.0-stable(_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE+0x288)[0xffffa31f8708]
/home/todd/.rustup/toolchains/1.88.0-aarch64-unknown-linux-gnu/lib/librustc_driver-d1bcda88f32c0faa.so(+0x25c48c4)[0xffffa8a748c4]
/home/todd/.rustup/toolchains/1.88.0-aarch64-unknown-linux-gnu/lib/librustc_driver-d1bcda88f32c0faa.so(+0x25c43b8)[0xffffa8a743b8]
/home/todd/.rustup/toolchains/1.88.0-aarch64-unknown-linux-gnu/lib/librustc_driver-d1bcda88f32c0faa.so(+0x2648f70)[0xffffa8af8f70]
/home/todd/.rustup/toolchains/1.88.0-aarch64-unknown-linux-gnu/lib/librustc_driver-d1bcda88f32c0faa.so(+0x2667cbc)[0xffffa8b17cbc]
/home/todd/.rustup/toolchains/1.88.0-aarch64-unknown-linux-gnu/lib/librustc_driver-d1bcda88f32c0faa.so(_RINvNtNtCsdh76HEVEXPq_3std3sys9backtrace28___rust_begin_short_backtraceNCINvXs0_CsbFTcdTdg4dK_18rustc_codegen_llvmNtB1g_18LlvmCodegenBackendNtNtNtCsbNKiJy6b52O_17rustc_codegen_ssa6traits7backend19ExtraBackendMethods18spawn_named_threadNCINvNtNtB2k_4back5write10spawn_workB1O_E0uE0uEB1g_+0x544)[0xffffa8b19090]
/home/todd/.rustup/toolchains/1.88.0-aarch64-unknown-linux-gnu/lib/librustc_driver-d1bcda88f32c0faa.so(+0x266b408)[0xffffa8b1b408]
/home/todd/.rustup/toolchains/1.88.0-aarch64-unknown-linux-gnu/lib/librustc_driver-d1bcda88f32c0faa.so(+0x3f8b0ec)[0xffffaa43b0ec]
/lib/aarch64-linux-gnu/libc.so.6(+0x80398)[0xffffa6320398]
/lib/aarch64-linux-gnu/libc.so.6(+0xe9e9c)[0xffffa6389e9c]

note: we would appreciate a report at https://github.com/rust-lang/rust
help: you can increase rustc's stack size by setting RUST_MIN_STACK=16000000
error: could not compile `kornia-icp` (lib)

[robo-todd]

What versions of Rust are known to work if it is working for you? I didn't find a "Minimum Supported Rust Version" in the README.

[robo-todd]

Trying again this appears to work on 1.87.0, so something introduced in 1.88 and remaining up through 1.90 combined with something in the korina-3d and korina-icp files really does not like each other. Hopefully this narrows down the issue?

[robo-todd]

Additional update: Fails to compile on nightly as well in a similar (but not identical) manner.

error: could not compile `kornia-3d` (lib)

Caused by:
  process didn't exit successfully: `/home/todd/.rustup/toolchains/nightly-aarch64-unknown-linux-gnu/bin/rustc --crate-name kornia_3d --edition=2021 crates/kornia-3d/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=80 --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --check-cfg 'cfg(docsrs,test)' --check-cfg 'cfg(feature, values())' -C metadata=fade85578d408a37 -C extra-filename=-90445010ddde3baa --out-dir /home/todd/Projects/kornia-rs/target/release/deps -C strip=debuginfo -L dependency=/home/todd/Projects/kornia-rs/target/release/deps --extern bincode=/home/todd/Projects/kornia-rs/target/release/deps/libbincode-de322aa8583af4a1.rmeta --extern faer=/home/todd/Projects/kornia-rs/target/release/deps/libfaer-736e0be4c32c1ce6.rmeta --extern serde=/home/todd/Projects/kornia-rs/target/release/deps/libserde-8120519d0e2f8bbe.rmeta --extern thiserror=/home/todd/Projects/kornia-rs/target/release/deps/libthiserror-15c9e0f4b72a5cb2.rmeta -C target-cpu=native` (signal: 5, SIGTRAP: trace/breakpoint trap)
warning: build failed, waiting for other jobs to finish...
error: could not compile `kornia-icp` (lib)

[AS1100K]

In rust 1.88 and 1.89 release note, I didn't find any significant changes that might cause the issue. But in rust 1.90 LLD is the default linker. Can you try disabling the lld linker.

I wasn't able to get my hands on jetson to reproduce the issue. I will try it once I get my hands on jetson. But imo the issue might be due to something outside the rust?

[robo-todd]

Thanks for the suggestion. The arm build of Rust seems to have a lot of differences after 1.87 or something.
I can't use options to disable the linker because they are unstable on this target specifically (apparently), so only nightly will allow me to try to disable the new linker.
I end up having to run something like this:

RUSTFLAGS="-Z unstable-options -C target-cpu=native -C linker-features=-lld" cargo +nightly build --release

However, this also fails in the same place/manner on kornia-icp / kornia-3d.

warning: build failed, waiting for other jobs to finish...
error: could not compile `kornia-icp` (lib)

Caused by:
  process didn't exit successfully: `/home/todd/.rustup/toolchains/nightly-aarch64-unknown-linux-gnu/bin/rustc --crate-name kornia_icp --edition=2021 crates/kornia-icp/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=80 --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --check-cfg 'cfg(docsrs,test)' --check-cfg 'cfg(feature, values())' -C metadata=6b080f855d9d86a0 -C extra-filename=-17f7323606e2d0e3 --out-dir /home/todd/Projects/kornia-rs/target/release/deps -C strip=debuginfo -L dependency=/home/todd/Projects/kornia-rs/target/release/deps --extern faer=/home/todd/Projects/kornia-rs/target/release/deps/libfaer-51afc4c9a13d279e.rmeta --extern kiddo=/home/todd/Projects/kornia-rs/target/release/deps/libkiddo-365b7b6fa4afb5db.rmeta --extern kornia_3d=/home/todd/Projects/kornia-rs/target/release/deps/libkornia_3d-eb407afbb37882f6.rmeta --extern log=/home/todd/Projects/kornia-rs/target/release/deps/liblog-9db707eed924e5f8.rmeta --extern thiserror=/home/todd/Projects/kornia-rs/target/release/deps/libthiserror-296cf29536729a14.rmeta -Z unstable-options -C target-cpu=native -C linker-features=-lld` (signal: 5, SIGTRAP: trace/breakpoint trap)

[robo-todd]

I'm just trying to figure out what is so special about kornia-icp and kornia-3d. Other kornia-rs subcrates compile just fine, but something in these two is different out of the ~1300 crates that build into this project only these are breaking. If we can crack this, hopefully this will contribute to the adoption of kornia-rs and growth of it, because the library looks great.

For folks who are authors on these crates, can you think of anything "unique" you're doing in there? I can try to look, but I don't think I would know what makes them so unusual. I'm not saying this is not a compiler bug (because it seems like it is), but how is it that zero other crates out of more than a thousand don't have an issue?

If we can come up with some idea, we might be able to put together a decent rust compiler bug with enough information for it to get solved, but I don't know the insides of kornia-rs well enough to think of the things that are different than other crates.

Suggestions?

[edgarriba]

if you see the deps of kornia-ipc are very few. I'd say if something is off, might be related to kiddo https://github.com/kornia/kornia-rs/blob/main/crates/kornia-icp/Cargo.toml#L18

[edgarriba]

i'm seeing now that there's a new version, worth to try

[robo-todd]

So I updated kiddo to 5.2.2 in Cargo.toml and made sure I ended up with it in Cargo.lock.
This fails in the same manner on kornia-icp and kornia-3d. This is back running the "current" 1.90 compiler.