Skip to content

Don't link with libdl on musl#195

Open
Bravo555 wants to merge 2 commits intonagisa:masterfrom
Bravo555:0.8.9
Open

Don't link with libdl on musl#195
Bravo555 wants to merge 2 commits intonagisa:masterfrom
Bravo555:0.8.9

Conversation

@Bravo555
Copy link

On musl dlopen is part of libc and there's no separate libdl.

I 'm using cryptoki crate, which uses libloading, and using cargo-zigbuild to compile for x86_64-unknown-linux-musl.
Without the change, I get the following error:

export 'CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_RUSTFLAGS=-Clink-self-contained=yes -Clinker=rust-lld'
+ CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_RUSTFLAGS='-Clink-self-contained=yes -Clinker=rust-lld'
+ MUSL_SYSROOT_DIR=/home/runner/.musl-cross/x86_64-unknown-linux-musl/x86_64-unknown-linux-musl/sysroot
+ export CFLAGS_x86_64_unknown_linux_musl=--sysroot=/home/runner/.musl-cross/x86_64-unknown-linux-musl/x86_64-unknown-linux-musl/sysroot
+ CFLAGS_x86_64_unknown_linux_musl=--sysroot=/home/runner/.musl-cross/x86_64-unknown-linux-musl/x86_64-unknown-linux-musl/sysroot
+ export BINDGEN_EXTRA_CLANG_ARGS=--sysroot=/home/runner/.musl-cross/x86_64-unknown-linux-musl/x86_64-unknown-linux-musl/sysroot
+ BINDGEN_EXTRA_CLANG_ARGS=--sysroot=/home/runner/.musl-cross/x86_64-unknown-linux-musl/x86_64-unknown-linux-musl/sysroot
+ target_lower=x86_64_unknown_linux_musl
+ '[' -n '' ']'
+ '[' -n 1 ']'
+ cc_var=CC_x86_64_unknown_linux_musl
+ declare -x CC_x86_64_unknown_linux_musl=clang-20
+ ar_var=AR_x86_64_unknown_linux_musl
+ declare -x AR_x86_64_unknown_linux_musl=llvm-ar-20
+ cargo +1.85 build --target=x86_64-unknown-linux-musl --release --bin tedge

...

error: linking with `rust-lld` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/runner/.rustup/toolchains/1.85-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/runner/.rustup/toolchains/1.85-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/self-contained:/home/runner/go/bin:/opt/hostedtoolcache/go/1.25.7/x64/bin:/snap/bin:/home/runner/.local/bin:/opt/pipx_bin:/home/runner/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" VSLANG="1033" "rust-lld" "-flavor" "gnu" "/home/runner/.rustup/toolchains/1.85-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/rcrt1.o" "/home/runner/.rustup/toolchains/1.85-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crti.o" "/home/runner/.rustup/toolchains/1.85-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtbeginS.o" "/tmp/rustcGufnkP/symbols.o" "<1 object files omitted>" "--as-needed" "-Bstatic" "/tmp/rustcGufnkP/{libpsm-cc7b4c60c1da627b.rlib,librquickjs_sys-49b63039f738dd1d.rlib}" "-ldl" "/tmp/rustcGufnkP/{libring-663d3b1d300a9115.rlib}" "-lunwind" "-lc" "/home/runner/.rustup/toolchains/1.85-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/{libcompiler_builtins-40918110fa23a9da.rlib}" "-Bdynamic" "--eh-frame-hdr" "-z" "noexecstack" "-L" "/home/runner/work/thin-edge.io/thin-edge.io/target/x86_64-unknown-linux-musl/release/build/ring-ef496e57d4f79bce/out" "-L" "/home/runner/work/thin-edge.io/thin-edge.io/target/x86_64-unknown-linux-musl/release/build/rquickjs-sys-5b5d11e0c34c9d4c/out" "-L" "/home/runner/work/thin-edge.io/thin-edge.io/target/x86_64-unknown-linux-musl/release/build/psm-3eb4793e05b003f8/out" "-L" "/home/runner/.rustup/toolchains/1.85-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained" "-L" "/home/runner/.rustup/toolchains/1.85-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib" "-o" "/home/runner/work/thin-edge.io/thin-edge.io/target/x86_64-unknown-linux-musl/release/deps/tedge-033e35ff27f7f158" "--gc-sections" "-static" "-pie" "--no-dynamic-linker" "-z" "text" "-z" "relro" "-z" "now" "--strip-all" "/home/runner/.rustup/toolchains/1.85-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtendS.o" "/home/runner/.rustup/toolchains/1.85-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtn.o"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: rust-lld: error: unable to find library -ldl
          

error: could not compile `tedge` (bin "tedge") due to 1 previous error
error: Recipe `release` failed on line 131 with exit code 101
Error: Process completed with exit code 101.

#[cfg_attr(
all(
any(target_os = "linux", target_os = "android"),
not(target_env = "musl")
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link attribute should still be specified, even if the symbols come from a library that's otherwise guaranteed to be linked in order to establish the correct link argument order during linking. See: freebsd/dragonfly below. The exception here is if these symbols get provided by the loader rather than libc, which is also common in bsd types.

Please double-check which approach musl uses.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally please add a test to gh actions workflow that tests this configuration.

@Bravo555
Copy link
Author

I realize now that my initial description lacks any context of what the output I pasted is - I didn't completely understand what the build process in the project i was working on was doing as I was not its author and only knew that changing this line made it build - so I took a bit of time to understand it more.

Now I know that the error is caused by my build process doing the wrong thing and not by libloading, but nevertheless the change could still be correct.

In short: The dlopen symbol is contained in musl's libc.so, not libdl.a, but musl provides libdl.a anyway as an empty stub 1 2. The current link attribute should not cause errors as during linking we include libc.so anyway, which is where dlopen comes from, but including libdl should be fine as it's provided (even if empty), so including it shouldn't cause an error.

Where dlopen comes from on musl

To confirm, I downloaded and built musl and checked dlopen is indeed in libc and not libdl:

$ wget https://musl.libc.org/releases/musl-1.2.5.tar.gz
$ tar xf musl-1.2.5.tar.gz 
$ cd musl-1.2.5 
$ ./configure
$ make
$ cd lib

$ nm libc.so | grep dlopen
0000000000025080 T dlopen

$ cat libdl.a
!<arch>

So the most precise link attribute setup should probably look something like:

#[cfg_attr(
    any(
        all(target_os = "linux", not(target_env = "musl")),
        target_os = "android"
    ),
    link(name = "dl")
)]
#[cfg_attr(all(target_os = "linux", target_env = "musl"), link(name = "c"))]
#[cfg_attr(any(target_os = "freebsd", target_os = "dragonfly"), link(name = "c"))]

But since linking with libdl also works, as musl people say in the mailing list link above, having it be present seems to be mandated by POSIX, then it's also fine to not make the change and close the PR. I'll leave to you decision whether to proceed with it.

The error

The error I've posted in the description happens when a binary which uses libloading is linked with
RUSTFLAGS="-Clinker=rust-lld" and x86_64-unknown-linux-musl target. It can be reproduced by creating a minimal program which uses libloading:

fn main() {
    unsafe {
        libloading::Library::new("libc.so.6").unwrap();
    }
}

and then compiling it with RUSTFLAGS="-Clinker=rust-lld" cargo run --target=x86_64-unknown-linux-musl:

error: linking with `rust-lld` failed: exit status: 1
  |
  = note:  "rust-lld" "-flavor" "gnu" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/rcrt1.o" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crti.o" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtbeginS.o" "/tmp/rustctldF84/symbols.o" "<17 object files omitted>" "--as-needed" "-Bstatic" "/home/marcel/dev/libloading-test/target/x86_64-unknown-linux-musl/debug/deps/liblibloading-c324c69f582b41b2.rlib" "-ldl" "/home/marcel/dev/libloading-test/target/x86_64-unknown-linux-musl/debug/deps/libcfg_if-19113c9e0f7f5b7a.rlib" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/{libstd-*,libpanic_unwind-*,libobject-*,libmemchr-*,libaddr2line-*,libgimli-*,libcfg_if-*,librustc_demangle-*,libstd_detect-*,libhashbrown-*,librustc_std_workspace_alloc-*,libminiz_oxide-*,libadler2-*,libunwind-*}.rlib" "-lunwind" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/liblibc-*.rlib" "-lc" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/{librustc_std_workspace_core-*,liballoc-*,libcore-*,libcompiler_builtins-*}.rlib" "-L" "/tmp/rustctldF84/raw-dylibs" "-Bdynamic" "--eh-frame-hdr" "-z" "noexecstack" "-L" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained" "-L" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib" "-o" "/home/marcel/dev/libloading-test/target/x86_64-unknown-linux-musl/debug/deps/libloading_test-726f482dd932e0ce" "--gc-sections" "-static" "-pie" "--no-dynamic-linker" "-z" "text" "-z" "relro" "-z" "now" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtendS.o" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtn.o"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: rust-lld: error: unable to find library -ldl

The error says that library -ldl can't be found, but as said above, musl provides libdl.a, so rust-lld should be able to find it. The root cause is that rust-lld doesn't have the paths to begin with, even when compiling for x86_64-unknown-linux-gnu:

$ RUSTFLAGS="-Clinker=rust-lld" cargo run --target=x86_64-unknown-linux-gnu
   Compiling cfg-if v1.0.4
   Compiling libloading v0.9.0
   Compiling libloading-test v0.1.0 (/home/marcel/dev/libloading-test)
error: linking with `rust-lld` failed: exit status: 1
  |
  = note:  "rust-lld" "-flavor" "gnu" "/tmp/rustcE2lZpX/symbols.o" "<17 object files omitted>" "--as-needed" "-Bstatic" "/home/marcel/dev/libloading-test/target/x86_64-unknown-linux-gnu/debug/deps/{liblibloading-fa4e639118f729d8,libcfg_if-81db6fe80b72171f}.rlib" "<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/lib/{libstd-*,libpanic_unwind-*,libobject-*,libmemchr-*,libaddr2line-*,libgimli-*,libcfg_if-*,librustc_demangle-*,libstd_detect-*,libhashbrown-*,librustc_std_workspace_alloc-*,libminiz_oxide-*,libadler2-*,libunwind-*,liblibc-*,librustc_std_workspace_core-*,liballoc-*,libcore-*,libcompiler_builtins-*}.rlib" "-Bdynamic" "-ldl" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-L" "/tmp/rustcE2lZpX/raw-dylibs" "--eh-frame-hdr" "-z" "noexecstack" "-L" "<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/marcel/dev/libloading-test/target/x86_64-unknown-linux-gnu/debug/deps/libloading_test-a83dd1a680c85d65" "--gc-sections" "-pie" "-z" "relro" "-z" "now"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: rust-lld: error: unable to find library -ldl
          rust-lld: error: unable to find library -lgcc_s
          rust-lld: error: unable to find library -lutil
          rust-lld: error: unable to find library -lrt
          rust-lld: error: unable to find library -lpthread
          rust-lld: error: unable to find library -lm
          rust-lld: error: unable to find library -ldl
          rust-lld: error: unable to find library -lc

I assume the reason for this is that rust-lld is just a linker, which doesn't know about these paths on its own, and should be invoked via a compiler (cc or clang), which provides correct paths.

So e.g. this works: RUSTFLAGS="-Clink-arg=-fuse-ld=lld" cargo run --target=x86_64-unknown-linux-gnu because -fuse-ld=lld argument is passed to cc which uses the linker and provides the paths, which can be confirmed by additionally including -Clink-arg=-v -Wlinker-messages in RUSTFLAGS:

$ RUSTFLAGS="-Clink-arg=-fuse-ld=lld -Clink-arg=-v -Wlinker-messages" cargo run --target=x86_64-unknown-linux-gnu
[...]
warning: linker stderr: Using built-in specs.
         COLLECT_GCC=cc
         COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-linux-gnu/15/lto-wrapper
         OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa
         OFFLOAD_TARGET_DEFAULT=1
         Target: x86_64-linux-gnu
[...]
/usr/libexec/gcc/x86_64-linux-gnu/15/collect2 [...] -fuse-ld=lld [...]

So I was just using it wrong and should've checked it before making the PR. Apologies!

In any case, thank you for your work with the library. I will fix things on my side and you can decide if it's worth it making this kind of small change for musl that doesn't change any behaviour.

@nagisa
Copy link
Owner

nagisa commented Feb 10, 2026

Hm, I'd rather not change anything in that case given that it has been working for a long time, but at the same time the suggested change in your comment above is actually providing the needed information most accurately, so if you want to adjust the PR to use that, it wouldn't be too terrible (and if it makes it link with plain rust-lld, that's just a side bonus.)

Do note that -fuse-ld=lld likely does not actually use the lld shipped with rustc (which can matter if you're e.g. doing some fancy LTO setup.)

Bravo555 added a commit to Bravo555/thin-edge.io that referenced this pull request Feb 11, 2026
Previously, -Clinker=rust-lld invoked the clang linker (lld) directly,
without providing paths for additional libraries like libdl.

-Clinker=clang invokes the clang compiler, which in turn invokes the
same clang linker (implicitly clang receives -fuse-ld=lld argument,
which is used to select the lld linker, which is the same as rust-lld),
but additionally providing the required paths.

This change is made to make crates that link to libraries other than
just libc, like libdl, build without issues. These libraries are
provided by musl libc, so we should be able to link with them properly.
Additional details in a [github comment][1].

[1]: nagisa/rust_libloading#195 (comment)

How the linker gets invoked can be observed by setting
RUSTFLAGS="-Clink-arg=-v -Wlinker-messages"

Signed-off-by: Marcel Guzik <marcel.guzik@cumulocity.com>
Bravo555 added a commit to Bravo555/thin-edge.io that referenced this pull request Feb 12, 2026
Provide --sysroot argument to rust-lld linker when compiling with musl,
such that crates that wish to link with libraries like libdl (which are
empty stubs but provided by musl) are able to do so.

This change is made to make crates that link to libraries other than
just libc, like libdl, build without issues. These libraries are
provided by musl libc, so we should be able to link with them properly.

Additional details in a [github comment][1].

[1]: nagisa/rust_libloading#195 (comment)

How the linker gets invoked can be observed by setting
RUSTFLAGS="-Clink-arg=-v -Wlinker-messages"

Signed-off-by: Marcel Guzik <marcel.guzik@cumulocity.com>
On musl dlopen is part of libc and there's no separate libdl.

Signed-off-by: Marcel Guzik <marcel.guzik@inetum.com>
@Bravo555
Copy link
Author

Fixed up the link attribute and included a test for the configuration (which should be failing without the link attribute change). Not completely happy though about having to create a somewhat useless example target just for this build test, but I'm not sure if there's a better approach.
So possibly should be good to merge now, unless you have any suggestions for any improvements.

Bravo555 added a commit to Bravo555/thin-edge.io that referenced this pull request Feb 13, 2026
Provide --sysroot argument to rust-lld linker when compiling with musl,
such that crates that wish to link with libraries like libdl (which are
empty stubs but provided by musl) are able to do so.

This change is made to make crates that link to libraries other than
just libc, like libdl, build without issues. These libraries are
provided by musl libc, so we should be able to link with them properly.

Additional details in a [github comment][1].

[1]: nagisa/rust_libloading#195 (comment)

How the linker gets invoked can be observed by setting
RUSTFLAGS="-Clink-arg=-v -Wlinker-messages"

Signed-off-by: Marcel Guzik <marcel.guzik@cumulocity.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants