Add `di-sanitizer` feature, allowing to disable DI sanitization by vadorovsky · Pull Request #320 · aya-rs/bpf-linker

vadorovsky · 2025-11-05T12:46:56Z

LLVM 21.1.5 contains fixes that emit correct BTF[0] without the need to sanitize it, except from the name fixup that:

Is going to be fixed up in Aya:
- aya-obj: Sanitize type names aya#1382
Should be fixed in the kernel.

Introduce the di-sanitizer feature, that explicitly activates the use of DISanitizer. Enable it by default for now, but run Aya integration tests without it.

[0] llvm/llvm-project#165154

This change is

tamird

@tamird reviewed 2 of 3 files at r1, 7 of 7 files at r2, 9 of 14 files at r3, all commit messages.
Reviewable status: 13 of 18 files reviewed, 1 unresolved discussion

-- commits line 9 at r1:
are you going to send that patch? 🍿

tests/sanitizer/btf/assembly/auxiliary/loop-panic-handler.rs line 1 at r3 (raw file):

// no-prefer-dynamic

can these various files that are copies be symlinks instead?

vadorovsky · 2025-11-06T06:59:56Z

Something fishy is going on with apt packages. The job that uses LLVM 21.1.5 from packages:

https://github.com/aya-rs/bpf-linker/actions/runs/19110300914/job/54605380603?pr=320

Get:1 https://apt.llvm.org/jammy llvm-toolchain-jammy-21/main amd64 libllvm21 amd64 1:21.1.5~++20251023083201+45afac62e373-1~exp1~20251023083316.53 [31.1 MB]
Get:2 https://apt.llvm.org/jammy llvm-toolchain-jammy-21/main amd64 libclang-cpp21 amd64 1:21.1.5~++20251023083201+45afac62e373-1~exp1~20251023083316.53 [14.0 MB]
Get:3 https://apt.llvm.org/jammy llvm-toolchain-jammy-21/main amd64 libpolly-21-dev amd64 1:21.1.5~++20251023083201+45afac62e373-1~exp1~20251023083316.53 [1854 kB]
Get:4 https://apt.llvm.org/jammy llvm-toolchain-jammy-21/main amd64 llvm-21-runtime amd64 1:21.1.5~++20251023083201+45afac62e373-1~exp1~20251023083316.53 [645 kB]
Get:5 https://apt.llvm.org/jammy llvm-toolchain-jammy-21/main amd64 llvm-21-linker-tools amd64 1:21.1.5~++20251023083201+45afac62e373-1~exp1~20251023083316.53 [1425 kB]
Get:6 https://apt.llvm.org/jammy llvm-toolchain-jammy-21/main amd64 llvm-21 amd64 1:21.1.5~++20251023083201+45afac62e373-1~exp1~20251023083316.53 [20.3 MB]
Get:7 https://apt.llvm.org/jammy llvm-toolchain-jammy-21/main amd64 llvm-21-tools amd64 1:21.1.5~++20251023083201+45afac62e373-1~exp1~20251023083316.53 [631 kB]
Get:8 https://apt.llvm.org/jammy llvm-toolchain-jammy-21/main amd64 llvm-21-dev amd64 1:21.1.5~++20251023083201+45afac62e373-1~exp1~20251023083316.53 [53.1 MB]

fails with:

failures:

---- tests::sk_storage::sk_storage_connect stdout ----
[DEBUG aya_obj::btf::btf] changing FUNC memcpy linkage to BTF_FUNC_STATIC
[DEBUG aya_obj::btf::btf] changing FUNC memmove linkage to BTF_FUNC_STATIC
[DEBUG aya_obj::btf::btf] changing FUNC memset linkage to BTF_FUNC_STATIC
[DEBUG aya_obj::btf::btf] changing FUNC memcmp linkage to BTF_FUNC_STATIC
[DEBUG aya_obj::btf::btf] [DATASEC] .maps: fixup size to 40
[DEBUG aya_obj::btf::btf] [DATASEC] .maps: VAR SOCKET_STORAGE: fixup offset 0
[WARN  aya::bpf] object BTF couldn't be loaded in the kernel: the BPF_BTF_LOAD syscall returned Invalid argument (os error 22). Verifier output: magic: 0xeb9f
    version: 1
    flags: 0x0
    hdr_len: 24
    type_off: 0
    type_len: 1120
    str_off: 1120
    str_len: 3738
    btf_total_size: 4882
    [1] PTR (anon) type_id=3
    [2] INT i32 size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
    [3] ARRAY (anon) type_id=2 index_type_id=4 nr_elems=24
    [4] INT __ARRAY_SIZE_TYPE__ size=4 bits_offset=0 nr_bits=32 encoding=(none)
    [5] PTR (anon) type_id=2
    [6] PTR (anon) type_id=7
    [7] STRUCT Value size=40 vlen=6
    	user_family type_id=8 bits_offset=0
    	user_ip type_id=9 bits_offset=32
    	user_port type_id=8 bits_offset=192
    	family type_id=8 bits_offset=224
    	type_ type_id=8 bits_offset=256
    	protocol type_id=8 bits_offset=288
    [8] INT u32 size=4 bits_offset=0 nr_bits=32 encoding=(none)
    [9] STRUCT Ip size=20 vlen=1
    	(anon) type_id=0 bits_offset=0 Invalid type_id
    

thread 'tests::sk_storage::sk_storage_connect' (27146) panicked at test/integration-test/src/tests/sk_storage.rs:17:62:
called `Result::unwrap()` on an `Err` value: MapError(CreateError { name: "SOCKET_STORAGE", io_error: Os { code: 22, kind: InvalidInput, message: "Invalid argument" } })
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: integration_test::tests::sk_storage::sk_storage_connect
   4: core::ops::function::FnOnce::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Given that the "source" job uses our fork https://github.com/aya-rs/llvm-project, which currently is LLVM 21.1.3 with my patches applied, I though there is some regression in 21.1.5, but... I'm not able to reproduce that with LLVM built from source from the upstream tag https://github.com/llvm/llvm-project/releases/tag/llvmorg-21.1.5. And the BTF of Ip with it looks like:

#9: <STRUCT> 'Ip' sz:20 n:1
        #00 '<anon>' off:0 --> [10]
#10: <UNION> '<anon>' sz:20 n:3
        #00 '<anon>' off:0 --> [8]
        #01 'V4' off:0 --> [11]
        #02 'V6' off:0 --> [12]
#11: <STRUCT> 'V4' sz:20 n:1
        #00 '__0' off:32 --> [8]
#12: <STRUCT> 'V6' sz:20 n:1
        #00 '__0' off:32 --> [13]
#13: <ARRAY> n:4 idx-->[4] val-->[8]

I was thinking about proposing an upgrade to LLVM 21.1.5 to the Rust fork of LLVM https://github.com/rust-lang/llvm-project/tree/rustc/21.1-2025-08-01, but I will do that after solving that mystery.

vadorovsky · 2025-11-06T16:34:32Z

OK, mystery solved. The current "21.1.5" deb packages from apt.llvm.org are not made from the actual 21.1.5 release (that was cut 2 days ago), but from some git snapshot of the release/21.x branch that was made on Oct 23.

https://apt.llvm.org/jammy/pool/main/l/llvm-toolchain-21/

🤡

I will re-enable fixups for the apt variant and remove them once packages are fresh enough to actually contain my fixes.

tamird · 2025-11-06T16:40:53Z

Jeez that's frustrating behavior from the LLVM apt maintainers.

vadorovsky

Reviewable status: 4 of 19 files reviewed, 1 unresolved discussion (waiting on @tamird)

-- commits line 9 at r1:

Previously, tamird (Tamir Duberstein) wrote…

are you going to send that patch? 🍿

Yes, probably some time this week. 😅

tests/sanitizer/btf/assembly/auxiliary/loop-panic-handler.rs line 1 at r3 (raw file):

Previously, tamird (Tamir Duberstein) wrote…

can these various files that are copies be symlinks instead?

Yes, done

tamird

@tamird reviewed 14 of 14 files at r4, 14 of 14 files at r6, 1 of 1 files at r7, 14 of 14 files at r8, all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @vadorovsky)

It was never necessary in the first place. There is no place in the kernel that enforces BTF map structs themselves to be anonymous. Only pointer types (that are members of BTF maps) have to be anonymous, but that's handled by a separate fixup. BTF maps work just fine without it.

`assembly/auxiliary/loop-panic-handler.rs` and `btf/assembly/auxiliary/loop-panic-handler.rs` had exactly the same content. Instead of keeping copies, make a symlink.

LLVM >=21.1.5 supports them thanks to llvm/llvm-project#155783.

Starting from LLVM 21.1.5, the only necessary fixup is the making sure that the type names are C-compatible and don't contain characters like `<`, `>`. That should be fixed in the kernel. Phasing out this behavior will take time and effort, but ability to disable fixups will make the development efforts easier.

tamird

@tamird partially reviewed 18 files and all commit messages, and made 2 comments.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @vadorovsky).

src/llvm/di.rs line 89 at r12 (raw file):

                        }

                        // BPF backend in LLVM <21.1.5 does not support Rust enums.

drop a link to the commit? llvm/llvm-project@ad582e3

.github/workflows/ci.yml line 301 at r13 (raw file):

        if: runner.os == 'Linux' && matrix.toolchain.rust == 'nightly'
        with:
          # FIXME: Switch to upstream after https://github.com/aya-rs/aya/pull/1382

can you add to this comment explaining that that PR sanitizes the Rust names and that without either that or DI sanitization here in bpf-linker tests won't pass?

Using any of the available LLVM builds is not satisfactionary for us. apt.llvm.org is constantly broken (either does not work or lies about versions shipped by the packages[0]) and does not always catch up with the latest release candidates. Rust CI makes assumptions about which type of linking is used for each platform. It ships only static libraries for `*-musl` targets, and only dynamic ones for `*-apple-darwin` and `*-gnu` targets. We would like to use both types of linking for `*-musl` targets. We prefer dynamic linking for initial rounds of tests (especially the ones using `cargo hack`) to save space, but we also prefer static linking for producing the final binaries. We need to chase LLVM versions used in Rust nightly, which include release candidates as soon as they are out, so packaging provided by Linux distributions is moving too slow for us. Furthermore, building LLVM from source allows us to enable assertions for all LLVM versions, which makes it easier to find regressions. We are in favor of keeping them enabled even for the end users. [0] aya-rs#320 (comment)

vadorovsky mentioned this pull request Nov 5, 2025

aya-obj: Sanitize type names aya-rs/aya#1382

Draft

vadorovsky force-pushed the disable-di-sanitizer branch 5 times, most recently from 0ad0afb to 18fda8c Compare November 5, 2025 15:43

tamird reviewed Nov 5, 2025

View reviewed changes

vadorovsky force-pushed the disable-di-sanitizer branch from 18fda8c to da10a1e Compare November 5, 2025 17:15

vadorovsky force-pushed the disable-di-sanitizer branch 2 times, most recently from 3d8c710 to 2423a99 Compare November 9, 2025 16:13

vadorovsky commented Nov 9, 2025

View reviewed changes

vadorovsky marked this pull request as ready for review November 9, 2025 16:15

tamird approved these changes Nov 9, 2025

View reviewed changes

vadorovsky added 5 commits January 29, 2026 23:09

tests: Symlink loop-panic-handler.rs to btf/

76c2a2e

`assembly/auxiliary/loop-panic-handler.rs` and `btf/assembly/auxiliary/loop-panic-handler.rs` had exactly the same content. Instead of keeping copies, make a symlink.

di: Avoid single-arm match statements

f309447

di: Remove Rust (data-carrying) enums only for LLVM 20

9687430

LLVM >=21.1.5 supports them thanks to llvm/llvm-project#155783.

vadorovsky force-pushed the disable-di-sanitizer branch from 2423a99 to 2ed8455 Compare January 29, 2026 22:32

tamird approved these changes Jan 30, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `di-sanitizer` feature, allowing to disable DI sanitization#320

Add `di-sanitizer` feature, allowing to disable DI sanitization#320
vadorovsky wants to merge 5 commits intoaya-rs:mainfrom
vadorovsky:disable-di-sanitizer

vadorovsky commented Nov 5, 2025 •

edited by tamird

Loading

Uh oh!

tamird left a comment

Uh oh!

vadorovsky commented Nov 6, 2025 •

edited

Loading

Uh oh!

vadorovsky commented Nov 6, 2025 •

edited

Loading

Uh oh!

tamird commented Nov 6, 2025

Uh oh!

vadorovsky left a comment

Uh oh!

tamird left a comment

Uh oh!

tamird left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

vadorovsky commented Nov 5, 2025 • edited by tamird Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tamird left a comment

Choose a reason for hiding this comment

Uh oh!

vadorovsky commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vadorovsky commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tamird commented Nov 6, 2025

Uh oh!

vadorovsky left a comment

Choose a reason for hiding this comment

Uh oh!

tamird left a comment

Choose a reason for hiding this comment

Uh oh!

tamird left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vadorovsky commented Nov 5, 2025 •

edited by tamird

Loading

vadorovsky commented Nov 6, 2025 •

edited

Loading

vadorovsky commented Nov 6, 2025 •

edited

Loading