AddressSanitizer segfaults in gitlab CI docker containers with PIE and ASLR enabled (compiler-rt)

[hwulfmeyer]

When building C++ code with -fsanitize=address,undefined and running tests in debian12 Docker containers via our gitlab CI on a KVM-virtualized Ubuntu host, we experienced:

• Non-deterministic segmentation faults, sometimes during runtime, occasionally even during build
• Random crash locations, no real pattern in where crashes occurred
• Simple code crashes, even simple tests like ASSERT_TRUE(true) would segfault randomly like 1 out of 4 times

A very much NOT recommend option was to disable ASLR with sudo sysctl -w kernel.randomize_va_space=0.
Only do this temporary for debugging purposes for public machines as it makes them insecure. For private and local machines this might be okay.

Our final solution was to build and link our sanitizer tests with -no-pie.
Should we keep this or is there another solution?

I would very much wish that this should be clearly mentioned in the documentation to save other people 1 week of debugging time.
Or better that the leak sanitizer does not crash with a not descriptive and random segfault.
Or is this even a bug? Was this fixed in a newer version?

version info

Debian clang version 14.0.6
Target: x86_64-pc-linux-gnu
Thread model: posix

NOTE: this issue also occurred with GCC 12.2.0 libasan.

[hwulfmeyer]

I found this issue which might be related: #87754
We did the exact same, simple test program like in the issue and could reproduce it.
I will do further tests with newer versions of clang/gcc to see it its reproduceable for us.

Until then -no-pie seems like a valid solution if you need to use older versions of clang / gcc.

[hwulfmeyer]

Our tests was to run a sanitizer on a simple gtest with ASSERT_TRUE(true).
We ran the test 1000 times.

Error occurs in:

• Clang 14.0.6
• Clang 15.0.7
• Clang 16.0.6

Fixed in:

• Clang 17.0.6
• Clang 18.1.8
• Clang 19.1.7
• Clang 20.1.8

Building and linking with -fno-PIE like so:

-DCMAKE_CXX_FLAGS="-fsanitize=address,undefined -fno-omit-frame-pointer -fno-PIE"
-DCMAKE_EXE_LINKER_FLAGS="-fsanitize=address,undefined -no-pie"

works flawlessly in all clang versions and the error does not occur anymore.

[thurstond]

This was fixed in LLVM 17 onwards by fb77ca0