Thread sanitizer fails on tests and segfaults #153145
Description
I'm running the tests of LLVM 20.1.8 on Sapphire Rapids CPUs on a Rocky 8.9 system and get many failures (probably 34 out of 42 failures according to a quick grep) that show
FATAL: ThreadSanitizer: memory layout is incompatible, even though ASLR is disabled.
They all look similar:
# note: command had no output on stdout or stderr
# executed command: env TSAN_OPTIONS=ignore_noninstrumented_modules=0:ignore_noninstrumented_modules=1 /build/runtimes/runtimes-bins/openmp/tools/archer/tests/critical/Output/lock.c.tmp
# note: command had no output on stdout or stderr
# error: command failed with exit status: 66
# executed command: tee /build/runtimes/runtimes-bins/openmp/tools/archer/tests/critical/Output/lock.c.tmp.log
# note: command had no output on stdout or stderr
# executed command: /build/./bin/FileCheck /source/LLVM/20.1.8/GCCcore-14.3.0/llvm-project-20.1.8.src/openmp/tools/archer/tests/critical/lock.c
# .---command stderr------------
# | /source/LLVM/20.1.8/GCCcore-14.3.0/llvm-project-20.1.8.src/openmp/tools/archer/tests/critical/lock.c:42:11: error: CHECK: expected string not found in input
# | // CHECK: DONE
# | ^
# | <stdin>:1:1: note: scanning from here
# | FATAL: ThreadSanitizer: memory layout is incompatible, even though ASLR is disabled.
# | ^
# | <stdin>:4:47: note: possible intended match here
# | 0x555555554000-0x5555555c3000 /build/runtimes/runtimes-bins/openmp/tools/archer/tests/critical/Output/lock.c.tmp
# | ^
# |
# | Input file: <stdin>
# | Check file: /source/LLVM/20.1.8/GCCcore-14.3.0/llvm-project-20.1.8.src/openmp/tools/archer/tests/critical/lock.c
# |
# | -dump-input=help explains the following input dump.
# |
# | Input was:
# | <<<<<<
# | 1: FATAL: ThreadSanitizer: memory layout is incompatible, even though ASLR is disabled.
# | check:42'0 X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
# | 2: Please file a bug.
# | check:42'0 ~~~~~~~~~~~~~~~~~~~
# | 3: ==3285126==Process memory map follows:
# | check:42'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 4: 0x555555554000-0x5555555c3000 /build/runtimes/runtimes-bins/openmp/tools/archer/tests/critical/Output/lock.c.tmp
# | check:42'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | check:42'1 ? possible intended match
# | 5: 0x5555555c3000-0x55555567a000 /build/runtimes/runtimes-bins/openmp/tools/archer/tests/critical/Output/lock.c.tmp
# | check:42'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 6: 0x55555567a000-0x55555567c000 /build/runtimes/runtimes-bins/openmp/tools/archer/tests/critical/Output/lock.c.tmp
# | check:42'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 7: 0x55555567c000-0x555555683000 /build/runtimes/runtimes-bins/openmp/tools/archer/tests/critical/Output/lock.c.tmp
# | check:42'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 8: 0x555555683000-0x5555569ef000 [heap]
# | check:42'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 9: 0x7ffff6e43000-0x7ffff6fff000 /usr/lib64/libc-2.28.so
# | check:42'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | .
# | .
# | .
# | >>>>>>
# `-----------------------------
# error: command failed with exit status: 1
When running them separately some pass and some even segfault:
$ env TSAN_OPTIONS=ignore_noninstrumented_modules=0:ignore_noninstrumented_modules=1 /source/LLVM/20.1.8/GCCcore-14.3.0/llvm.obj.3/runtimes/runtimes-bins/openmp/tools/archer/tests/task/Output/taskwait-depend.c.tmp
ThreadSanitizer:DEADLYSIGNAL
==682483==ERROR: ThreadSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7ffff3af903b bp 0x7ffffffe8660 sp 0x7ffffffe83d0 T682483)
==682483==The signal is caused by a READ memory access.
==682483==Hint: address points to the zero page.
#0 <null> <null> (libarcher.so+0x503b) (BuildId: 38322ef050e45ea05df9cf570d413ef76bd95059)
#1 __kmpc_omp_taskwait_deps_51 <null> (libomp.so+0xe1174)
#2 foo /source/LLVM/20.1.8/GCCcore-14.3.0/llvm-project-20.1.8.src/openmp/tools/archer/tests/task/taskwait-depend.c:38:1 (taskwait-depend.c.tmp+0x125a0f)
#3 main.omp_outlined_debug__ /source/LLVM/20.1.8/GCCcore-14.3.0/llvm-project-20.1.8.src/openmp/tools/archer/tests/task/taskwait-depend.c:50:3 (taskwait-depend.c.tmp+0x125bae)
#4 main.omp_outlined /source/LLVM/20.1.8/GCCcore-14.3.0/llvm-project-20.1.8.src/openmp/tools/archer/tests/task/taskwait-depend.c:48:1 (taskwait-depend.c.tmp+0x125bae)
#5 __kmp_invoke_microtask <null> (libomp.so+0xea2b8)
#6 __kmp_invoke_task_func kmp_runtime.cpp (libomp.so+0x6744e)
#7 __kmp_fork_call <null> (libomp.so+0x61b54)
#8 __kmpc_fork_call <null> (libomp.so+0x5166c)
#9 main /source/LLVM/20.1.8/GCCcore-14.3.0/llvm-project-20.1.8.src/openmp/tools/archer/tests/task/taskwait-depend.c:48:1 (taskwait-depend.c.tmp+0x125b6f)
#10 __libc_start_main <null> (libc.so.6+0x3ad84) (BuildId: 574d156ec0c828321a4038189fc1cfe74d0bb2ec)
#11 _start <null> (taskwait-depend.c.tmp+0x6f68d)
==682483==Register values:
rax = 0x0000000000000000 rbx = 0x0000000000000000 rcx = 0x0000000000000000 rdx = 0x0000000000000000
rdi = 0x0000726c00000910 rsi = 0x0000000000000008 rbp = 0x00007ffffffe8660 rsp = 0x00007ffffffe83d0
r8 = 0x0000000000000000 r9 = 0x00007ffffffe86a0 r10 = 0x00007ffff253b980 r11 = 0x0000000000000000
r12 = 0x00007ffff5d110d0 r13 = 0x00007ffff5d11040 r14 = 0x0000000000000000 r15 = 0x0000728000002d80
ThreadSanitizer can not provide additional info.
SUMMARY: ThreadSanitizer: SEGV (/lib64/libarcher.so+0x503b) (BuildId: 38322ef050e45ea05df9cf570d413ef76bd95059)
==682483==ABORTING
So this is quite hard to get more info. Any ideas what might cause this or how to debug and/or resolve it?