[compiler-rt] support build compiler-rt asan lit test case with libc++

[PikachuHyA]

As mentioned in #72108.
after we build compiler-rt test success with libc++ and patch #72111
we got another error about compiler-rt asan lit test cases ( e.g.contiguous_container.cpp)

FAIL: AddressSanitizer-x86_64-linux :: TestCases/contiguous_container.cpp (746 of 5776)
******************** TEST 'AddressSanitizer-x86_64-linux :: TestCases/contiguous_container.cpp' FAILED ********************
Exit Code: 1

Command Output (stderr):
--
RUN: at line 1: /llvm/build_compiler_rt_test_with_libcxx/./bin/clang  --driver-mode=g++ -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only   -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta   -fexceptions -O /llvm/compiler-rt/test/asan/TestCases/contiguous_container.cpp -o /llvm/build_compiler_rt_test_with_libcxx/runtimes/runtimes-bins/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Output/contiguous_container.cpp.tmp && env ASAN_OPTIONS=detect_stack_use_after_return=0  /llvm/build_compiler_rt_test_with_libcxx/runtimes/runtimes-bins/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Output/contiguous_container.cpp.tmp
+ /llvm/build_compiler_rt_test_with_libcxx/./bin/clang --driver-mode=g++ -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta -fexceptions -O /llvm/compiler-rt/test/asan/TestCases/contiguous_container.cpp -o /llvm/build_compiler_rt_test_with_libcxx/runtimes/runtimes-bins/compiler-rt/test/asan/X86_64LinuxConfig/TestCases/Output/contiguous_container.cpp.tmp
In file included from /llvm/compiler-rt/test/asan/TestCases/contiguous_container.cpp:5:
In file included from /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/algorithm:61:
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/bits/stl_algobase.h:1206:42: error: type '__gnu_cxx::__normal_iterator<const int *, std::vector<int>>' does not provide a call operator
 1206 |       while (__first1 != __last1 && bool(__binary_pred(*__first1, *__first2)))
      |                                          ^~~~~~~~~~~~~
/llvm/compiler-rt/test/asan/TestCases/contiguous_container.cpp:39:24: note: in instantiation of function template specialization 'std::mismatch<__gnu_cxx::__normal_iterator<const int *, std::vector<int>>, __gnu_cxx::__normal_iterator<const int *, std::vector<int>>, __gnu_cxx::__normal_iterator<const int *, std::vector<int>>>' requested here
   39 |   auto mismatch = std::mismatch(a.begin(), a.end(), b.begin(), b.end());
      |                        ^
1 error generated.

--

********************
FAIL: HWAddressSanitizer-x86_64 :: TestCases/new-test.cpp (1527 of 5776)
******************** TEST 'HWAddressSanitizer-x86_64 :: TestCases/new-test.cpp' FAILED ********************
Exit Code: 1

Command Output (stderr):
--
RUN: at line 2: /llvm/build_compiler_rt_test_with_libcxx/./bin/clang  --driver-mode=g++   -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta   -gline-tables-only -fsanitize=hwaddress -fuse-ld=lld -fsanitize-hwaddress-experimental-aliasing -mllvm -hwasan-generate-tags-with-calls=1 -mllvm -hwasan-globals -mllvm -hwasan-use-short-granules -mllvm -hwasan-instrument-landing-pads=0 -mllvm -hwasan-instrument-personality-functions -std=c++17 /llvm/compiler-rt/test/hwasan/TestCases/new-test.cpp -o /llvm/build_compiler_rt_test_with_libcxx/runtimes/runtimes-bins/compiler-rt/test/hwasan/X86_64/TestCases/Output/new-test.cpp.tmp -fsized-deallocation
+ /llvm/build_compiler_rt_test_with_libcxx/./bin/clang --driver-mode=g++ -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta -gline-tables-only -fsanitize=hwaddress -fuse-ld=lld -fsanitize-hwaddress-experimental-aliasing -mllvm -hwasan-generate-tags-with-calls=1 -mllvm -hwasan-globals -mllvm -hwasan-use-short-granules -mllvm -hwasan-instrument-landing-pads=0 -mllvm -hwasan-instrument-personality-functions -std=c++17 /llvm/compiler-rt/test/hwasan/TestCases/new-test.cpp -o /llvm/build_compiler_rt_test_with_libcxx/runtimes/runtimes-bins/compiler-rt/test/hwasan/X86_64/TestCases/Output/new-test.cpp.tmp -fsized-deallocation
/llvm/compiler-rt/test/hwasan/TestCases/new-test.cpp:36:44: error: no type named 'align_val_t' in namespace 'std'
   36 | void operator_new_delete(size_t size, std::align_val_t align) {
      |                                       ~~~~~^
/llvm/compiler-rt/test/hwasan/TestCases/new-test.cpp:50:50: error: no type named 'align_val_t' in namespace 'std'
   50 | void operator_new_delete_array(size_t size, std::align_val_t align) {
      |                                             ~~~~~^
/llvm/compiler-rt/test/hwasan/TestCases/new-test.cpp:78:44: error: no type named 'align_val_t' in namespace 'std'
   78 | void operator_new_delete(size_t size, std::align_val_t align, const std::nothrow_t &tag) {
      |                                       ~~~~~^
/llvm/compiler-rt/test/hwasan/TestCases/new-test.cpp:86:50: error: no type named 'align_val_t' in namespace 'std'
   86 | void operator_new_delete_array(size_t size, std::align_val_t align, const std::nothrow_t &tag) {
      |                                             ~~~~~^
4 errors generated.

--

The reason is that the gcc and libstdc++ versions are too old.

We tried to find relevant options to compile the lit test case using libc++, but failed.

Environment

centos 7

the latest LLVM cannot be built with gcc 4.8 on CentOS 7, then we use a prebuilt LLVM 17 with custom path (/opt/prebuilt_llvm).

reproducer

• centos 7

docker pull centos:7

• prebuilt llvm 17
  assume put in /opt/prebuilt_llvm
• llvm latest code (main branch)
  assume put in /llvm

set up build prerequisite

yum -y install gcc g++ gcc-c++ libatomic libstdc++-devel libstdc++-static binutils binutils-devel glibc glibc-static coreutils bash
yum -y install make dos2unix git chrpath perl-generators perl-Digest-MD5 libxml2-devel libffi-devel zlib-devel ncurses-devel libedit-devel perl-Sys-Syslog aone-yumtools perl-Data-Dumper perl-Encode elfutils-libelf-devel
yum -y install  python3 python3-pip python3-devel
yum -y install openssl-devel.x86_64

create build directory

cd /llvm
mkdir build_compiler_rt_test_with_libcxx
cd build_compiler_rt_test_with_libcxx

configure

LDFLAGS="-fuse-ld=lld -Wl,-q -Wl,-znow -Wl,-build-id=sha1" cmake -G Ninja \
    -DCMAKE_BUILD_TYPE="Release" \
    -DCMAKE_C_FLAGS="-Wno-backend-plugin" \
    -DCMAKE_CXX_FLAGS="-Wno-backend-plugin" \
    -DCMAKE_C_COMPILER="/opt/prebuilt_llvm/bin/clang" \
    -DCMAKE_CXX_COMPILER="/opt/prebuilt_llvm/bin/clang++" \
    -DLLVM_ENABLE_CLASSIC_FLANG=ON \
    -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld;openmp" \
    -DLLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi" \
    -DLLVM_LIBDIR_SUFFIX=64 \
    -DLLVM_USE_LINKER=lld \
    -DLLVM_PARALLEL_COMPILE_JOBS="96" \
    -DLLVM_PARALLEL_LINK_JOBS=96 \
    -DLLVM_ENABLE_NEW_PASS_MANAGER=ON \
    -DLLVM_TARGETS_TO_BUILD="X86;BPF;WebAssembly" \
    -DLLVM_ENABLE_LIBCXX=ON \
    -DLLVM_STATIC_LINK_CXX_STDLIB=ON \
    -DLLVM_BINUTILS_INCDIR="/usr/include" \
    -DLLVM_ENABLE_TERMINFO=OFF \
    -DLLVM_ENABLE_LIBXML2=OFF \
    -DLLVM_ENABLE_LIBEDIT=OFF \
    -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF \
    -DCOMPILER_RT_BUILD_LIBFUZZER=OFF \
    -DCOMPILER_RT_BUILD_XRAY=OFF \
    -DCOMPILER_RT_BUILD_ORC=OFF \
    -DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON \
    -DLIBCXX_ENABLE_STATIC_ABI_LIBRARY=OFF \
    -DOPENMP_ENABLE_LIBOMPTARGET=OFF \
    -DOPENMP_ENABLE_OMPT_TOOLS=OFF \
    -DOPENMP_ENABLE_TESTING=OFF \
    -DCOMPILER_RT_LIT_USE_LIBCXX=ON \
    -DCOMPILER_RT_CXX_LIBRARY=libcxx \
    -DSANITIZER_CXX_ABI=libc++ \
    -DSANITIZER_CXX_ABI_INTREE=ON \
    -DSANITIZER_USE_STATIC_CXX_ABI=ON \
    -DSANITIZER_TEST_CXX=libc++ \
    -DSANITIZER_TEST_CXX_INTREE=ON \
    -DSANITIZER_USE_STATIC_TEST_CXX=OFF \
    ../llvm

build

ninja
ninja check-compiler-rt

and errors occurred.

[ldionne]

FWIW I think compiler-rt should probably strive to have the same compiler requirements as libc++, that is Clang latest-2 and the latest GCC. Anything else could be considered unsupported, since in practice if you're building compiler-rt you are generally building your compiler from scratch (or have access to a recent one) anyways.

[PikachuHyA]

if you're building compiler-rt you are generally building your compiler from scratch

Yes, we build a distribution of LLVM. The "stage-1" compiler is built by Clang latest. However, the problem is that the gcc and libstdc++ is limited to version of 4.8.

compiler-rt should probably strive to have the same compiler requirements as libc++, that is Clang latest-2 and the latest GCC.

In theory, we should use the latest GCC, but in reality, this is quite difficult.
Using the latest GCC would cause the Clang/LLVM to introduce newer libstdc++ dependencies, which would then prevent Clang from functioning properly in our development/production environment(e.g. /lib64/libstdc++.so.6: version GLIBCXX_3.4.26 not found). This is because we have a large amount of code running on an older version of libstdc++ in our production environment and migrating to use newer GCC and libstdc++ is not feasible in a short period of time.

To address this issue, our idea is to replace all the dependencies on libstdc++ with libc++ by statically linking it. This requires all LLVM components to be compiled using libc++. Currently, we have successfully built the compiler-rt library, but we are unable to build unit tests and integration tests for it (due to use libstdc++). Therefore, we hope to find a way to compile the unit tests and integration tests using libc++.

[PikachuHyA]

FWIW I think compiler-rt should probably strive to have the same compiler requirements as libc++, that is Clang latest-2 and the latest GCC.

I test with GCC 13 and Clang 17 (#72108 (comment)) with patch #72111.
the lit test case works fine.

If it is possible to use libc++ to build the lit test case, it would be more convenient.