Skip to content

Conversation

@phuang
Copy link
Contributor

@phuang phuang commented Dec 16, 2024

The problem in original change is because OHOS::getCompilerRT()
pickes a wrong builtin runtime ./lib/clang/20/lib/linux/libclang_rt.builtins-x86_64.a,
if ./lib/clang/20/lib/linux/libclang_rt.builtins-x86_64.a exist on the
test filesystem. It shouldn't happen with a clean build.

@llvmbot llvmbot added clang Clang issues not falling into any other category clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' labels Dec 16, 2024
@llvmbot
Copy link
Member

llvmbot commented Dec 16, 2024

@llvm/pr-subscribers-clang

@llvm/pr-subscribers-clang-driver

Author: Peng Huang (phuang)

Changes

The problem in original change is because OHOS::getCompilerRT()
pickes a wrong builtin runtime (./lib/clang/20/lib/linux/libclang_rt.builtins-x86_64.a),
if ./lib/clang/20/lib/linux/libclang_rt.builtins-x86_64.a does exist on
the test filesystem. Address the problem by adding a env suffix
"-ohos" into the runtime library file.


Full diff: https://github.com/llvm/llvm-project/pull/120159.diff

2 Files Affected:

  • (modified) clang/lib/Driver/ToolChain.cpp (+7-1)
  • (modified) clang/lib/Driver/ToolChains/OHOS.cpp (+26-34)
diff --git a/clang/lib/Driver/ToolChain.cpp b/clang/lib/Driver/ToolChain.cpp
index 9f174fbda398b5..f12a2744391e9a 100644
--- a/clang/lib/Driver/ToolChain.cpp
+++ b/clang/lib/Driver/ToolChain.cpp
@@ -745,7 +745,13 @@ std::string ToolChain::buildCompilerRTBasename(const llvm::opt::ArgList &Args,
   std::string ArchAndEnv;
   if (AddArch) {
     StringRef Arch = getArchNameForCompilerRTLib(*this, Args);
-    const char *Env = TT.isAndroid() ? "-android" : "";
+    const char *Env = NULL;
+    if (TT.isAndroid())
+      Env = "-android";
+    else if (TT.isOHOSFamily())
+      Env = "-ohos";
+    else
+      Env = "";
     ArchAndEnv = ("-" + Arch + Env).str();
   }
   return (Prefix + Twine("clang_rt.") + Component + ArchAndEnv + Suffix).str();
diff --git a/clang/lib/Driver/ToolChains/OHOS.cpp b/clang/lib/Driver/ToolChains/OHOS.cpp
index 6e1a09ae908b2f..c9a532771b99e5 100644
--- a/clang/lib/Driver/ToolChains/OHOS.cpp
+++ b/clang/lib/Driver/ToolChains/OHOS.cpp
@@ -19,8 +19,8 @@
 #include "llvm/ProfileData/InstrProf.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Path.h"
-#include "llvm/Support/VirtualFileSystem.h"
 #include "llvm/Support/ScopedPrinter.h"
+#include "llvm/Support/VirtualFileSystem.h"
 
 using namespace clang::driver;
 using namespace clang::driver::toolchains;
@@ -58,11 +58,9 @@ static bool findOHOSMuslMultilibs(const Driver &D,
   return false;
 }
 
-static bool findOHOSMultilibs(const Driver &D,
-                                      const ToolChain &TC,
-                                      const llvm::Triple &TargetTriple,
-                                      StringRef Path, const ArgList &Args,
-                                      DetectedMultilibs &Result) {
+static bool findOHOSMultilibs(const Driver &D, const ToolChain &TC,
+                              const llvm::Triple &TargetTriple, StringRef Path,
+                              const ArgList &Args, DetectedMultilibs &Result) {
   Multilib::flags_list Flags;
   bool IsA7 = false;
   if (const Arg *A = Args.getLastArg(options::OPT_mcpu_EQ))
@@ -172,8 +170,7 @@ OHOS::OHOS(const Driver &D, const llvm::Triple &Triple, const ArgList &Args)
       Paths);
 }
 
-ToolChain::RuntimeLibType OHOS::GetRuntimeLibType(
-    const ArgList &Args) const {
+ToolChain::RuntimeLibType OHOS::GetRuntimeLibType(const ArgList &Args) const {
   if (Arg *A = Args.getLastArg(clang::driver::options::OPT_rtlib_EQ)) {
     StringRef Value = A->getValue();
     if (Value != "compiler-rt")
@@ -184,20 +181,19 @@ ToolChain::RuntimeLibType OHOS::GetRuntimeLibType(
   return ToolChain::RLT_CompilerRT;
 }
 
-ToolChain::CXXStdlibType
-OHOS::GetCXXStdlibType(const ArgList &Args) const {
+ToolChain::CXXStdlibType OHOS::GetCXXStdlibType(const ArgList &Args) const {
   if (Arg *A = Args.getLastArg(options::OPT_stdlib_EQ)) {
     StringRef Value = A->getValue();
     if (Value != "libc++")
       getDriver().Diag(diag::err_drv_invalid_stdlib_name)
-        << A->getAsString(Args);
+          << A->getAsString(Args);
   }
 
   return ToolChain::CST_Libcxx;
 }
 
 void OHOS::AddClangSystemIncludeArgs(const ArgList &DriverArgs,
-                                        ArgStringList &CC1Args) const {
+                                     ArgStringList &CC1Args) const {
   const Driver &D = getDriver();
   const llvm::Triple &Triple = getTriple();
   std::string SysRoot = computeSysRoot();
@@ -258,7 +254,7 @@ void OHOS::AddClangCXXStdlibIncludeArgs(const ArgList &DriverArgs,
 }
 
 void OHOS::AddCXXStdlibLibArgs(const ArgList &Args,
-                                  ArgStringList &CmdArgs) const {
+                               ArgStringList &CmdArgs) const {
   switch (GetCXXStdlibType(Args)) {
   case ToolChain::CST_Libcxx:
     CmdArgs.push_back("-lc++");
@@ -291,7 +287,8 @@ ToolChain::path_list OHOS::getRuntimePaths() const {
 
   // First try the triple passed to driver as --target=<triple>.
   P.assign(D.ResourceDir);
-  llvm::sys::path::append(P, "lib", D.getTargetTriple(), SelectedMultilib.gccSuffix());
+  llvm::sys::path::append(P, "lib", D.getTargetTriple(),
+                          SelectedMultilib.gccSuffix());
   Paths.push_back(P.c_str());
 
   // Second try the normalized triple.
@@ -340,26 +337,20 @@ std::string OHOS::getDynamicLinker(const ArgList &Args) const {
 
 std::string OHOS::getCompilerRT(const ArgList &Args, StringRef Component,
                                 FileType Type) const {
+  std::string CRTBasename =
+      buildCompilerRTBasename(Args, Component, Type, /*AddArch=*/false);
+
   SmallString<128> Path(getDriver().ResourceDir);
   llvm::sys::path::append(Path, "lib", getMultiarchTriple(getTriple()),
-                          SelectedMultilib.gccSuffix());
-  const char *Prefix =
-      Type == ToolChain::FT_Object ? "" : "lib";
-  const char *Suffix;
-  switch (Type) {
-  case ToolChain::FT_Object:
-    Suffix = ".o";
-    break;
-  case ToolChain::FT_Static:
-    Suffix = ".a";
-    break;
-  case ToolChain::FT_Shared:
-    Suffix = ".so";
-    break;
-  }
-  llvm::sys::path::append(
-      Path, Prefix + Twine("clang_rt.") + Component + Suffix);
-  return static_cast<std::string>(Path.str());
+                          SelectedMultilib.gccSuffix(), CRTBasename);
+  if (getVFS().exists(Path))
+    return std::string(Path);
+
+  std::string NewPath = ToolChain::getCompilerRT(Args, Component, Type);
+  if (getVFS().exists(NewPath))
+    return NewPath;
+
+  return std::string(Path);
 }
 
 void OHOS::addExtraOpts(llvm::opt::ArgStringList &CmdArgs) const {
@@ -396,7 +387,7 @@ SanitizerMask OHOS::getSupportedSanitizers() const {
 
 // TODO: Make a base class for Linux and OHOS and move this there.
 void OHOS::addProfileRTLibs(const llvm::opt::ArgList &Args,
-                             llvm::opt::ArgStringList &CmdArgs) const {
+                            llvm::opt::ArgStringList &CmdArgs) const {
   // Add linker option -u__llvm_profile_runtime to cause runtime
   // initialization module to be linked in.
   if (needsProfileRT(Args))
@@ -413,7 +404,8 @@ ToolChain::path_list OHOS::getArchSpecificLibPaths() const {
   return Paths;
 }
 
-ToolChain::UnwindLibType OHOS::GetUnwindLibType(const llvm::opt::ArgList &Args) const {
+ToolChain::UnwindLibType
+OHOS::GetUnwindLibType(const llvm::opt::ArgList &Args) const {
   if (Args.getLastArg(options::OPT_unwindlib_EQ))
     return Generic_ELF::GetUnwindLibType(Args);
   return GetDefaultUnwindLibType();

@nico
Copy link
Contributor

nico commented Dec 17, 2024

IIRC there are two possible compiler-rt directory layouts. @MaskRay worked on moving many platforms from one to another. I think the newer layout doesn't use platform suffixes.

We shouldn't change where clang looks for things just for my bot. Thanks for figuring it out; I can delete the file from that bot. (What about developers that are in the same position though -- maybe it'd make sense to have a cmake action that deletes the file in the old location for a bit? Wouldn't help my bot since it doesn't use cmake, but might help others.)

We should figure out which path we want independently of my bot :) I don't know the current naming rules. Maybe @MaskRay remembers if we have documentation on this. Else, just relanding the patch as it was is fine.

Copy link
Member

@MaskRay MaskRay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without a test this should not land. #118192 (comment)

@phuang
Copy link
Contributor Author

phuang commented Dec 17, 2024

Without a test this should not land. #118192 (comment)

Hi @MaskRay could you please suggest how to test the changes? I though ohos.c should already cover it.

@phuang phuang requested a review from MaskRay December 18, 2024 18:53
@phuang
Copy link
Contributor Author

phuang commented Dec 19, 2024

@MaskRay friendly ping. thanks.

Copy link
Member

@MaskRay MaskRay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need a test to clang/test/Driver/ohos.c similar to linux-ld.c clang_rt.crtbegin.o.

You can add UNSUPPORTED: system-windows to ohos.c so that we don't need backslashes.

@phuang
Copy link
Contributor Author

phuang commented Dec 25, 2024

We need a test to clang/test/Driver/ohos.c similar to linux-ld.c clang_rt.crtbegin.o.

You can add UNSUPPORTED: system-windows to ohos.c so that we don't need backslashes.

ohos.c already has many tests for it. This PR changes behavior of function OHOS::getCompilerRT(). It will search extra folders for compiler rt builtin libraries. But in tests, OHOS::getCompilerRT()'s return value will not be changed, becasue those extra folders don't have those requested builtin files. It will just returns the default library file path.

@phuang
Copy link
Contributor Author

phuang commented Jan 1, 2025

Thanks for reviewing it. Could you please land it for me? Seems I cannot land it.

@MaskRay MaskRay merged commit bd154e8 into llvm:main Jan 1, 2025
8 checks passed
@llvm-ci
Copy link
Collaborator

llvm-ci commented Jan 1, 2025

LLVM Buildbot has detected a new failure on builder openmp-s390x-linux running on systemz-1 while building clang at step 6 "test-openmp".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/88/builds/6297

Here is the relevant piece of the build log for the reference
Step 6 (test-openmp) failure: test (failure)
******************** TEST 'libomp :: tasking/issue-94260-2.c' FAILED ********************
Exit Code: -11

Command Output (stdout):
--
# RUN: at line 1
/home/uweigand/sandbox/buildbot/openmp-s390x-linux/llvm.build/./bin/clang -fopenmp   -I /home/uweigand/sandbox/buildbot/openmp-s390x-linux/llvm.build/runtimes/runtimes-bins/openmp/runtime/src -I /home/uweigand/sandbox/buildbot/openmp-s390x-linux/llvm.src/openmp/runtime/test -L /home/uweigand/sandbox/buildbot/openmp-s390x-linux/llvm.build/runtimes/runtimes-bins/openmp/runtime/src  -fno-omit-frame-pointer -mbackchain -I /home/uweigand/sandbox/buildbot/openmp-s390x-linux/llvm.src/openmp/runtime/test/ompt /home/uweigand/sandbox/buildbot/openmp-s390x-linux/llvm.src/openmp/runtime/test/tasking/issue-94260-2.c -o /home/uweigand/sandbox/buildbot/openmp-s390x-linux/llvm.build/runtimes/runtimes-bins/openmp/runtime/test/tasking/Output/issue-94260-2.c.tmp -lm -latomic && /home/uweigand/sandbox/buildbot/openmp-s390x-linux/llvm.build/runtimes/runtimes-bins/openmp/runtime/test/tasking/Output/issue-94260-2.c.tmp
# executed command: /home/uweigand/sandbox/buildbot/openmp-s390x-linux/llvm.build/./bin/clang -fopenmp -I /home/uweigand/sandbox/buildbot/openmp-s390x-linux/llvm.build/runtimes/runtimes-bins/openmp/runtime/src -I /home/uweigand/sandbox/buildbot/openmp-s390x-linux/llvm.src/openmp/runtime/test -L /home/uweigand/sandbox/buildbot/openmp-s390x-linux/llvm.build/runtimes/runtimes-bins/openmp/runtime/src -fno-omit-frame-pointer -mbackchain -I /home/uweigand/sandbox/buildbot/openmp-s390x-linux/llvm.src/openmp/runtime/test/ompt /home/uweigand/sandbox/buildbot/openmp-s390x-linux/llvm.src/openmp/runtime/test/tasking/issue-94260-2.c -o /home/uweigand/sandbox/buildbot/openmp-s390x-linux/llvm.build/runtimes/runtimes-bins/openmp/runtime/test/tasking/Output/issue-94260-2.c.tmp -lm -latomic
# executed command: /home/uweigand/sandbox/buildbot/openmp-s390x-linux/llvm.build/runtimes/runtimes-bins/openmp/runtime/test/tasking/Output/issue-94260-2.c.tmp
# note: command had no output on stdout or stderr
# error: command failed with exit status: -11

--

********************


@phuang
Copy link
Contributor Author

phuang commented Jan 1, 2025

I checked the log. And I don't think the failed test is caused by this PR.

@phuang phuang deleted the reland_ohos branch January 2, 2025 01:47
@nico
Copy link
Contributor

nico commented Jan 2, 2025

I did remove the contents of out/gn/lib/clang/20/lib/linux, but ./lib/clang/20/lib/linux/libclang_rt.builtins-x86_64.a just reappeared on the next build.

I'm guessing this might break tests in cmake builds that use -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF (?)

http://45.33.8.238/linux/156390/step_6.txt

@phuang
Copy link
Contributor Author

phuang commented Jan 2, 2025

In that case, probably we have to add -ohos suffix in ToolChain::buildCompilerRTBasename() to fix this confliction. WDYT?

@nico
Copy link
Contributor

nico commented Jan 2, 2025

Here's a command you can use to reproduce this:

cmake -GNinja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_PROJECTS='clang;lld;compiler-rt' -DLLVM_APPEND_VC_REV=NO -DLLVM_TARGETS_TO_BUILD='X86' -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF ../llvm-project/llvm
$ time ninja check-clang

nico added a commit that referenced this pull request Jan 2, 2025
#120159)"

This reverts commit bd154e8.
Test fails with -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF, see
#120159 (comment)
@phuang
Copy link
Contributor Author

phuang commented Jan 2, 2025

Thanks for the repro command. I reproduced the problem and verified it can be fixed with adding -ohos suffix.

@phuang
Copy link
Contributor Author

phuang commented Jan 2, 2025

I created a PR to fix the issue. @nico could you please try it? thanks

@MaskRay
Copy link
Member

MaskRay commented Jan 3, 2025

This patch is reverted.

We can disable the test on -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF layouts, but I don't find a lit feature that.

The usual way is to create a compiler-rt file hierarchy under Inputs. Just fd libclang_rt.builtins.a for some examples.

#121484 , which is intrusive to the generic code, should not be allowed. Android is popular and has that hack for historical reasons. New platforms should not do that.

@phuang
Copy link
Contributor Author

phuang commented Jan 3, 2025

We can disable the test on -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF layouts, but I don't find a lit feature that.

It cannot work, becasue in the failed case, llvm runtime x86_64-unknown-linux-gnu target is built with -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF, but at same time runtime x86_64-unknown-linux-ohos is built with -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON. In this case, the ohos.c test case will be run, but it will failed becasuse runtime x86_64-unknown-linux-gnu will install the libclang_rt.builtin-x86_64.a with the old layout. It will cause the test failure. So adding a suffix -ohos can fix the problem. Or instead of reusing ToolChain::getCompilerRT() in OHOS, we can copy ToolChain::getCompilerRT() into OHOS::getCompilerRT() and not search libraries in the old layout.

@phuang
Copy link
Contributor Author

phuang commented Jan 3, 2025

I created a new PR , PTAL. Thanks

github-actions bot pushed a commit to arm/arm-toolchain that referenced this pull request Jan 10, 2025
…S (#118192)" (#120159)"

This reverts commit bd154e8.
Test fails with -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF, see
llvm/llvm-project#120159 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' clang Clang issues not falling into any other category

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants