Option to reduce startup syscalls via environment caching

This commit provides the possibility to pass system library search
paths as well as some compiler include paths to ROOT as environment
variables.
This has the advantage that ROOT will spawn less sub-processes
and we can do the setup only once, instead of doing it for every
single executable that is linked to ROOT.

The commit does not change any default behaviour! Rather, expert-users may use
the new feature by moving the initialization of the search paths to say
software environment loading.

In ALICE, we do something like

```
export ROOT_LDSYSPATH=$(LD_DEBUG=libs LD_PRELOAD=DOESNOTEXIST ls /tmp/DOESNOTEXIST 2>&1 | grep -m 1 "system search path" | sed 's/.*=//g' | awk '//{print $1}')
export CLING_LDSYSPATH=ROOT_LDSYSPATH
export CLING_CPPSYSINCL=$(LC_ALL=C c++ -xc++ -E -v /dev/null 2>&1 | sed -n '/^.include/,${/^ \/.*++/{p}}' | tr '\n' ':' | tr ' ' ':')
```

speeding up the initialization of our executables at runtime and doing
less syscalls that create short-lived processes, for instance calling the compiler.

The effect from this operation can be seen by counting the `execve` syscalls
in a small example:

```
strace -e execve -f root.exe -q -e "double x=1;"  # ---> 14 calls

export ROOT_LDSYSPATH=...
export CLING_CPPSYSINCL=...
strace -e execve -f root.exe -q -e "double x=1;"  # ---> 6 calls
```

This gain can accumulate to significant savings when used in a multi-process environment
such as ALICE is using.

Author: sawenzel

README/ReleaseNotes/v638/index.md

@@ -31,7 +31,8 @@ The following people have contributed to this new version:
  Silia Taider, CERN/EP-SFT,\
  Florian Uhlig, GSI,\
  Devajith Valaparambil Sreeramaswamy, CERN/EP-SFT,\
- Vassil Vassilev, Princeton
+ Vassil Vassilev, Princeton,\
+ Sandro Wenzel, CERN/ALICE
 
 ## Deprecation and Removal
 
@@ -49,6 +50,13 @@ The following people have contributed to this new version:
 
 ## Core Libraries
 * Behavior change: when selecting a template instantiation for a dictionary, all the template arguments have to be fully defined - the forward declarations are not enough any more. The error prompted by the dictionary generator will be `Warning: Unused class rule: MyTemplate<MyFwdDeclaredClass>`.
+* New expert option to reduce static startup cost of ROOT by setting environment variables
+```bash
+export ROOT_LDSYSPATH=$(LD_DEBUG=libs LD_PRELOAD=DOESNOTEXIST ls /tmp/DOESNOTEXIST 2>&1 | grep -m 1 "system search path" | sed 's/.*=//g' | awk '//{print $1}')
+export CLING_LDSYSPATH=ROOT_LDSYSPATH
+export CLING_CPPSYSINCL=$(LC_ALL=C c++ -xc++ -E -v /dev/null 2>&1 | sed -n '/^.include/,${/^ \/.*++/{p}}' | tr '\n' ':' | tr ' ' ':')
+```
+This caching reduces sub-process creation during initialization and can be useful when multiple ROOT instances or binaries linked to ROOT are executed (less system-calls, cleaner debugging).
 
 ## I/O
 

core/unix/src/TUnixSystem.cxx

@@ -4599,6 +4599,12 @@ int TUnixSystem::UnixSend(int sock, const void *buffer, int length, int flag)
 
 ////////////////////////////////////////////////////////////////////////////////
 /// Get shared library search path. Static utility function.
+///
+/// The runtime cost (syscalls) of this function can be shortened by injecting
+/// a pre-calculated result for the system library search path in form of
+/// `export ROOT_LDSYSPATH=$(LD_DEBUG=libs LD_PRELOAD=DOESNOTEXIST ls /tmp/DOESNOTEXIST 2>&1 | grep -m 1 "system search
+/// path" | sed 's/.*=//g' | awk '//{print $1}')` This might be useful in scenarios, where ROOT is instantiated many
+/// times.
 
 static const char *DynamicPath(const char *newpath = nullptr, Bool_t reset = kFALSE)
 {
@@ -4684,23 +4690,34 @@ static const char *DynamicPath(const char *newpath = nullptr, Bool_t reset = kFA
       dynpath_syspart = "/usr/local/lib:/usr/X11R6/lib:/usr/lib:/lib:";
       dynpath_syspart += "/lib/x86_64-linux-gnu:/usr/local/lib64:/usr/lib64:/lib64:";
    #else
-      // trick to get the system search path
-      std::string cmd("LD_DEBUG=libs LD_PRELOAD=DOESNOTEXIST ls 2>&1");
-      FILE *pf = popen(cmd.c_str (), "r");
-      std::string result = "";
-      char buffer[128];
-      while (!feof(pf)) {
-         if (fgets(buffer, 128, pf) != nullptr)
-            result += buffer;
-      }
-      pclose(pf);
-      std::size_t from = result.find("search path=", result.find("(LD_LIBRARY_PATH)"));
-      std::size_t to = result.find("(system search path)");
-      if (from != std::string::npos && to != std::string::npos) {
-         from += 12;
-         std::string sys_path = result.substr(from, to-from);
-         sys_path.erase(std::remove_if(sys_path.begin(), sys_path.end(), isspace), sys_path.end());
-         dynpath_syspart = sys_path.c_str();
+      // Obtain the system search path ...
+
+      // First of all, let's see if an outside entity gave us the system path.
+      // This is a power-user feature for users bringing up many executables with ROOT linked in.
+      // In this case, the system path could be cached in an environment variable ROOT_LDSYSPATH and prevent
+      // repeated sys-calls (popen) and to having to search through potentially long LD_LIBRARY_PATH strings.
+      const auto ldsyspath = std::getenv("ROOT_LDSYSPATH");
+      if (ldsyspath != nullptr) {
+         dynpath_syspart = ldsyspath;
+      } else {
+         // trick to get the system search path at runtime (default case)
+         std::string cmd("LD_DEBUG=libs LD_PRELOAD=DOESNOTEXIST ls 2>&1");
+         FILE *pf = popen(cmd.c_str(), "r");
+         std::string result = "";
+         char buffer[128];
+         while (!feof(pf)) {
+            if (fgets(buffer, 128, pf) != nullptr)
+               result += buffer;
+         }
+         pclose(pf);
+         std::size_t from = result.find("search path=", result.find("(LD_LIBRARY_PATH)"));
+         std::size_t to = result.find("(system search path)");
+         if (from != std::string::npos && to != std::string::npos) {
+            from += 12;
+            std::string sys_path = result.substr(from, to - from);
+            sys_path.erase(std::remove_if(sys_path.begin(), sys_path.end(), isspace), sys_path.end());
+            dynpath_syspart = sys_path.c_str();
+         }
       }
       dynpath_envpart.ReplaceAll("::", ":");
       dynpath_syspart.ReplaceAll("::", ":");

interpreter/cling/lib/Interpreter/CIFactory.cpp

@@ -114,6 +114,23 @@ namespace {
                                        llvm::SmallVectorImpl<char>& Buf,
                                        AdditionalArgList& Args,
                                        bool Verbose) {
+    // For power-users: Let's see if the path is available as a predefined env
+    // variable to save repeated system calls when ROOT/CLING is initialized for
+    // each process in a many process system. CLING_CPPSYSINCL should contain
+    // paths separated by a ":" and otherwise contain the same paths as returned
+    // from the CppInclQuery further below.
+    auto PrefCppSystemIncl = std::getenv("CLING_CPPSYSINCL");
+    if (PrefCppSystemIncl != nullptr) {
+      llvm::StringRef PathsString(PrefCppSystemIncl);
+      llvm::SmallVector<StringRef, 10> Paths;
+      PathsString.split(Paths, ":");
+      for (auto& P : Paths) {
+        P = P.trim();
+        Args.addArgument("-cxx-isystem", P.str());
+      }
+      return;
+    }
+
     std::string CppInclQuery("LC_ALL=C ");
     CppInclQuery.append(Compiler);
 

interpreter/cling/lib/Utils/PlatformPosix.cpp

@@ -182,6 +182,19 @@ bool GetSystemLibraryPaths(llvm::SmallVectorImpl<std::string>& Paths) {
   Paths.push_back("/lib64/");
  #endif
 #else
+  // Power-user mode: See if the result of this query is cached/provided in env
+  // variable to avoid sys-calls and spawning processes
+  auto ldsyspath = std::getenv("CLING_LDSYSPATH");
+  if (ldsyspath != nullptr) {
+    std::string SysPath(ldsyspath);
+    llvm::SmallVector<llvm::StringRef, 10> CurPaths;
+    SplitPaths(SysPath, CurPaths);
+    for (const auto& Path : CurPaths) {
+      Paths.push_back(Path.str());
+    }
+    return true;
+  }
+
   llvm::SmallString<1024> Buf;
   platform::Popen("LD_DEBUG=libs LD_PRELOAD=DOESNOTEXIST ls", Buf, true);
   const llvm::StringRef Result = Buf.str();