Improve Bazel support to avoid legacy external links

[fzakaria]

The current lit tests find FileCheck using the legacy way for finding external files in your runfiles structure.

Most lit_tests use package_path to find the directory for FileCheck.

example:

expand_template(
    name = "lit_site_cfg_py",
    testonly = True,
    out = "lit.site.cfg.py",
    substitutions = {
        "@LIT_SITE_CFG_IN_HEADER@": LIT_SITE_CFG_IN_HEADER,
        "@LLVM_TOOLS_DIR@": package_path("//llvm:BUILD"),

This can create a lot of inodes that are unecessary, especially if you use rules_python which adds python as a runfile for hermeticness.

The recommended solution by Bazel right now is to do the two settings:

# Disabling runfiles links drastically increases performance in slow disk IO situations
# Do not build runfile trees by default. If an execution strategy relies on runfile
# symlink teee, the tree is created on-demand. See: https://github.com/bazelbuild/bazel/issues/6627
# and https://github.com/bazelbuild/bazel/commit/03246077f948f2790a83520e7dccc2625650e6df
build --nobuild_runfile_links
test --nobuild_runfile_links

 # https://bazel.build/reference/command-line-reference#flag--legacy_external_runfiles
 build --nolegacy_external_runfiles
 test --nolegacy_external_runfiles

In order to remove the legacy_external_runfiles however we have to change the mapping of LLVM_TOOLS_DIR.
The easy way to do this is as follows:

"@LLVM_TOOLS_DIR@": "../llvm-project/llvm",

The following wiki has a lot of good information: https://github.com/bazelbuild/bazel/wiki/Updating-the-runfiles-tree-structure

[keith]

++, i think this matters for remote cache performance as well, since you end up sending less info about runfiles up

[keith]

Bazel 8.x flipped the default on this, so it might be gone by 9.x #119462, we should fix sometime