[MLIR] [macOS] Cannot build `check-mlir` on macOS

[Luohaothu]

With CMake config:

-DLLVM_ENABLE_PROJECTS="mlir;clang"
-DLLVM_BUILD_EXAMPLES=ON
-DLLVM_ENABLE_ASSERTIONS=ON
-DLLVM_CCACHE_BUILD=ON
-DLLVM_ENABLE_LLD=ON
-DLLVM_USE_SANITIZER="Address;Undefined"
-DMLIR_INCLUDE_INTEGRATION_TESTS=ON
-DLLVM_USE_SPLIT_DWARF=ON
-DBUILD_SHARED_LIBS=ON
-DCLANG_ENABLE_CIR=ON

and clang compiler on macOS, check-mlir got error message:

[804/805] Running the MLIR regression tests
llvm-lit: /Volumes/nvme/projects/llvm-project/llvm/utils/lit/lit/TestingConfig.py:156: fatal: unable to parse config file '/Volumes/nvme/projects/llvm-project/mlir/test/lit.cfg.py', traceback: Traceback (most recent call last):
  File "/Volumes/nvme/projects/llvm-project/llvm/utils/lit/lit/TestingConfig.py", line 144, in load_from_path
    exec(compile(data, path, "exec"), cfg_globals, None)
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/nvme/projects/llvm-project/mlir/test/lit.cfg.py", line 254, in <module>
    real_python_executable = find_real_python_interpreter()
  File "/Volumes/nvme/projects/llvm-project/mlir/test/lit.cfg.py", line 109, in find_real_python_interpreter
    copied_python = os.path.join(config.lldb_build_directory, "copied-python")
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'TestingConfig' object has no attribute 'lldb_build_directory'

Is this a bug of the script or some misconfig of my environment?

[Lewuathe]

With my macOS (Sonoma 14.6, M1 MAX), I can reproduce the same problem. Let me check what's going on.

[5408/5409] Running the MLIR regression tests
llvm-lit: /Users/sasaki/dev/llvm-project/llvm/utils/lit/lit/TestingConfig.py:156: fatal: unable to parse config file '/Users/sasaki/dev/llvm-project/mlir/test/lit.cfg.py', traceback: Traceback (most recent call last):
  File "/Users/sasaki/dev/llvm-project/llvm/utils/lit/lit/TestingConfig.py", line 144, in load_from_path
    exec(compile(data, path, "exec"), cfg_globals, None)
  File "/Users/sasaki/dev/llvm-project/mlir/test/lit.cfg.py", line 254, in <module>
    real_python_executable = find_real_python_interpreter()
  File "/Users/sasaki/dev/llvm-project/mlir/test/lit.cfg.py", line 109, in find_real_python_interpreter
    copied_python = os.path.join(config.lldb_build_directory, "copied-python")
AttributeError: 'TestingConfig' object has no attribute 'lldb_build_directory'

[Lewuathe]

When I added the following line in lit.site.cfg.py.in, the original error valinished.

config.lldb_build_directory = "@LLDB_TEST_BUILD_DIRECTORY@"

But the some MLIR specs failed due to some reasons...

[Lewuathe]

When we enable the address sanitizer, the error always happens.

-DLLVM_USE_SANITIZER="Address;Undefined"

[Luohaothu]

I've bisected the log and found the problematic commit:

 /Volumes/nvme/projects/llvm-project  ➦ beff2bacae76 <B> $ git bisect bad 
beff2bacae76fe25c86c5bf7bccb2cbef93de047 is the first bad commit
commit beff2bacae76fe25c86c5bf7bccb2cbef93de047
Author: Kasper Nielsen <kasper0406@gmail.com>
Date:   Fri Nov 22 16:34:03 2024 -0800

    [MLIR, Python] Make it easy to run tests with ASan on mac (#115524)
    
    There are two things that make running MLIR tests with ASan on a mac
    tedious:
    
    1. The `DYLD_INSERT_LIBRARIES` environment variable needs to be set to
    point to `libclang_rt.asan_osx_dynamic.dylib`
    2. Mac is wrapping Python, which means that the `DYLD_INSERT_LIBRARIES`
    environment variable is not being respected in the Python tests. The
    solution is to find and use a non-wrapped Python binary.
    
    With the above two changes, ASan works out of the box on mac's by
    setting the `-DLLVM_USE_SANITIZER=Address` cmake flag.
    
    I have stolen most of the code in this PR from other LLVM projects. It
    may be a good idea to reconcile it somewhere.

 mlir/test/get_darwin_real_python.py | 16 ++++++
 mlir/test/lit.cfg.py                | 99 +++++++++++++++++++++++++++++++++++--
 2 files changed, 111 insertions(+), 4 deletions(-)
 create mode 100644 mlir/test/get_darwin_real_python.py

Still trying to figure out what happened...

[Luohaothu]

@kasper0406 Hi Kasper, could you please help us here?