Running clang-tidy with compile command database containing multiple entries for the same source file

[adambadura]

According to the documentation clang-tidy can be run with -p command line argument described (at 22.0.0) as:

-p <build-path> is used to read a compile command database.

  For example, it can be a CMake build directory in which a file named
  compile_commands.json exists (use -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
  CMake option to get this output). When no build path is specified,
  a search for compile_commands.json will be attempted through all
  parent paths of the first input file . See:
  https://clang.llvm.org/docs/HowToSetupToolingForLLVM.html for an
  example of setting up Clang Tooling on a source tree.

The "compile command database" may contain multiple entries for the same source file. It is even documented in the JSON Compilation Database Format Specification (again, at 22.0.0):

• file: The main translation unit source processed by this compilation step. This is used by tools as the key into the compilation database. There can be multiple command objects for the same file, for example if the same source file is compiled with different configurations.

Although, when reading it now I wonder what would it mean that it "is used by tools as the key into the compilation database" when it can show up more than once. And then whether the "There can be multiple command objects for the same file" means multiple entries in the database or multiple command elements (array? same for arguments?) within single entry?

Either way, in my case I have a CMake-based project and some source files show up multiple times, with different compilation options. So, they should be checked multiple times as each such check might yield different issues.

Yet, I see no way to either make clang-tidy pick a specific entry nor to even see which entry it picked. My wild guess would be that it picks the first entry, but results seem to indicate it picks the last perhaps?

For now what I though of was to either don't use -p and instead provide the compile options directly or to split the compile_commands.json file into each entry separately and then feed clang-tidy with such mono-entry files instead, to have full control over which options get used.

[llvmbot]

@llvm/issue-subscribers-clang-tidy

Author: Adam Badura (adambadura)

According to the [documentation](https://clang.llvm.org/extra/clang-tidy/) `clang-tidy` can be run with `-p` command line argument described (at 22.0.0) as: > ```plain > -p <build-path> is used to read a compile command database. > > For example, it can be a CMake build directory in which a file named > compile_commands.json exists (use -DCMAKE_EXPORT_COMPILE_COMMANDS=ON > CMake option to get this output). When no build path is specified, > a search for compile_commands.json will be attempted through all > parent paths of the first input file . See: > https://clang.llvm.org/docs/HowToSetupToolingForLLVM.html for an > example of setting up Clang Tooling on a source tree. > ```

The "compile command database" may contain multiple entries for the same source file. It is even documented in the JSON Compilation Database Format Specification (again, at 22.0.0):
> - file: The main translation unit source processed by this compilation step. This is used by tools as the key into the compilation database. There can be multiple command objects for the same file, for example if the same source file is compiled with different configurations.

Although, when reading it now I wonder what would it mean that it "is used by tools as the key into the compilation database" when it can show up more than once. And then whether the "There can be multiple command objects for the same file" means multiple entries in the database or multiple command elements (array? same for arguments?) within single entry?

Either way, in my case I have a CMake-based project and some source files show up multiple times, with different compilation options. So, they should be checked multiple times as each such check might yield different issues.

Yet, I see no way to either make clang-tidy pick a specific entry nor to even see which entry it picked. My wild guess would be that it picks the first entry, but results seem to indicate it picks the last perhaps?

For now what I though of was to either don't use -p and instead provide the compile options directly or to split the compile_commands.json file into each entry separately and then feed clang-tidy with such mono-entry files instead, to have full control over which options get used.

[adambadura]

For now what I though of was to either don't use -p and instead provide the compile options directly or to split the compile_commands.json file into each entry separately and then feed clang-tidy with such mono-entry files instead, to have full control over which options get used.

The second workaround idea here, manually splitting the compile_commands.json, calls for another improvement. It would be nice if -p accepted not only a directory (containing the compile_commands.json) but also a full file path, assuming then that the file contains the "compile commands database", regardless of how it is named.

Otherwise for now I need to split the compile_commands.json into separate directories (so that they can all be still named compile_commands.json). And this in turn raises question if relative paths there would require any handling.

[EugeneZelenko]

Related to #169168?

[adambadura]

@EugeneZelenko, seems related, but I don't think the fix to #169168 will help here.

[adambadura]

My wild guess would be that it picks the first entry, but results seem to indicate it picks the last perhaps?

I have made an experiment (with LLVM version 19.1.7) and it seems my initial description was not correct. It seems clang-tidy checks all matching entries rather than just the first one (or any other arbitrary one).

For the experiment I used the following PoC.cpp:

#if defined(MODE_1)
#define POC_1(x) x
#endif

#if defined(MODE_2)
#define POC_2(x) x
#endif

#if defined(MODE_3)
#define POC_3(x) x
#endif

#if defined(MODE_4)
#define POC_4(x) x
#endif

and then in CMake:

add_library(poc_mode_1 STATIC PoC.cpp)
target_compile_definitions(poc_mode_1 PRIVATE MODE_1)

add_library(poc_mode_2 STATIC PoC.cpp)
target_compile_definitions(poc_mode_2 PRIVATE MODE_2)

add_library(poc_mode_3 STATIC PoC.cpp)
target_compile_definitions(poc_mode_3 PRIVATE MODE_3)

which in compile_commands.json produced three entries for same "file" (and "directory") but different "command" and "output".

The output of clang-tidy indicated three issues of clang-diagnostic-unused-macros for all three #define POC_... cases. Note, the POC_4 case is added to ensure the issue is not reported by mere presence of #define or similar.

So, the problem is less severe than I thought. However, it would still be nice to be able to tell precisely which entry is supposed to be used. Especially that with the current approach what seems to be checking a single file might in fact be checking multiple entries and parallelization is made worse this way.

[vbvictor]

Related: #148819, #148814.

clang-tidy (and other LibTooling tools) runs over all files in the database, and there is no way to tell which files to choose without manual preprocessing.

Any suggestions/efforts on fixing it are appreciated.