[Modules][libc++] Can't use modules with precompiled headers and libc++

[Algorithm0]

I'm trying to build a project that uses C++20 modules and precompiled headers at the same time. When I use the LLVM libc++ library in such a scenario, I get a build error.
If I use GCC libstdc++ instead of LLVM libc++, the build goes correctly.
The build also passes correctly if I don't use precompiled headers (write #include inside source files).
Build log is attached as an attachment.
Also, here's a repo that cannot build with libc++.

build_log.txt

[llvmbot]

@llvm/issue-subscribers-clang-modules

Author: Sergey (Algorithm0)

I'm trying to build a project that uses C++20 modules and precompiled headers at the same time. When I use the LLVM libc++ library in such a scenario, I get a build error. If I use GCC libstdc++ instead of LLVM libc++, the build goes correctly. The build also passes correctly if I don't use precompiled headers (write `#include` inside source files).\ Build log is attached as an attachment.\ Also, [here's](https://github.com/Algorithm0/modules-and-pch-libcxx-bug) a repo that cannot build with libc++.

build_log.txt

[mordante]

There are some issues mixing includes and modules. However I'm not sure whether the combination of pch and modules is intended to work. @ChuanqiXu9 can you tell whether modules and pch's can be combined? If not, will this ever be added?

[koplas]

This should be fixed in trunk. @Algorithm0 Can you specify the version you tested?

[Algorithm0]

@koplas, you can check out the link in my repository for more details. But the latest version I tried is clang/libc++ 21.0.0 (commit 148111fdcf0e807fe74274b18fcf65c4cff45d63).

[ChuanqiXu9]

Yeah, I think @koplas had fixed this.

[koplas]

@Algorithm0 I can reproduce this issue with 53b4830. We had similar issues like #60358 and #58540.

I created a reproducer that no longer fails with trunk. While creating the reproducer, I noticed that wide char support needs to be enabled to fail. This support is automatically enabled with the LIBCXX_ENABLE_WIDE_CHARACTERS flag when building libc++. libstdc++ does not have this feature, which could be why it doesn't fail.

The next step is to check if this still fails with wide character support disabled, and what my reproducer is missing.

Yeah, I think @koplas had fixed this.

I did not create a fix for this issue, so I don't know which commit fixed #60358. I just noticed it was fixed while trying to debug it.

Update: It still fails with wide character support disabled. Can you simplify the reproducer you sent? It would be better if there is only one CMakeLists.txt and all headers are preprocessed. This would make it easier to create a test case for this issue and identify what is different from similar issues that are fixed.

[koplas]

Note: The reproducer does not use a uniform precompiled header, which could be why it fails. The next step is to create a reproducer with two modules that use different precompiled headers. It needs to be tested if this also occurs with header units.

@ChuanqiXu9 Are header units and precompiled headers implemented differently?

[ChuanqiXu9]

Note: The reproducer does not use a uniform precompiled header, which could be why it fails. The next step is to create a reproducer with two modules that use different precompiled headers. It needs to be tested if this also occurs with header units.

@ChuanqiXu9 Are header units and precompiled headers implemented differently?

header units are more similar with clang header modules. I don't have a lot insight here

[koplas]

Here is a reduced reproducer: https://github.com/koplas/clang-headerunits/tree/pch

// a.hpp
namespace std {
inline namespace __1 {
class reverse_iterator {};
inline constexpr reverse_iterator operator+(const reverse_iterator &__x) {
  return reverse_iterator();
}
}; // namespace __1
} // namespace std
namespace std {
inline namespace __1 {

template <class... _Args> class __promote {
  static double __test(int);

public:
  using type = decltype((__test(_Args()) + ...));
};
} // namespace __1
} // namespace std

// b.hpp
namespace std {
inline namespace __1 {

template <class... _Args> class __promote {

  static double __test(int);

public:
  using type = decltype((__test(_Args()) + ...));
};

} // namespace __1
} // namespace std

// some.cpp
export module some;

export namespace some {
void function() { using result = std::__promote<int>::type; }
} // namespace some

// main.cpp
import some;

int main() {
  using result = std::__promote<int>::type;
  some::function();
}

# build.sh

#!/bin/bash
DIR="$(
  cd -- "$(dirname "$0")" >/dev/null 2>&1
  pwd -P
)"

clang++ -stdlib=libc++ -std=c++20 $DIR/a.hpp -Xclang -emit-pch -o a.hpp.pch
clang++ -stdlib=libc++ -std=c++20 $DIR/b.hpp -Xclang -emit-pch -o b.hpp.pch
clang++ -Xclang -include-pch -Xclang a.hpp.pch -stdlib=libc++ -std=c++20 --precompile $DIR/some.cppm -o some.pcm
clang++ -Xclang -include-pch -Xclang b.hpp.pch -stdlib=libc++ -std=c++20 -fmodule-file=some=some.pcm -c $DIR/main.cpp -o main.o

Result:

In module 'some' imported from /home/paul/Documents/clang-headerunits/main.cpp:1:
/home/***/Documents/clang-headerunits/a.hpp:16:9: error: 'std::__promote::type' from module '' is not present in definition of '__promote<_Args...>' provided earlier
   16 |   using type = decltype((__test(_Args()) + ...));
      |         ^
/home/***/Documents/clang-headerunits/b.hpp:9:9: note: declaration of 'type' does not match
    9 |   using type = decltype((__test(_Args()) + ...));
      |         ^
1 error generated.

[koplas]

Here is the AST dump:

clang++ -Xclang -ast-dump -fsyntax-only a.hpp

TranslationUnitDecl 0x5dae9b264d28 <<invalid sloc>> <invalid sloc>
|-TypedefDecl 0x5dae9b2659b0 <<invalid sloc>> <invalid sloc> implicit __int128_t '__int128'
| `-BuiltinType 0x5dae9b2652f0 '__int128'
|-TypedefDecl 0x5dae9b265a20 <<invalid sloc>> <invalid sloc> implicit __uint128_t 'unsigned __int128'
| `-BuiltinType 0x5dae9b265310 'unsigned __int128'
|-TypedefDecl 0x5dae9b2b3640 <<invalid sloc>> <invalid sloc> implicit __NSConstantString '__NSConstantString_tag'
| `-RecordType 0x5dae9b265b10 '__NSConstantString_tag'
|   `-CXXRecord 0x5dae9b265a78 '__NSConstantString_tag'
|-TypedefDecl 0x5dae9b2655e8 <<invalid sloc>> <invalid sloc> implicit __builtin_ms_va_list 'char *'
| `-PointerType 0x5dae9b2655a0 'char *'
|   `-BuiltinType 0x5dae9b264dd0 'char'
|-TypedefDecl 0x5dae9b265940 <<invalid sloc>> <invalid sloc> implicit __builtin_va_list '__va_list_tag[1]'
| `-ConstantArrayType 0x5dae9b2658f0 '__va_list_tag[1]' 1
|   `-RecordType 0x5dae9b2656d0 '__va_list_tag'
|     `-CXXRecord 0x5dae9b265640 '__va_list_tag'
|-NamespaceDecl 0x5dae9b2b3698 <a.hpp:1:1, line:8:1> line:1:11 std
| |-NamespaceDecl 0x5dae9b2b3708 <line:2:1, line:7:1> line:2:18 __1 inline
| | |-CXXRecordDecl 0x5dae9b2b3778 <line:3:1, col:25> col:7 referenced class reverse_iterator definition
| | | |-DefinitionData pass_in_registers empty aggregate standard_layout trivially_copyable pod trivial literal has_constexpr_non_copy_move_ctor can_const_default_init
| | | | |-DefaultConstructor exists trivial constexpr defaulted_is_constexpr
| | | | |-CopyConstructor simple trivial has_const_param implicit_has_const_param
| | | | |-MoveConstructor exists simple trivial
| | | | |-CopyAssignment simple trivial has_const_param needs_implicit implicit_has_const_param
| | | | |-MoveAssignment exists simple trivial needs_implicit
| | | | `-Destructor simple irrelevant trivial
| | | |-CXXRecordDecl 0x5dae9b2b3898 <col:1, col:7> col:7 implicit class reverse_iterator
| | | |-CXXConstructorDecl 0x5dae9b2b3bc8 <col:7> col:7 implicit referenced constexpr reverse_iterator 'void () noexcept' inline default trivial
| | | |-CXXConstructorDecl 0x5dae9b2b3ce8 <col:7> col:7 implicit constexpr reverse_iterator 'void (const reverse_iterator &)' inline default trivial noexcept-unevaluated 0x5dae9b2b3ce8
| | | | `-ParmVarDecl 0x5dae9b2b3e20 <col:7> col:7 'const reverse_iterator &'
| | | |-CXXConstructorDecl 0x5dae9b2b3f00 <col:7> col:7 implicit constexpr reverse_iterator 'void (reverse_iterator &&)' inline default trivial noexcept-unevaluated 0x5dae9b2b3f00
| | | | `-ParmVarDecl 0x5dae9b2b4030 <col:7> col:7 'reverse_iterator &&'
| | | `-CXXDestructorDecl 0x5dae9b2b4260 <col:7> col:7 implicit referenced ~reverse_iterator 'void () noexcept' inline default trivial
| | `-FunctionDecl 0x5dae9b2b3af0 <line:4:1, line:6:1> line:4:35 constexpr operator+ 'reverse_iterator (const reverse_iterator &)' inline
| |   |-ParmVarDecl 0x5dae9b2b39e8 <col:45, col:69> col:69 __x 'const reverse_iterator &'
| |   `-CompoundStmt 0x5dae9b2b4368 <col:74, line:6:1>
| |     `-ReturnStmt 0x5dae9b2b4358 <line:5:3, col:27>
| |       `-CXXTemporaryObjectExpr 0x5dae9b2b4230 <col:10, col:27> 'reverse_iterator':'std::reverse_iterator' 'void () noexcept' zeroing
| `-EmptyDecl 0x5dae9b2b4380 <line:7:2> col:2
`-NamespaceDecl 0x5dae9b2b43a8 prev 0x5dae9b2b3698 <line:9:1, line:19:1> line:9:11 std
  |-original Namespace 0x5dae9b2b3698 'std'
  `-NamespaceDecl 0x5dae9b2b4418 prev 0x5dae9b2b3708 <line:10:1, line:18:1> line:10:18 __1 inline
    |-original Namespace 0x5dae9b2b3708 '__1'
    `-ClassTemplateDecl 0x5dae9b2d6b20 <line:12:1, line:17:1> line:12:33 __promote
      |-TemplateTypeParmDecl 0x5dae9b2b4488 <col:11, col:20> col:20 referenced class depth 0 index 0 ... _Args
      `-CXXRecordDecl 0x5dae9b2d6a90 <col:27, line:17:1> line:12:33 class __promote definition
        |-DefinitionData empty aggregate standard_layout trivially_copyable pod trivial literal has_constexpr_non_copy_move_ctor can_const_default_init
        | |-DefaultConstructor exists trivial constexpr needs_implicit defaulted_is_constexpr
        | |-CopyConstructor simple trivial has_const_param needs_implicit implicit_has_const_param
        | |-MoveConstructor exists simple trivial needs_implicit
        | |-CopyAssignment simple trivial has_const_param needs_implicit implicit_has_const_param
        | |-MoveAssignment exists simple trivial needs_implicit
        | `-Destructor simple irrelevant trivial needs_implicit
        |-CXXRecordDecl 0x5dae9b2d6e48 <col:27, col:33> col:33 implicit class __promote
        |-CXXMethodDecl 0x5dae9b2d6ff8 <line:13:3, col:27> col:17 referenced __test 'double (int)' static
        | `-ParmVarDecl 0x5dae9b2d6ef0 <col:24> col:27 'int'
        |-AccessSpecDecl 0x5dae9b2d70a8 <line:15:1, col:7> col:1 public
        `-TypeAliasDecl 0x5dae9b2d7290 <line:16:3, col:48> col:9 type 'decltype((__test(_Args()) + ...))'
          `-DecltypeType 0x5dae9b2d7250 'decltype((__test(_Args()) + ...))' dependent
            `-CXXFoldExpr 0x5dae9b2d71d8 <col:25, col:47> '<dependent type>'
              |-UnresolvedLookupExpr 0x5dae9b2d7190 <col:44> '<overloaded function type>' lvalue (ADL) = 'operator+' 0x5dae9b2b3af0
              |-CallExpr 0x5dae9b2d7168 <col:26, col:40> '<dependent type>'
              | |-DeclRefExpr 0x5dae9b2d7118 <col:26> 'double (int)' lvalue CXXMethod 0x5dae9b2d6ff8 '__test' 'double (int)' non_odr_use_unevaluated
              | `-CXXUnresolvedConstructExpr 0x5dae9b2d7148 <col:33, col:39> '_Args' '_Args'
              `-<<<NULL>>>

clang++ -Xclang -ast-dump -fsyntax-only b.hpp    

TranslationUnitDecl 0x624ca0895ba8 <<invalid sloc>> <invalid sloc>
|-TypedefDecl 0x624ca0896830 <<invalid sloc>> <invalid sloc> implicit __int128_t '__int128'
| `-BuiltinType 0x624ca0896170 '__int128'
|-TypedefDecl 0x624ca08968a0 <<invalid sloc>> <invalid sloc> implicit __uint128_t 'unsigned __int128'
| `-BuiltinType 0x624ca0896190 'unsigned __int128'
|-TypedefDecl 0x624ca08e45e0 <<invalid sloc>> <invalid sloc> implicit __NSConstantString '__NSConstantString_tag'
| `-RecordType 0x624ca0896990 '__NSConstantString_tag'
|   `-CXXRecord 0x624ca08968f8 '__NSConstantString_tag'
|-TypedefDecl 0x624ca0896468 <<invalid sloc>> <invalid sloc> implicit __builtin_ms_va_list 'char *'
| `-PointerType 0x624ca0896420 'char *'
|   `-BuiltinType 0x624ca0895c50 'char'
|-TypedefDecl 0x624ca08967c0 <<invalid sloc>> <invalid sloc> implicit __builtin_va_list '__va_list_tag[1]'
| `-ConstantArrayType 0x624ca0896770 '__va_list_tag[1]' 1
|   `-RecordType 0x624ca0896550 '__va_list_tag'
|     `-CXXRecord 0x624ca08964c0 '__va_list_tag'
`-NamespaceDecl 0x624ca08e4638 <b.hpp:1:1, line:13:1> line:1:11 std
  `-NamespaceDecl 0x624ca08e46a8 <line:2:1, line:12:1> line:2:18 __1 inline
    `-ClassTemplateDecl 0x624ca08e4880 <line:4:1, line:10:1> line:4:33 __promote
      |-TemplateTypeParmDecl 0x624ca08e4718 <col:11, col:20> col:20 referenced class depth 0 index 0 ... _Args
      `-CXXRecordDecl 0x624ca08e47f0 <col:27, line:10:1> line:4:33 class __promote definition
        |-DefinitionData empty aggregate standard_layout trivially_copyable pod trivial literal has_constexpr_non_copy_move_ctor can_const_default_init
        | |-DefaultConstructor exists trivial constexpr needs_implicit defaulted_is_constexpr
        | |-CopyConstructor simple trivial has_const_param needs_implicit implicit_has_const_param
        | |-MoveConstructor exists simple trivial needs_implicit
        | |-CopyAssignment simple trivial has_const_param needs_implicit implicit_has_const_param
        | |-MoveAssignment exists simple trivial needs_implicit
        | `-Destructor simple irrelevant trivial needs_implicit
        |-CXXRecordDecl 0x624ca08e4ba8 <col:27, col:33> col:33 implicit class __promote
        |-CXXMethodDecl 0x624ca08e4d58 <line:6:3, col:27> col:17 referenced __test 'double (int)' static
        | `-ParmVarDecl 0x624ca08e4c50 <col:24> col:27 'int'
        |-AccessSpecDecl 0x624ca08e4e08 <line:8:1, col:7> col:1 public
        `-TypeAliasDecl 0x624ca08e4fa0 <line:9:3, col:48> col:9 type 'decltype((__test(_Args()) + ...))'
          `-DecltypeType 0x624ca08e4f60 'decltype((__test(_Args()) + ...))' dependent
            `-CXXFoldExpr 0x624ca08e4ef0 <col:25, col:47> '<dependent type>'
              |-<<<NULL>>>
              |-CallExpr 0x624ca08e4ec8 <col:26, col:40> '<dependent type>'
              | |-DeclRefExpr 0x624ca08e4e78 <col:26> 'double (int)' lvalue CXXMethod 0x624ca08e4d58 '__test' 'double (int)' non_odr_use_unevaluated
              | `-CXXUnresolvedConstructExpr 0x624ca08e4ea8 <col:33, col:39> '_Args' '_Args'
              `-<<<NULL>>>

The difference seems to be with UnresolvedLookupExpr.

[koplas]

I just checked the AST dump of my reproducer that contains the fully preprocessed headers (https://github.com/koplas/clang-headerunits/tree/pch-full) and I couldn’t really identify the reason for the errors. It can be possible that fixing the error of the minimal reproducer may not be enough.

[ChuanqiXu9]

One idea is, to check if these two ASTs are identified as the same AST entity. And if not, we should think if they should be identified as the same AST entity.