[clang][modules] Inline function incorrectly marked as hidden leads to undefined reference

[ldionne]

In some circumstances, functions are incorrectly given hidden visibility when Clang modules are enabled, which leads to undefined references. Here's a reproducer where we basically have two small header files, each of which is a module:

// construct_at:
template <class T>
constexpr void __construct_at(T* location, T arg) {
  void* loc = __voidify(*location);
  (void)loc;
}

// voidify
template <class T>
__attribute__((__always_inline__)) void* __voidify(T& from) {
  return const_cast<void*>(static_cast<const volatile void*>(&from));
}

We then have a small test program that uses construct_at:

#include <construct_at.h>

void foo(int* ptr) {
    __construct_at(ptr, 3);
}

and the following modulemap:

module voidify      { header "voidify.h" }
module construct_at { header "construct_at.h" }

Finally, we compile with modules enabled:

clang++ -std=c++17 -fmodules -fcxx-modules -fmodules-cache-path=ModuleCache.noindex -fbuiltin-module-map -c test.cpp -o test.o

And we can see that an undefined reference to __voidify is being generated in test.o, which should never happen for a few reasons:

1. It's a template, so we need to have a linkonce_odr definition in test.o
2. Even worse, it's marked as always_inline so it's absolutely evident that this can't end happily. Indeed, we know at the declaration that a definition will never be emitted anywhere else, so this is bound to fail.

In other words, this shouldn't create an undefined reference to __voidify regardless of whether that function is marked as __always_inline__, but it's just a bit more vexing that it does.

Full reproducer attached.

Also tracked as rdar://121551667.

[llvmbot]

@llvm/issue-subscribers-clang-modules

Author: Louis Dionne (ldionne)

In some circumstances, functions are incorrectly given hidden visibility when Clang modules are enabled, which leads to undefined references. Here's a reproducer where we basically have two small header files, each of which is a module:

// construct_at:
template <class T>
constexpr void __construct_at(T* location, T arg) {
  void* loc = __voidify(*location);
  (void)loc;
}

// voidify
template <class T>
__attribute__((__always_inline__)) void* __voidify(T& from) {
  return const_cast<void*>(static_cast<const volatile void*>(&from));
}

We then have a small test program that uses construct_at:

#include <construct_at.h>

void foo(int* ptr) {
    __construct_at(ptr, 3);
}

Finally, we compile with modules enabled:

clang++ -std=c++17 -fmodules -fcxx-modules -fmodules-cache-path=ModuleCache.noindex -fbuiltin-module-map -c test.cpp -o test.o

And we can see that an undefined reference to __voidify is being generated in test.o, which should never happen for a few reasons:

1. It's a template, so we need to have a linkonce_odr definition in test.o
2. Even worse, it's marked as always_inline so it's absolutely evident that this can't end happily. Indeed, we know at the declaration that a definition will never be emitted anywhere else, so this is bound to fail.

In other words, this shouldn't create an undefined reference to __voidify regardless of whether that function is marked as __always_inline__, but it's just a bit more vexing that it does.

Full reproducer attached.

Also tracked as rdar://121551667.

[ldionne]

@Bigcheese found this:

Without modules:

|-NamespaceDecl 0x13f065de8 <.../voidify.h:4:1, line:11:2> line:4:11 std
| `-NamespaceDecl 0x13f065e58 <col:17, line:11:1> line:4:34 __1 inline
|   `-FunctionTemplateDecl 0x13f066198 <line:6:1, line:9:1> line:7:42 __voidify
|     |-TemplateTypeParmDecl 0x13f065ec8 <line:6:11, col:17> col:17 referenced class depth 0 index 0 _Tp
|     |-FunctionDecl 0x13f0660f0 <line:7:1, line:9:1> line:7:42 __voidify 'void *(_Tp &)'
|     | |-ParmVarDecl 0x13f065ff8 <col:52, col:57> col:57 referenced __from '_Tp &'
|     | |-CompoundStmt 0x13f0663c8 <col:65, line:9:1>
|     | | `-ReturnStmt 0x13f0663b8 <line:8:3, col:70>
|     | |   `-CXXConstCastExpr 0x13f066388 <col:10, col:70> 'void *' const_cast<void *> <NoOp>
|     | |     `-CXXStaticCastExpr 0x13f066340 <col:28, col:69> 'const volatile void *' static_cast<const volatile void *> <Dependent>
|     | |       `-UnaryOperator 0x13f0662e8 <col:62, col:63> '<dependent type>' prefix '&' cannot overflow
|     | |         `-DeclRefExpr 0x13f0662c8 <col:63> '_Tp' lvalue ParmVar 0x13f065ff8 '__from' '_Tp &'
|     | `-AlwaysInlineAttr 0x13f0661f8 <line:7:16> always_inline
|     `-FunctionDecl 0x13b029040 <col:1, line:9:1> line:7:42 used __voidify 'void *(int &)'
|       |-TemplateArgument type 'int'
|       | `-BuiltinType 0x13b012110 'int'
|       |-ParmVarDecl 0x13b028f48 <col:52, col:57> col:57 used __from 'int &'
|       |-CompoundStmt 0x13b0294b8 <col:65, line:9:1>
|       | `-ReturnStmt 0x13b0294a8 <line:8:3, col:70>
|       |   `-CXXConstCastExpr 0x13b029478 <col:10, col:70> 'void *' const_cast<void *> <NoOp>
|       |     `-CXXStaticCastExpr 0x13b029448 <col:28, col:69> 'const volatile void *' static_cast<const volatile void *> <NoOp>
|       |       `-ImplicitCastExpr 0x13b029430 <col:62, col:63> 'const volatile void *' <BitCast> part_of_explicit_cast
|       |         `-UnaryOperator 0x13b029418 <col:62, col:63> 'int *' prefix '&' cannot overflow
|       |           `-DeclRefExpr 0x13b0293d0 <col:63> 'int':'int' lvalue ParmVar 0x13b028f48 '__from' 'int &'
|       `-AlwaysInlineAttr 0x13b029148 <line:7:16> always_inline

With modules:

|-NamespaceDecl 0x14c127660 <.../voidify.h:4:1, line:11:2> line:4:11 imported in voidify hidden <undeserialized declarations> std
| `-NamespaceDecl 0x14c127910 <col:17, line:11:1> line:4:34 imported in voidify hidden <undeserialized declarations> __1 inline
|   `-FunctionTemplateDecl 0x14c81a690 <line:6:1, line:9:1> line:7:42 imported in voidify hidden __voidify
|     |-TemplateTypeParmDecl 0x14c81a768 <line:6:11, col:17> col:17 imported in voidify hidden referenced class depth 0 index 0 _Tp
|     |-FunctionDecl 0x14c81a808 <line:7:1, line:9:1> line:7:42 imported in voidify hidden __voidify 'void *(_Tp &)'
|     | |-ParmVarDecl 0x14c81aa08 <col:52, col:57> col:57 imported in voidify hidden referenced __from '_Tp &'
|     | |-CompoundStmt 0x14c81d678 <col:65, line:9:1>
|     | | `-ReturnStmt 0x14c81d668 <line:8:3, col:70>
|     | |   `-CXXConstCastExpr 0x14c81d620 <col:10, col:70> 'void *' const_cast<void *> <NoOp>
|     | |     `-CXXStaticCastExpr 0x14c81d5a8 <col:28, col:69> 'const volatile void *' static_cast<const volatile void *> <Dependent>
|     | |       `-UnaryOperator 0x14c81d590 <col:62, col:63> '<dependent type>' prefix '&' cannot overflow
|     | |         `-DeclRefExpr 0x14c81d570 <col:63> '_Tp' lvalue ParmVar 0x14c81aa08 '__from' '_Tp &'
|     | `-AlwaysInlineAttr 0x14c81a8b0 <line:7:16> always_inline
|     `-FunctionDecl 0x14c81d1c0 <col:1, line:9:1> line:7:42 used __voidify 'void *(int &)'
|       |-TemplateArgument type 'int'
|       | `-BuiltinType 0x14c04e710 'int'
|       |-ParmVarDecl 0x14c81d0c8 <col:52, col:57> col:57 __from 'int &'
|       `-AlwaysInlineAttr 0x14c81d2c8 <col:16> always_inline

It’s somehow getting marked hidden, and the instantiation just doesn’t happen.

[ldionne]

Note that this was encountered again recently in the context of #118837

[ilya-biryukov]

Note that Chrome hit this too when trying to use modularized libc++ for their builds (https://issues.chromium.org/issues/390537876), the workaround that worked there is #129195.

There are likely more examples where this breaks, maybe a more severe workaround is possible that adds export * to everything?
Reverting such a workaround later would entail some errors on the side of people that used internal APIs, but would not hurt those who only use public APIs. (Or are there more complications I'm missing).

@ldionne is that a direction libc++ would be willing to consider?

PS the best option is clearly fixing Clang, @ChuanqiXu9 do you happen to have thoughts on how to fix this?

[ilya-biryukov]

Sorry for accidentally closing the issue, I've hit the wrong button.

[hokein]

Adding more context here, it seems like this issue is related to a clang bug regarding symbol visibility in clang modules.

Specifically, the template function __voidify is not visible from the module construct_at. As a result, template instantiation for __voidify is not performed, meaning we don't generate a definition for __voidify<int>() in the AST. Consequently, no undefined reference error is reported at the linking stage.

$ clang -std=c++20 -Wundefined-func-template -fmodules -fmodules-cache-path=output -fbuiltin-module-map -I include test.cpp
In module 'construct_at' imported from test.cpp:1:
include/construct_at.h:8:15: warning: instantiation of function '__voidify<int>' required here, but no definition is available [-Wundefined-func-template]
    8 |   void* loc = __voidify(*location);
      |               ^
include/voidify.h:5:42: note: forward declaration of template entity is here
__attribute__((__always_inline__)) void* __voidify(T& from) {
                                         ^
include/construct_at.h:8:15: note: add an explicit instantiation declaration to suppress this warning if '__voidify<int>' is explicitly instantiated in another translation unit    
void* loc = __voidify(*location);
              ^
1 warning generated.
/usr/bin/ld: /lib/x86_64-linux-gnu/Scrt1.o: in function `_start':
(.text+0x17): undefined reference to `main'
/usr/bin/ld: /tmp/test-6703dd.o: in function `void __construct_at<int>(int*, int)':
test.cpp:(.text._Z14__construct_atIiEvPT_S0_[_Z14__construct_atIiEvPT_S0_]+0x14): undefined reference to `void* __voidify<int>(int&)'