`std::any` is still broken with `visbility=hidden` and `libc++` when used cross DSO boundaries

[milianw]

This is related to #48743 which seems to have fixed this issue for libstdc++, but apparently there are remaining problems when using libc++:

libc++ with clang-20.1.0: https://godbolt.org/z/4TW4rMYoe note that the typeid name is the same, but the hash differs already:

NSt3__110shared_ptrI10WrappedIntEE 134520565932116
NSt3__110shared_ptrI10WrappedIntEE 105617327562758

this then leads to:

libc++abi: terminating due to uncaught exception of type std::bad_any_cast: bad any cast
Program terminated with signal: SIGSEGV

If we don't enforce libc++ and thus default to libstdc++ then everything works:

https://godbolt.org/z/bsGvfzWsh

St10shared_ptrI10WrappedIntE 6190920035151885185
St10shared_ptrI10WrappedIntE 6190920035151885185

[ldionne]

If you switch to struct __attribute__((visibility("default"))) WrappedInt {, it works: https://godbolt.org/z/KEf6accK9

Without it, you end up generating two copies of the RTTI for WrappedInt, one in main.cpp and one in library.cpp, and both are marked as hidden.

[ldionne]

(feel free to comment or reopen if you don't think my explanation makes sense)

[milianw]

But why is this needed for libc++ when it isn't required for libstdc++? Note that wrapped int is completely inline, and as such should not need to be exported at all - this goes against everything I learned about symbol visibility so far.

[milianw]

Maybe to explain this a bit more on why I still consider this to be a defect: Some types are not under our control, think e.g. templates types from some thirdparty libraries, those are usually not exported and thus std::any would break with them like here: https://godbolt.org/z/oWqqjEbM1 With libstdc++ it works fine: https://godbolt.org/z/r8sMbavEc

[StefanoD]

@milianw We should also mention, that we don't have this problem with Windows and Visual Studio Compiler and GCC with glibc

[philnik777]

@milianw @StefanoD With libstdc++ this most likely happens to work out because the function taking the typeinfo doesn't get inlined and libstdc++ gives all functions default visibility. Because of that it's always the same function that gets the typeinfo. As soon as there is more than one function taking the actual typeinfo you're out of luck. We could work around the issue by having a single function in the library with default visibility and [[gnu::noinline]], but that's super hacky and doesn't solve the problem fundamentally. As Louis already pointed out, the correct way to fix this is to annotate the type if you want to use -fvisibility=hidden. FWIW, if you want to work around it you can add [[clang::type_visibilty("default")]] via pragmas.

[milianw]

The "super hacky" thing you describe sounds ideal to me, as this works on all other platforms and only libc++ suffers from this.

Is there at least an arcane way for std::any or similar to detect that one tries to use it with a non-exported type, to then warn/error out? Because right now everything looks fine at compile time only to then fail fatally at runtime (and only on macOS for us, as that's the only platform we target with clang and libc++).

I also would like to have confirmation: Are you really saying one should export fully inline types? That goes against what I learned, one does not want to do that normally.

[StefanoD]

@philnik777 You break essentially std::any. As @milianw pointed out: For thirdparty libraries, this is out of our control. And this is the only lib which behaves like that. From an user perspective: It's totally unexpected and it breaks std::any

[philnik777]

The "super hacky" thing you describe sounds ideal to me, as this works on all other platforms and only libc++ suffers from this.

It's not. This only happens to work out right now with libstdc++. Not annotating the types properly is a latent bug in your code. Also, other people who annotate their types properly would also complain because we'd essentially be adding an entirely unnecessary optimization barrier.

Is there at least an arcane way for std::any or similar to detect that one tries to use it with a non-exported type, to then warn/error out? Because right now everything looks fine at compile time only to then fail fatally at runtime (and only on macOS for us, as that's the only platform we target with clang and libc++).

Not that I'm aware of. Also note that this is only a problem across dylib boundaries, so most uses don't suffer from this problem.

I also would like to have confirmation: Are you really saying one should export fully inline types? That goes against what I learned, one does not want to do that normally.

Yes, you have to export the typeinfo of all types. I don't know what you've learned, but that's exactly what both libc++ and libstdc++ do to make using typeinfo for stdlib types across dylib boundaries work.

@philnik777 You break essentially std::any. As @milianw pointed out: For thirdparty libraries, this is out of our control.

You can always wrap it into your own type that has the correct annotations. I also pointed out above that you can hack your way around third library code not annotating their code via pragmas. Another option is to not use -fvisibility=hidden and instead annotate your code, as both libc++ and libstdc++ do.

And this is the only lib which behaves like that. From an user perspective: It's totally unexpected and it breaks std::any

Why? You're saying you don't need the typeinfo across dylib boundaries via -fvisibility=hidden and we respect that.

[milianw]

I found the source where I learned not to apply export to fully inline types, see: https://wiki.qt.io/Things_To_Look_Out_For_In_Reviews see Value Classes #6 "Never export a non-polymorphic class wholesale"

you can hack your way around third library code not annotating their code via pragmas

How so? Can you please expand on this? Are you saying I should do something like (pseudo code)

#push pragma force export
#include <thirdparty headers>
#pop pragma

[philnik777]

I found the source where I learned not to apply export to fully inline types, see: https://wiki.qt.io/Things_To_Look_Out_For_In_Reviews see Value Classes #6 "Never export a non-polymorphic class wholesale"

Given that the rationale talks specifically about MSVC I don't think it applies here. Also note that we're not asking you to export everything. We're only asking you to export the RTTI.

you can hack your way around third library code not annotating their code via pragmas

How so? Can you please expand on this? Are you saying I should do something like (pseudo code)

#push pragma force export
#include <thirdparty headers>
#pop pragma

Yes, exactly. It would be something like

#pragma clang attribute VisibilityAnnotations.push([[clang::type_visibility("default")]], apply_to=record)
#include <whatever>
#pragma clang attribute VisibilityAnnotations.pop

You can change VisibilityAnnotations to anything you want though.

[milianw]

Given that the rationale talks specifically about MSVC I don't think it applies here.

Well, when writing cross platform code, we must find a solution that works everywhere. As such, I cannot ignore MSVC. It sounds as-if we would need separate export macros then, to pleasen the different platforms which makes me quite sad. If only the platforms could work a bit more similar to simplify things dream

Also note that we're not asking you to export everything. We're only asking you to export the RTTI.

This is the first time I hear about the distinction here, thanks! For others who are equally unaware: https://releases.llvm.org/20.1.0/tools/clang/docs/AttributeReference.html#type-visibility

Sadly, this seems to be a clang specific feature, I cannot find something similar for GCC. And there is also apparently no flag to configure it globally while compiling. If that would exist, one would ideally be able to do

clang -ftype-visibility=default -fvisibility=hidden -fvisibility-inlines-hidden

And get a result that works in a similar way to other platforms, right? Can we maybe repurpose this bug report accordingly then, to kindly ask for the addition for this global -ftype-visibility flag?

The "custom pragma" approach you are giving doesn't sound scalable - we would have to do that everywhere we include the thirdparty code after all, and that is extremely errorprone, no?