[clang] Improve nested name specifier AST representation #147835
Merged
+11,195
−10,586
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mizvekov
requested review from
zygoloid,
rjmccall,
cor3ntin,
AaronBallman,
erichkeane and
zyn0217
mizvekov
requested review from
daniel-grumberg,
QuietMisdreavus,
cyndyishida and
Endilll
as code owners
llvmbot
added
clang
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/200/builds/14983 Here is the relevant piece of the build log for the reference |
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/162/builds/28595 Here is the relevant piece of the build log for the reference |
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/59/builds/22354 Here is the relevant piece of the build log for the reference |
mizvekov
added a commit
that referenced
this pull request
Aug 9, 2025
This completes #147835 by adapting the changes in the API to the rewriter. Fixes build failures such as reported here: #147835 (comment) We previously missed this because it was not tested in pre-commit CI. This patch also addresses that problem.
mizvekov
added a commit
that referenced
this pull request
Aug 9, 2025
This completes #147835 by adapting the changes in the API to the rewriter. Fixes build failures such as reported here: #147835 (comment) We previously missed this because it was not tested in pre-commit CI.
mizvekov
added a commit
that referenced
this pull request
Aug 9, 2025
This completes #147835 by adapting the changes in the API to the rewriter. Fixes build failures such as reported here: #147835 (comment) We previously missed this because it was not tested in pre-commit CI.
llvm-sync bot
pushed a commit
to arm/arm-toolchain
that referenced
this pull request
Aug 9, 2025
This completes llvm/llvm-project#147835 by adapting the changes in the API to the rewriter. Fixes build failures such as reported here: llvm/llvm-project#147835 (comment) We previously missed this because it was not tested in pre-commit CI.
searlmc1
pushed a commit
to ROCm/llvm-project
that referenced
this pull request
Aug 10, 2025
…m#147835)" needs dependent PR [clang][CGRecordLayout] Remove dependency on isZeroSize (llvm#96422) This reverts commit 91cdd35.
|
This has changed the output of lldb's std::string formatter when it's not a valid std::string object: We create the (not a) std::string like this: So in some sense, This is from Linaro's bot where we only run libstdcxx tests. Libcxx testing seems fine (https://green.lab.llvm.org/job/llvm.org/view/LLDB/job/lldb-cmake-matrix/). This could be because the formatter has different behaviour for the two standard libraries. @Michael137 might know where to look for this problem, in the meantime, I'll do my own digging. |
DavidSpickett
added a commit
to DavidSpickett/llvm-project
that referenced
this pull request
Aug 11, 2025
llvm#147835 made changes that caused libstdc++ std::string tests to fail: ``` ====================================================================== FAIL: test_unavailable_summary_libstdcxx_dwo (TestDataFormatterStdString.StdStringDataFormatterTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1805, in test_method return attrvalue(self) File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/functionalities/data-formatter/data-formatter-stl/generic/string/TestDataFormatterStdString.py", line 223, in test_unavailable_summary_libstdcxx self.do_test_summary_unavailable() File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/functionalities/data-formatter/data-formatter-stl/generic/string/TestDataFormatterStdString.py", line 213, in do_test_summary_unavailable self.assertEqual(summary, "Summary Unavailable", "No summary for bad value") AssertionError: '(null)' != 'Summary Unavailable' - (null) + Summary Unavailable : No summary for bad value Config=aarch64-/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/bin/clang ---------------------------------------------------------------------- ``` This test constructs an invalid std::string by starting with a null pointer. Somehow this improvement in Clang uncovered a bug in the formatter. The formatter tries to access `_M_p` and checked whether the resulting ValueObjectSP was null, but not that it did not contain an error value. I think that error value can be there if you are able to access one part of the path, `_M_dataplus`, but another part fails. Since the layout looks like this: ``` struct _Alloc_hider : { pointer _M_p; // The actual data. }; _Alloc_hider _M_dataplus; void _M_data(pointer __p) { _M_dataplus._M_p = __p; } ``` So I think we were able to read `_M_dataplus` just by offset, but then failed because it contains, or points to something containing nulls or a null pointer. Or perhaps an error value means we know what the class member is, but could not read from it. I found this by comparing with the libcxx formatter, so I've copied the same handling from there to fix the issue.
|
#152993 fixes this. Pretty sure there's no problem with this PR, it uncovered an existing problem in the formatter itself. |
DavidSpickett
added a commit
that referenced
this pull request
Aug 11, 2025
#147835 made changes that caused libstdc++ std::string tests to fail: ``` ====================================================================== FAIL: test_unavailable_summary_libstdcxx_dwo (TestDataFormatterStdString.StdStringDataFormatterTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1805, in test_method return attrvalue(self) File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/functionalities/data-formatter/data-formatter-stl/generic/string/TestDataFormatterStdString.py", line 223, in test_unavailable_summary_libstdcxx self.do_test_summary_unavailable() File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/functionalities/data-formatter/data-formatter-stl/generic/string/TestDataFormatterStdString.py", line 213, in do_test_summary_unavailable self.assertEqual(summary, "Summary Unavailable", "No summary for bad value") AssertionError: '(null)' != 'Summary Unavailable' - (null) + Summary Unavailable : No summary for bad value Config=aarch64-/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/bin/clang ---------------------------------------------------------------------- ``` This test constructs an invalid std::string by starting with a null pointer. Somehow this improvement in Clang uncovered a bug in the formatter. Perhaps because we now know the type of the field we tried to access is char *, so fall back to a C string formatter that produces `(null)`. The formatter tries to access `_M_p` and checked whether the resulting ValueObjectSP was null, but not that it did not contain an error value. I think that error value can be there if you are able to access one part of the path, `_M_dataplus`, but another part fails. Since the layout looks like this: ``` struct _Alloc_hider : { pointer _M_p; // The actual data. }; _Alloc_hider _M_dataplus; void _M_data(pointer __p) { _M_dataplus._M_p = __p; } ``` So I think we were able to read `_M_dataplus` just by offset, but then failed because it contains, or points to something containing nulls or a null pointer. Or perhaps an error value means we know what the class member is, but could not read from it. I found this by comparing with the libcxx formatter, so I've copied the same handling from there to fix the issue.
|
Observation using clangd: In the following construct The "arcana" field of the AST node of "Foo" inside the function reports the "QualType" as "Foo", whereas before it was "N::Foo". Is that expected? |
llvm-sync bot
pushed a commit
to arm/arm-toolchain
that referenced
this pull request
Aug 11, 2025
…#152993) llvm/llvm-project#147835 made changes that caused libstdc++ std::string tests to fail: ``` ====================================================================== FAIL: test_unavailable_summary_libstdcxx_dwo (TestDataFormatterStdString.StdStringDataFormatterTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1805, in test_method return attrvalue(self) File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/functionalities/data-formatter/data-formatter-stl/generic/string/TestDataFormatterStdString.py", line 223, in test_unavailable_summary_libstdcxx self.do_test_summary_unavailable() File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/functionalities/data-formatter/data-formatter-stl/generic/string/TestDataFormatterStdString.py", line 213, in do_test_summary_unavailable self.assertEqual(summary, "Summary Unavailable", "No summary for bad value") AssertionError: '(null)' != 'Summary Unavailable' - (null) + Summary Unavailable : No summary for bad value Config=aarch64-/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/bin/clang ---------------------------------------------------------------------- ``` This test constructs an invalid std::string by starting with a null pointer. Somehow this improvement in Clang uncovered a bug in the formatter. Perhaps because we now know the type of the field we tried to access is char *, so fall back to a C string formatter that produces `(null)`. The formatter tries to access `_M_p` and checked whether the resulting ValueObjectSP was null, but not that it did not contain an error value. I think that error value can be there if you are able to access one part of the path, `_M_dataplus`, but another part fails. Since the layout looks like this: ``` struct _Alloc_hider : { pointer _M_p; // The actual data. }; _Alloc_hider _M_dataplus; void _M_data(pointer __p) { _M_dataplus._M_p = __p; } ``` So I think we were able to read `_M_dataplus` just by offset, but then failed because it contains, or points to something containing nulls or a null pointer. Or perhaps an error value means we know what the class member is, but could not read from it. I found this by comparing with the libcxx formatter, so I've copied the same handling from there to fix the issue.
|
Did this PR intentionally break the It's a bit hard to test, because the NNS changes also break a lot of code in IWYU, so it's tricky to play with things in isolation. |
|
@kimgr, I've just succeeded in building IWYU against this and getting the tests green. However, I want to spend some time on self-review, probably writing additional tests, and so on. |
llvm-sync bot
pushed a commit
to arm/arm-toolchain
that referenced
this pull request
Aug 11, 2025
…ifier AST representation (#152846) After AST representation, new modifications landed in (llvm/llvm-project#147835). ClangIR requires some changes in how we use Clang AST to be compatible with the new changes
Yeah, the change in API helps in that we keep the same order of traversal as before, which makes it easier to keep existing code working. |
It looks correct in that this is the as-written type. Now I am not sure clangd wants to present the type that way, maybe they want to fully qualify it for printing instead? |
|
We are seeing an assertion failure in one of Fuchsia's builders due to this: Can you revert or fix forward? I've no idea whether this helps with debugging at all, but this is the source file that produced the assertion failure in clang: https://cs.opensource.google/fuchsia/fuchsia/+/main:sdk/lib/ld/zircon-startup.cc?q=zircon-startup.cc&ss=fuchsia |
Can you please attach the crash reproducer which clang should have generated? Thanks. |
|
After this commit, the comment in isn't accurate any more, there's no more |
clangd doesn't really have a preference here, we just surface one can repro the situation with where a.cc is: namespace N {
struct Foo {};
} // namespace N
void f() {
using namespace N;
Foo foo;
N::Foo x;
}pre this change output looks like: emphasis on after thins change this is now reported as this was handled by https://github.com/llvm/llvm-project/blob/main/clang/lib/AST/TextNodeDumper.cpp#L909-L926, I am not sure if change is intended (i think we should still be able to detect & desugar) |
@mizvekov, anyway, it would be good to know if However, I don't see where in the world QualifiedTypeLoc is created...
|
Thanks for reporting, I'll fix in a follow up patch.
Right, the difference here is that before this patch, if you print a fully desugared tag type (that's what the 'aka' in the diagnostic does), the type is printed fully qualified. That's because the name qualifiers were carried in the 'ElaboratedType' sugar node, and we used to handle this situation specially: we assumed a bare tag type only occurred because of canonicalization. With this patch the tag types themselves carry the name qualifier, so just removing the top level sugar doesn't make them lose name qualifier information. We still fully qualify canonical tag types when printing them though, but that now happens because a canonical tag type is a special value which is not identical to any other tag type for the same declaration. We may still want to fully qualify types for the 'aka' diagnostic, but I deferred that for a follow up patch in order to limit the amount of test churn. For clangd, I assume the same thing is happening here, for this arcana" field, you are fully desugaring the type before printing it.
The TraverseQualifier flag means, 'should I traverse NestedNameSpecifiers?' If you are simply customizing one of the Traverse* methods which take it, but still delegating to the base class Traverse method, you should just forward that flag along to the base method. If you are implementing your own Traverse method for a type from scratch, then if this flag is false, you should not traverse the nested name specifier for that type.
QualifiedTypeLoc is created implicitly for QualTypes which have top level qualifiers, and they represent the source locations for type qualifiers (not name qualifiers, different thing.) This assert here means that a top level type qualifier can't appear within a nested name specifier (but it could appear for example in the underlying type for a typedef). The C++ grammar doesn't allow you to write const / volatile in a way which applies to an entity in the name qualifier itself. While such a NestedNameSpecifier could still be produced as part of template argument deduction, we simply remove the top level qualifiers in that case, as the NestedNameSpecifier AST node itself still can't represent it. |
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a major change on how we represent nested name qualifications in the AST.
This patch offers a great performance benefit.
It greatly improves compilation time for stdexec. For one datapoint, for
test_on2.cppin that project, which is the slowest compiling test, this patch improves-ccompilation time by about 7.2%, with the-fsyntax-onlyimprovement being at ~12%.This has great results on compile-time-tracker as well:
This patch also further enables other optimziations in the future, and will reduce the performance impact of template specialization resugaring when that lands.
It has some other miscelaneous drive-by fixes.
About the review: Yes the patch is huge, sorry about that. Part of the reason is that I started by the nested name specifier part, before the ElaboratedType part, but that had a huge performance downside, as ElaboratedType is a big performance hog. I didn't have the steam to go back and change the patch after the fact.
There is also a lot of internal API changes, and it made sense to remove ElaboratedType in one go, versus removing it from one type at a time, as that would present much more churn to the users. Also, the nested name specifier having a different API avoids missing changes related to how prefixes work now, which could make existing code compile but not work.
How to review: The important changes are all in
clang/include/clang/ASTandclang/lib/AST, with also important changes inclang/lib/Sema/TreeTransform.h.The rest and bulk of the changes are mostly consequences of the changes in API.
PS: TagType::getDecl is renamed to
getOriginalDeclin this patch, just for easier to rebasing. I plan to rename it back after this lands.Fixes #136624
Fixes #43179
Fixes #68670
Fixes #92757