[lld] Warn for missing needed sections by --gdb-index

[aury6623]

Issue a warning when lld is passed --gdb-index and some object files have general debug info but not the specific debug sections generated by -ggnu-pubnames or -gsplit-dwarf. Missing those sections on all inputs leads to an empty index, which gdb ignores. Missing those sections on only some sources leads to a partially complete index, which confuses gdb.

Addresses #156732 and #34168

[github-actions]

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

[llvmbot]

@llvm/pr-subscribers-lld

@llvm/pr-subscribers-lld-elf

Author: None (aury6623)

Changes

Issue a warning when lld is passed --gdb-index and some object files have general debug info but not the specific debug sections generated by -ggnu-pubnames or -gsplit-dwarf. Missing those sections on all inputs leads to an empty index, which gdb ignores. Missing those sections on only some sources leads to a partially complete index, which confuses gdb.

Addresses #156732 and #34168

---

Full diff: https://github.com/llvm/llvm-project/pull/166626.diff

1 Files Affected:

• (modified) lld/ELF/SyntheticSections.cpp (+10)

diff --git a/lld/ELF/SyntheticSections.cpp b/lld/ELF/SyntheticSections.cpp
index a4150ebfa1653..77487de805960 100644
--- a/lld/ELF/SyntheticSections.cpp
+++ b/lld/ELF/SyntheticSections.cpp
@@ -3566,6 +3566,16 @@ std::unique_ptr<GdbIndexSection> GdbIndexSection::create(Ctx &ctx) {
     DWARFContext dwarf(std::make_unique<LLDDwarfObj<ELFT>>(file));
     auto &dobj = static_cast<const LLDDwarfObj<ELFT> &>(dwarf.getDWARFObj());
 
+    if (dobj.getGnuPubnamesSection().sec == nullptr &&
+        dobj.getGnuPubtypesSection().sec == nullptr) {
+      Warn(ctx)
+          << files[i]
+          << ": file contains debug info but is missing .debug_gnu_pubnames or "
+             ".debug_gnu_pubtypes sections required by --gdb-index. Compile "
+             "the file using -gsplit-dwarf or -ggnu-pubnames to generate these "
+             "sections.";
+    }
+
     // If the are multiple compile units .debug_info (very rare ld -r --unique),
     // this only picks the last one. Other address ranges are lost.
     chunks[i].sec = dobj.getInfoSection();

[aury6623]

This edit is a solution for Issue #156732 I posted a few months ago. Having this warning would have saved me a day or two of debugging! In particular, when only some files are compiled with -ggnu-pubnames, but --gdb-index is still used, gdb can act in strange ways. E.g. in one case, gdb would only acknowledge the existence of a global type within certain scopes of the same function!

Now knowing what the issue is, I found this forum post where someone is running into a similar issue:
https://groups.google.com/g/llvm-dev/c/hxnPll-6de0
and this feature request asking for --gdb-index to work without -ggnu-pubnames:
#34168

which suggests that other people may find this warning to be helpful too.

---

This is my first time contributing to llvm-project. I know adding new tests for a change is normally expected, but I'm not sure what test changes (if any) are expected for only an addition of a warning message, or where that test should be added. Please advise. Thanks!

[lenary]

Please can you add a test?

[aury6623]

@lenary

Please can you add a test?

Sure, happy to! This is my first contribution to llvm-project, so I'm not quite familiar with the testing infrastructure. I've read this page:
https://llvm.org/docs/TestingGuide.html
But I can't tell which category of test this edit fits into. Based on the guide, and that this issue is best tested by compiling a whole program (it requires submitting multiple .o files to lld), it seems like the external test-suite repo is best suited. However, I don't see anywhere in that repo dedicated to lld tests.

If you could let me know where a test for an edit like this should go, that would be helpful! Thanks!

[lenary]

Yeah, that guide is good at the higher level, but I can see why you need more guidance :)

I think what we'd want is an assembly + linker test. This would consist of 1 (maybe 2) assembly files, which are assembled with llvm-mc, and then linked. These can be written all in the same file using the split-file utility, and the approach documented in "Extra Files" in that document.

The core assertion behind the test would be to invoke LLD, and then run FileCheck on its output. FileCheck would then check for the warnings in LLD's output. This tends to be done something like:

# REQUIRES: <target you chose>

# RUN: rm -rf %t && split-file %s %t && cd %t
# RUN: llvm-mc -filetype=obj -triple=<target you choose> input.s -o input.o

# RUN: ld.lld input.o -o output.exe 2>&1 | FileCheck %s --check-prefix=WARNING --implicit-check-not=warning:

# WARNING: <the text from your warning>

#--- input.s

<contents of your assembly file>

It's fine to take a simple C example, and compile it on your own using clang -S -g file.c -o - to get some assembly to base your input on - I know assembling dwarf by hand isn't fun/easy.

[aury6623]

@lenary Thanks for the help! I added a test. I also updated an existing test which I realized was failing because it was expecting no output from ld.lld but was now getting the warning. I wasn't sure if adding those the missing sections to that test could impact the test's intent, so I figured the best thing to do is to just make the test ignore that particular warning by stripping it out with sed.

[lenary]

Thanks for adding the test.

I think using sed isn't great in the pipeline, as I don't know if it's available on Windows.

I think it would be fine to update that test to also check for the warning, but I'm not entirely sure.

[MaskRay]

Thanks for the motivation to improve things but I believe this will certainly break some corporate users. Left a comment on #34168 .

While warnings are generally acceptable in Linux distros and OSS environments, corporate users and large projects using custom toolchains often upgrade warnings to errors by specifying --fatal-warnings. Reporting a warning should probably be deferred to a lint tool. It's also quite difficult to ensure that all object files have .debug_gnu_pubnames - when you have prebuilt object files.

[aury6623]

@lenary

I think using sed isn't great in the pipeline, as I don't know if it's available on Windows.

I was wondering that too, but I checked to see if other tests use sed, and found lots of them. Here's one in the same directory which only requires x86 and uses sed:

llvm-project/lld/test/ELF/debug-names-different-aug-string.s

Lines 1 to 3 in 1c19645

	# REQUIRES: x86
	# RUN: rm -rf %t && mkdir %t && cd %t
	# RUN: sed 's/LLVM0700/LLVM9999/' %S/Inputs/debug-names-a.s | llvm-mc -filetype=obj -triple=x86_64 -o a.o

and here's one from llvm-objdump which specifically requires windows but still uses sed:

llvm-project/llvm/test/tools/llvm-objdump/X86/source-interleave-prefix-windows.test

Lines 2 to 7 in 39fbec0

	; REQUIRES: system-windows
	;; Test removal of trailing separators (both '/' and '\' on Windows systems).
	;; The prefix 'myprefix/\' is changed to 'myprefix'.
	; RUN: sed -e "s,SRC_COMPDIR,/Inputs,g" %p/Inputs/source-interleave.ll > %t.ll

So it seems fine to me.

@MaskRay

Thanks for the motivation to improve things but I believe this will certainly break some corporate users.

For what it's worth, I am a corporate user, working towards enabling fatal warnings on our compilations :)
In this case, the warning is really indicating that --gdb-index won't work as intended. Having this warning would have saved me a few days of debugging the tools.

That said, I understand a warning could cause trouble here, so what about making it a remark/info message instead of a warning? I just want the tool to provide some feedback that it's not doing what the user probably intended.

Latest gdb has .debug_names support. And I've found this statement "I consider .gdb_index to be near the end of its life."

I was unable to get the latest gdb to work with DWARF-5 debugging info and C macro debugging, but maybe I was doing something wrong. (EDIT: gdb does not currently support .debug_names unless produced by gdb itself.)
And yeah, .gdb_index is nearing end of life, but it's not dead yet, and plus a lot of people will continue using old tools for a few years to come. I know my corporate dev environment is still running a very old version of gdb by default unless you decide to build it for yourself. In fairness, running old gdb but using a modern ld.lld with this warning is probably an unusual combo, but it's the situation I'm in.

[lenary]

@MaskRay what do you think about something opt-in like -z gdb-report={none,warning,error} as we do for a lot of architectural features?

I think this would solve the usecase of trying to improve a build enough use the gdb-index, without making life worse for people who don't care to make that improvement.

---

@aury6623 ok, sed seems justified, it seems I'm not quite aligned with the status quo.

I don't know the next steps forward for the issue you're facing with trying to get gdb working with dwarf 5 and C macros. It might be that you were just missing -fdebug-macro, but if you think the issue is more complex, and Clang/LLD is generating dwarf 5 that a recent, specific version of GDB cannot process, then filing a bug for that would be a good idea.

[aury6623]

what do you think about something opt-in like -z gdb-report={none,warning,error} as we do for a lot of architectural features?

I'm not sure anyone would ever know to look for flag like that, if it's opt-in. I certainly wouldn't have known.

---

@MaskRay

Latest gdb has .debug_names support. And I've found this statement "I consider .gdb_index to be near the end of its life."

@lenary

I don't know the next steps forward for the issue you're facing with trying to get gdb working with dwarf 5 and C macros.

Right, I had forgotten the exact problem that I encountered when trying dwarf-5 with gdb. The problem is that gdb only supports .debug_names when generated by gdb itself (e.g. using gdb-add-index -dwarf-5. Also possibly by gcc/GNU ld, but I'm not sure, it looks like they're still working on that)

You can see in the lastest gdb sources right now, it ignores .debug_names unless it was generated by gdb.

https://gitlab.com/gnutools/binutils-gdb/-/blob/fd3b1c4f4c111444dfe6ff9c3f00a5469e7cd8a6/gdb/dwarf2/read-debug-names.c#L626-630

That's the message I see when I try it myself.

Here's the gdb bug ticket for supporting .debug_names from clang:
https://sourceware.org/bugzilla/show_bug.cgi?id=32616

For my project's 600 MB debug binary (without an index), using gdb-add-index takes 1.5 minutes (which would need to be done after every compilation) while using ld.lld --gdb-index seems to take no significant extra time.

I'd love to use standardized dwarf-5 features, but clearly they're not quite ready yet and I don't think we should dismiss .gdb_index as near end of life.

[MaskRay]

If we have to add a diagnostic, an option like -z gdb-index-report={none,warning,error} is probably the way to go.
I'm concerned that this would not be general enough and users would complain that they have -g object files not built with the toolchain, leading to warnings. And then they would like more options like --gdb-index-report-exclude=a.o.
If you are a corporate user and you care much about debug info quality, you likely have following steps that process debug info (debuginfod/SymbolStore/...), and you can a debug info lint post linking.