clang produces invalid DWARF: "Offset is in between DIEs"

[gregbedwell]

https://godbolt.org/z/Kz1rrKTdo

Reduced testcase (from a full LTO link):

define void @a() !dbg !10 {
  br label %for.b.c.c, !dbg !13
  for.b.c.c:
    br label %for.b.c.c
}

!llvm.dbg.cu = !{!0, !6}
!llvm.module.flags = !{!8}

!0 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus_20, file: !1, emissionKind: FullDebug, globals: !2)
!1 = !DIFile(filename: "foo.cpp", directory: "")
!2 = !{!3}
!3 = !DIGlobalVariableExpression(var: !4, expr: !DIExpression())
!4 = !DIGlobalVariable(type: !5)
!5 = !DICompositeType(tag: DW_TAG_class_type)
!6 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus_20, file: !7, emissionKind: FullDebug)
!7 = !DIFile(filename: "bar.cpp", directory: "")
!8 = !{i32 2, !"Debug Info Version", i32 3}
!10 = distinct !DISubprogram(type: !11, unit: !6)
!11 = !DISubroutineType(types: !12)
!12 = !{}
!13 = !DILocation(scope: !14, inlinedAt: !15)
!14 = distinct !DISubprogram(unit: !6)
!15 = !DILocation(scope: !16, inlinedAt: !25)
!16 = distinct !DISubprogram(type: !11, unit: !6, declaration: !17)
!17 = !DISubprogram(scope: !5, type: !11, spFlags: DISPFlagOptimized, templateParams: !18)
!18 = !{!19}
!19 = !DITemplateTypeParameter(type: !20)
!20 = !DICompositeType(tag: DW_TAG_class_type, scope: !21)
!21 = distinct !DISubprogram(unit: !6, retainedNodes: !22)
!22 = !{!23}
!23 = !DILocalVariable(scope: !21, type: !24)
!24 = !DIBasicType()
!25 = !DILocation(scope: !21, inlinedAt: !26)
!26 = !DILocation(scope: !10)

Annoyingly, I don't have llc available in my current environment, but I can repro with clang:

clang -x ir -w -c reduce.ll -o reduce.o
llvm-dwarfdump --verify reduce.o

Verifying <source>:	file format elf64-x86-64
Verifying .debug_abbrev...
Verifying .debug_info Unit Header Chain...
Verifying .debug_types Unit Header Chain...
Verifying non-dwo Units...
Verifying unit: 1 / 2, "foo.cpp"
error: invalid DIE reference 0x0000002e. Offset is in between DIEs:

0x0000002d: DW_TAG_variable [8]   (0x00000029)
              DW_AT_type [DW_FORM_ref4]	(cu + 0x002e => {0x0000002e})


Verifying unit: 2 / 2, "bar.cpp"
Verifying dwo Units...
Verifying .debug_line...
Verifying .debug_str_offsets...
error: Aggregated error counts:
error: Invalid DIE reference occurred 1 time(s).
Errors detected.

This appears to be a recent regression within the last few days, but I haven't had a chance to bisect it yet.

[llvmbot]

@llvm/issue-subscribers-debuginfo

Author: Greg Bedwell (gregbedwell)

https://godbolt.org/z/Kz1rrKTdo

Reduced testcase (from a full LTO link):

define void @<!-- -->a() !dbg !10 {
  br label %for.b.c.c, !dbg !13
  for.b.c.c:
    br label %for.b.c.c
}

!llvm.dbg.cu = !{!0, !6}
!llvm.module.flags = !{!8}

!0 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus_20, file: !1, emissionKind: FullDebug, globals: !2)
!1 = !DIFile(filename: "foo.cpp", directory: "")
!2 = !{!3}
!3 = !DIGlobalVariableExpression(var: !4, expr: !DIExpression())
!4 = !DIGlobalVariable(type: !5)
!5 = !DICompositeType(tag: DW_TAG_class_type)
!6 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus_20, file: !7, emissionKind: FullDebug)
!7 = !DIFile(filename: "bar.cpp", directory: "")
!8 = !{i32 2, !"Debug Info Version", i32 3}
!10 = distinct !DISubprogram(type: !11, unit: !6)
!11 = !DISubroutineType(types: !12)
!12 = !{}
!13 = !DILocation(scope: !14, inlinedAt: !15)
!14 = distinct !DISubprogram(unit: !6)
!15 = !DILocation(scope: !16, inlinedAt: !25)
!16 = distinct !DISubprogram(type: !11, unit: !6, declaration: !17)
!17 = !DISubprogram(scope: !5, type: !11, spFlags: DISPFlagOptimized, templateParams: !18)
!18 = !{!19}
!19 = !DITemplateTypeParameter(type: !20)
!20 = !DICompositeType(tag: DW_TAG_class_type, scope: !21)
!21 = distinct !DISubprogram(unit: !6, retainedNodes: !22)
!22 = !{!23}
!23 = !DILocalVariable(scope: !21, type: !24)
!24 = !DIBasicType()
!25 = !DILocation(scope: !21, inlinedAt: !26)
!26 = !DILocation(scope: !10)

Annoyingly, I don't have llc available in my current environment, but I can repro with clang:

clang -x ir -w -c reduce.ll -o reduce.o
llvm-dwarfdump --verify reduce.o

Verifying <source>:	file format elf64-x86-64
Verifying .debug_abbrev...
Verifying .debug_info Unit Header Chain...
Verifying .debug_types Unit Header Chain...
Verifying non-dwo Units...
Verifying unit: 1 / 2, "foo.cpp"
error: invalid DIE reference 0x0000002e. Offset is in between DIEs:

0x0000002d: DW_TAG_variable [8]   (0x00000029)
              DW_AT_type [DW_FORM_ref4]	(cu + 0x002e => {0x0000002e})


Verifying unit: 2 / 2, "bar.cpp"
Verifying dwo Units...
Verifying .debug_line...
Verifying .debug_str_offsets...
error: Aggregated error counts:
error: Invalid DIE reference occurred 1 time(s).
Errors detected.

This appears to be a recent regression within the last few days, but I haven't had a chance to bisect it yet.

[jmorse]

NB: I can only make sense of the metadata by drawing it on a piece of paper. I don't suppose anyone's aware of graph-visualisation-tools that can be applied to metadata? (It wouldn't be as flexible as doing things by hand sadly).

There are two nests of metadata based on CUs !0 and !6. The former is the smallest, having a chain (!0 !2 !3 !4 !5) of declaring a global variable in the context of a DICompositeType. There's then a long and complicated nest of inlining chains from subprogram !10 / DILocation !13 in CU !6.

Where the CUs cross over in the DWARF is the declaration field of !16. Here's a gist of the DWARF output: link. The faulty reference is in the variable at 0x2d. It looks like the DICompositeType for !5 is placed at offset 0x1f in CU !0, probably because it's needed to describe the global variable in that CU. When the DISubprogram for !16 is produced, it refers to the declaration at !17, which is created/placed inside its scope, in CU !0. This cross-CU reference works fine. The trouble comes when the variable-type reference is produced in CU !0 within the declaration: the DIBasicType referred to is in CU !6 and shareable across CUs. Adding the DW_AT_type:

   701  void DwarfUnit::addType(DIE &Entity, const DIType *Ty,
   702                          dwarf::Attribute Attribute) {
   703    assert(Ty && "Trying to add a type that doesn't exist?");
-> 704    addDIEEntry(Entity, Attribute, DIEEntry(*getOrCreateTypeDIE(Ty)));
   705  }

Here getOrCreateTypeDIE will return a DIE in a different unit to the Variable that's being created. And as the Variable hasn't yet been inserted into a unit, the following code fires:

void DwarfUnit::addDIEEntry(DIE &Die, dwarf::Attribute Attribute,
                            DIEEntry Entry) {
  const DIEUnit *CU = Die.getUnit();
  const DIEUnit *EntryCU = Entry.getEntry().getUnit();
  if (!CU)
    // We assume that Die belongs to this CU, if it is not linked to any CU yet.
    CU = getUnitDie().getUnit();

Where that last assumption is false.

I believe errors like these have popped up in the past -- I get the impression that solve this, we need to instrument all of the dwarf-printing code with knowledge about what unit something is being placed in, which sounds like a serious undertaking.

[jmorse]

CC @dzhidzhoev for the analysis above -- as mentioned in the PR, I think this is a latent issue being exposed, rather than being a fundamental problem with the DICompositeType changes that are landing/planned.

@dwblaikie does the analysis sounds correct, and as someone more familiar with the dwarf printing backend, is there anything more elegant we can do aside from passing around a "destination unit" every time we create DIEs?

[gregbedwell]

Just to confirm, this does bisect to 432b589, although as @jmorse notes it's probably just a latent bug now being exposed. I will note that it's pretty pervasive in our testing however. Multiple different large codebases that we test with are now failing the DWARF verification checks in our testing when built with Full LTO.

[dzhidzhoev]

@jmorse Thank you for the analysis! It is really helpful.
I think the reason lies in

llvm-project/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp

Line 1840 in 432b589

return &getCU().getOrCreateAbstractSubprogramDIE(SP);

or in DwarfCompileUnit::getOrCreateAbstractSubprogramContextDIE.
I believe this

0x00000029:   DW_TAG_subprogram
                DW_AT_external  (true) 
                DW_AT_inline    (DW_INL_inlined)

0x0000002b:     DW_TAG_class_type
                  DW_AT_byte_size       (0x00)

0x0000002d:     DW_TAG_variable
                  DW_AT_type    (0x0000002e)

0x00000032:     NULL

should be in another CU, as it's specified in LLVM IR

!21 = distinct !DISubprogram(unit: !6, retainedNodes: !22)

[dzhidzhoev]

Just to confirm, this does bisect to 432b589, although as @jmorse notes it's probably just a latent bug now being exposed. I will note that it's pretty pervasive in our testing however. Multiple different large codebases that we test with are now failing the DWARF verification checks in our testing when built with Full LTO.

I've made a fix for the issue #160786. Could you please help with testing it? Or, do the mentioned codebases contain a relatively small open source project which was affected by the issue? So that I could try building it with the fix #160786 in Full LTO mode locally, for the purpose of testing.