[clang][CodeGen] Zero init unspecified fields in initializers in C (#97459)

When an initializer is provided to a variable, the Linux kernel relied
on the compiler to zero-initialize unspecified fields, as clarified in
https://www.spinics.net/lists/netdev/msg1007244.html.

But clang doesn't guarantee this:
1. For a union type, if an empty initializer is given, clang only
   initializes bytes for the first field, left bytes for other (larger)
   fields are marked as undef. Accessing those undef bytes can lead
   to undefined behaviors.
2. For a union type, if an initializer explicitly sets a field, left
   bytes for other (larger) fields are marked as undef.
3. When an initializer is given, clang doesn't zero initialize padding.

So this patch makes the following change:
1. In C, when an initializer is provided for a variable, zero-initialize
   undef and padding fields in the initializer.
2. Document the change in LanguageExtensions.rst.

As suggested in #78034 (comment),
the change isn't required by C23, but it's standards conforming to do
so.

Author: yabinc

clang/docs/LanguageExtensions.rst

@@ -5821,3 +5821,24 @@ specify the starting offset to begin embedding from. The resources is treated
 as being empty if the specified offset is larger than the number of bytes in
 the resource. The offset will be applied *before* any ``limit`` parameters are
 applied.
+
+Union and aggregate initialization in C
+=======================================
+
+In C23 (N2900), when an object is initialized from initializer ``= {}``, all
+elements of arrays, all members of structs, and the first members of unions are
+empty-initialized recursively. In addition, all padding bits are initialized to
+zero.
+
+Clang guarantees the following behaviors:
+
+* ``1:`` Clang supports initializer ``= {}`` mentioned above in all C
+  standards.
+
+* ``2:`` When unions are initialized from initializer ``= {}``, bytes outside
+  of the first members of unions are also initialized to zero.
+
+* ``3:`` When unions, structures and arrays are initialized from initializer
+  ``= { initializer-list }``, all members not explicitly initialized in
+  the initializer list are empty-initialized recursively. In addition, all
+  padding bits are initialized to zero.

clang/lib/CodeGen/CGDecl.cpp

@@ -335,6 +335,13 @@ llvm::Constant *CodeGenModule::getOrCreateStaticVarDecl(
   return Addr;
 }
 
+enum class IsPattern { No, Yes };
+
+static llvm::Constant *replaceUndef(CodeGenModule &CGM, IsPattern isPattern,
+                                    llvm::Constant *constant);
+static llvm::Constant *constWithPadding(CodeGenModule &CGM, IsPattern isPattern,
+                                        llvm::Constant *constant);
+
 /// AddInitializerToStaticVarDecl - Add the initializer for 'D' to the
 /// global variable that has already been created for it.  If the initializer
 /// has a different type than GV does, this may free GV and return a different
@@ -361,6 +368,51 @@ CodeGenFunction::AddInitializerToStaticVarDecl(const VarDecl &D,
     }
     return GV;
   }
+  if (!getLangOpts().CPlusPlus) {
+    // In C23 (N3096) $6.7.10:
+    // """
+    // If any object is initialized with an empty iniitializer, then it is subject to default
+    // initialization:
+    //  - if it is an aggregate, every member is initialized (recursively) according to these rules,
+    //    and any padding is initialized to zero bits;
+    //  - if it is a union, the first named member is initialized (recursively) according to these
+    //    rules, and any padding is initialized to zero bits.
+    //
+    // If the aggregate or union contains elements or members that are aggregates or unions, these
+    // rules apply recursively to the subaggregates or contained unions.
+    //
+    // If there are fewer initializers in a brace-enclosed list than there are elements or members
+    // of an aggregate, or fewer characters in a string literal used to initialize an array of
+    // known size than there are elements in the array, the remainder of the aggregate is subject
+    // to default initialization.
+    // """
+    //
+    // From my understanding, the standard is ambiguous in the following two areas:
+    // 1. For a union type with empty initializer, if the first named member is not the largest
+    // member, then the bytes comes after the first named member but before padding are left
+    // unspecified. An example is:
+    //    union U { int a; long long b;};
+    //    union U u = {};  // The first 4 bytes are 0, but 4-8 bytes are left unspecified.
+    //
+    // 2. It only mentions padding for empty initializer, but doesn't mention padding for a
+    // non empty initialization list. And if the aggregation or union contains elements or members
+    // that are aggregates or unions, and some are non empty initializers, while others are empty
+    // initiailizers, the padding initialization is unclear. An example is:
+    //    struct S1 { int a; long long b; };
+    //    struct S2 { char c; struct S1 s1; };
+    //    // The values for paddings between s2.c and s2.s1.a, between s2.s1.a and s2.s1.b are
+    //    // unclear.
+    //    struct S2 s2 = { 'c' };
+    //
+    // Here we choose to zero initiailize left bytes of a union type. Because projects like the
+    // Linux kernel are relying on this behavior. If we don't explicitly zero initialize them, the
+    // undef values can be optimized to return gabage data.
+    // We also choose to zero initialize paddings for aggregates and unions, no matter they are
+    // initialized by empty initializers or non empty initializers. This can provide a consistent
+    // behavior. So projects like the Linux kernel can rely on it.
+    Init = constWithPadding(CGM, IsPattern::No,
+                            replaceUndef(CGM, IsPattern::No, Init));
+  }
 
 #ifndef NDEBUG
   CharUnits VarSize = CGM.getContext().getTypeSizeInChars(D.getType()) +
@@ -1038,8 +1090,6 @@ static bool shouldSplitConstantStore(CodeGenModule &CGM,
   return false;
 }
 
-enum class IsPattern { No, Yes };
-
 /// Generate a constant filled with either a pattern or zeroes.
 static llvm::Constant *patternOrZeroFor(CodeGenModule &CGM, IsPattern isPattern,
                                         llvm::Type *Ty) {
@@ -1049,9 +1099,6 @@ static llvm::Constant *patternOrZeroFor(CodeGenModule &CGM, IsPattern isPattern,
     return llvm::Constant::getNullValue(Ty);
 }
 
-static llvm::Constant *constWithPadding(CodeGenModule &CGM, IsPattern isPattern,
-                                        llvm::Constant *constant);
-
 /// Helper function for constWithPadding() to deal with padding in structures.
 static llvm::Constant *constStructWithPadding(CodeGenModule &CGM,
                                               IsPattern isPattern,
@@ -1109,6 +1156,9 @@ static llvm::Constant *constWithPadding(CodeGenModule &CGM, IsPattern isPattern,
     if (ZeroInitializer) {
       OpValue = llvm::Constant::getNullValue(ElemTy);
       PaddedOp = constWithPadding(CGM, isPattern, OpValue);
+      // Avoid iterating large arrays with zero initializer when possible.
+      if (PaddedOp->getType() == ElemTy)
+        return constant;
     }
     for (unsigned Op = 0; Op != Size; ++Op) {
       if (!ZeroInitializer) {
@@ -1954,21 +2004,22 @@ void CodeGenFunction::EmitAutoVarInit(const AutoVarEmission &emission) {
       D.mightBeUsableInConstantExpressions(getContext())) {
     assert(!capturedByInit && "constant init contains a capturing block?");
     constant = ConstantEmitter(*this).tryEmitAbstractForInitializer(D);
-    if (constant && !constant->isZeroValue() &&
-        (trivialAutoVarInit !=
-         LangOptions::TrivialAutoVarInitKind::Uninitialized)) {
-      IsPattern isPattern =
-          (trivialAutoVarInit == LangOptions::TrivialAutoVarInitKind::Pattern)
-              ? IsPattern::Yes
-              : IsPattern::No;
-      // C guarantees that brace-init with fewer initializers than members in
-      // the aggregate will initialize the rest of the aggregate as-if it were
-      // static initialization. In turn static initialization guarantees that
-      // padding is initialized to zero bits. We could instead pattern-init if D
-      // has any ImplicitValueInitExpr, but that seems to be unintuitive
-      // behavior.
-      constant = constWithPadding(CGM, IsPattern::No,
-                                  replaceUndef(CGM, isPattern, constant));
+    if (constant && !constant->isZeroValue()) {
+      if (!getLangOpts().CPlusPlus) {
+        // See comment in CodeGenFunction::AddInitializerToStaticVarDecl().
+        constant = constWithPadding(CGM, IsPattern::No,
+                                    replaceUndef(CGM, IsPattern::No, constant));
+      } else if (trivialAutoVarInit !=
+                 LangOptions::TrivialAutoVarInitKind::Uninitialized) {
+        IsPattern isPattern =
+            (trivialAutoVarInit == LangOptions::TrivialAutoVarInitKind::Pattern)
+                ? IsPattern::Yes
+                : IsPattern::No;
+        // We could instead pattern-init padding if D has any
+        // ImplicitValueInitExpr, but that seems to be unintuitive behavior.
+        constant = constWithPadding(CGM, IsPattern::No,
+                                    replaceUndef(CGM, isPattern, constant));
+      }
     }
 
     if (D.getType()->isBitIntType() &&
@@ -2861,3 +2912,9 @@ CodeGenModule::getOMPAllocateAlignment(const VarDecl *VD) {
   }
   return std::nullopt;
 }
+
+llvm::Constant *
+CodeGenModule::zeroInitGlobalVarInitializer(llvm::Constant *Init) {
+  return constWithPadding(*this, IsPattern::No,
+                          replaceUndef(*this, IsPattern::No, Init));
+}

clang/lib/CodeGen/CGExprAgg.cpp

@@ -1720,6 +1720,9 @@ void AggExprEmitter::VisitCXXParenListOrInitListExpr(
   // Prepare a 'this' for CXXDefaultInitExprs.
   CodeGenFunction::FieldConstructionScope FCS(CGF, Dest.getAddress());
 
+  // See comment in CodeGenFunction::AddInitializerToStaticVarDecl().
+  bool ZeroInitPadding = !CGF.getLangOpts().CPlusPlus && !Dest.isZeroed();
+
   if (record->isUnion()) {
     // Only initialize one field of a union. The field itself is
     // specified by the initializer list.
@@ -1748,13 +1751,38 @@ void AggExprEmitter::VisitCXXParenListOrInitListExpr(
       // Default-initialize to null.
       EmitNullInitializationToLValue(FieldLoc);
     }
+    if (ZeroInitPadding) {
+      CharUnits TotalSize = Dest.getPreferredSize(CGF.getContext(), DestLV.getType());
+      CharUnits FieldSize = CGF.getContext().getTypeSizeInChars(FieldLoc.getType());
+      if (FieldSize < TotalSize) {
+        CharUnits LeftSize = TotalSize - FieldSize;
+        llvm::Constant *LeftSizeVal = CGF.Builder.getInt64(LeftSize.getQuantity());
+        Address BaseLoc = Dest.getAddress().withElementType(CGF.Int8Ty);
+        Address LeftLoc = CGF.Builder.CreateConstGEP(BaseLoc, LeftSize.getQuantity());
+        CGF.Builder.CreateMemSet(LeftLoc, CGF.Builder.getInt8(0), LeftSizeVal, false);
+      }
+    }
 
     return;
   }
 
   // Here we iterate over the fields; this makes it simpler to both
   // default-initialize fields and skip over unnamed fields.
+  const ASTRecordLayout &Layout = CGF.getContext().getASTRecordLayout(record);
+  Address BaseLoc = Dest.getAddress().withElementType(CGF.Int8Ty);
+  uint64_t SizeSoFar = 0;
   for (const auto *field : record->fields()) {
+    if (ZeroInitPadding) {
+      unsigned FieldIndex = field->getFieldIndex();
+      uint64_t CurOff = Layout.getFieldOffset(FieldIndex) / CGF.getContext().getCharWidth();
+      if (SizeSoFar < CurOff) {
+        llvm::Constant* PaddingSizeVal = CGF.Builder.getInt64(CurOff - SizeSoFar);
+        Address PaddingLoc = CGF.Builder.CreateConstGEP(BaseLoc, SizeSoFar);
+        CGF.Builder.CreateMemSet(PaddingLoc, CGF.Builder.getInt8(0), PaddingSizeVal, false);
+      }
+      CharUnits FieldSize = CGF.getContext().getTypeSizeInChars(field->getType());
+      SizeSoFar = CurOff + FieldSize.getQuantity();
+    }
     // We're done once we hit the flexible array member.
     if (field->getType()->isIncompleteArrayType())
       break;
@@ -1796,6 +1824,14 @@ void AggExprEmitter::VisitCXXParenListOrInitListExpr(
       }
     }
   }
+  if (ZeroInitPadding) {
+    uint64_t TotalSize = Dest.getPreferredSize(CGF.getContext(), DestLV.getType()).getQuantity();
+    if (SizeSoFar < TotalSize) {
+      llvm::Constant* PaddingSizeVal = CGF.Builder.getInt64(TotalSize - SizeSoFar);
+      Address PaddingLoc = CGF.Builder.CreateConstGEP(BaseLoc, SizeSoFar);
+      CGF.Builder.CreateMemSet(PaddingLoc, CGF.Builder.getInt8(0), PaddingSizeVal, false);
+    }
+  }
 }
 
 void AggExprEmitter::VisitArrayInitLoopExpr(const ArrayInitLoopExpr *E,

clang/lib/CodeGen/CodeGenModule.cpp

@@ -5473,6 +5473,10 @@ void CodeGenModule::EmitGlobalVarDefinition(const VarDecl *D,
         Init = llvm::UndefValue::get(getTypes().ConvertType(T));
       }
     } else {
+      if (!getLangOpts().CPlusPlus) {
+        // See comment in CodeGenFunction::AddInitializerToStaticVarDecl().
+        Initializer = zeroInitGlobalVarInitializer(Initializer);
+      }
       Init = Initializer;
       // We don't need an initializer, so remove the entry for the delayed
       // initializer position (just in case this entry was delayed) if we

clang/lib/CodeGen/CodeGenModule.h

@@ -1866,6 +1866,8 @@ class CodeGenModule : public CodeGenTypeCache {
 
   llvm::Metadata *CreateMetadataIdentifierImpl(QualType T, MetadataTypeMap &Map,
                                                StringRef Suffix);
+
+  llvm::Constant *zeroInitGlobalVarInitializer(llvm::Constant *Init);
 };
 
 }  // end namespace CodeGen

clang/test/CodeGen/2008-07-22-bitfield-init-after-zero-len-array.c

@@ -8,4 +8,4 @@ struct et7 {
   52, 
 };
 
-// CHECK: @yv7 ={{.*}} global %struct.et7 { [0 x float] zeroinitializer, i8 52 }
+// CHECK: @yv7 ={{.*}} global { [0 x float], i8, [3 x i8] } { [0 x float] zeroinitializer, i8 52, [3 x i8] zeroinitializer }

clang/test/CodeGen/2008-08-07-AlignPadding1.c

@@ -20,9 +20,9 @@ struct gc_generation {
 
 #define GEN_HEAD(n) (&generations[n].head)
 
-// The idea is that there are 6 undefs in this structure initializer to cover
+// The idea is that there are 6 zeroinitializers in this structure initializer to cover
 // the padding between elements.
-// CHECK: @generations ={{.*}} global [3 x %struct.gc_generation] [%struct.gc_generation { %union._gc_head { %struct.anon { ptr @generations, ptr @generations, i64 0 }, [8 x i8] undef }, i32 700, i32 0, [8 x i8] undef }, %struct.gc_generation { %union._gc_head { %struct.anon { ptr getelementptr (i8, ptr @generations, i64 48), ptr getelementptr (i8, ptr @generations, i64 48), i64 0 }, [8 x i8] undef }, i32 10, i32 0, [8 x i8] undef }, %struct.gc_generation { %union._gc_head { %struct.anon { ptr getelementptr (i8, ptr @generations, i64 96), ptr getelementptr (i8, ptr @generations, i64 96), i64 0 }, [8 x i8] undef }, i32 10, i32 0, [8 x i8] undef }]
+// CHECK: @generations ={{.*}} global [3 x %struct.gc_generation] [%struct.gc_generation { %union._gc_head { %struct.anon { ptr @generations, ptr @generations, i64 0 }, [8 x i8] zeroinitializer }, i32 700, i32 0, [8 x i8] zeroinitializer }, %struct.gc_generation { %union._gc_head { %struct.anon { ptr getelementptr (i8, ptr @generations, i64 48), ptr getelementptr (i8, ptr @generations, i64 48), i64 0 }, [8 x i8] zeroinitializer }, i32 10, i32 0, [8 x i8] zeroinitializer }, %struct.gc_generation { %union._gc_head { %struct.anon { ptr getelementptr (i8, ptr @generations, i64 96), ptr getelementptr (i8, ptr @generations, i64 96), i64 0 }, [8 x i8] zeroinitializer }, i32 10, i32 0, [8 x i8] zeroinitializer }]
 /* linked lists of container objects */
 struct gc_generation generations[3] = {
         /* PyGC_Head,                           threshold,      count */

clang/test/CodeGen/2009-06-14-anonymous-union-init.c

@@ -7,7 +7,7 @@ struct sysfs_dirent {
 };
 struct sysfs_dirent sysfs_root = { {}, 16877 };
 
-// CHECK: @sysfs_root = {{.*}}global %struct.sysfs_dirent { %union.anon zeroinitializer, i16 16877 }
+// CHECK: @sysfs_root = {{.*}}global { %union.anon, i16, [2 x i8] } { %union.anon zeroinitializer, i16 16877, [2 x i8] zeroinitializer }
 
 struct Foo {
  union { struct empty {} x; };
@@ -16,4 +16,4 @@ struct Foo {
 struct Foo foo = { {}, 16877 };
 
 // EMPTY:      @foo = {{.*}}global %struct.Foo { i16 16877 }
-// EMPTY-MSVC: @foo = {{.*}}global %struct.Foo { [4 x i8] undef, i16 16877 }
+// EMPTY-MSVC: @foo = {{.*}}global %struct.Foo { [4 x i8] zeroinitializer, i16 16877 }

clang/test/CodeGen/64bit-swiftcall.c

@@ -14,8 +14,6 @@
 
 // CHECK-DAG: %struct.atomic_padded = type { { %struct.packed, [7 x i8] } }
 // CHECK-DAG: %struct.packed = type <{ i64, i8 }>
-//
-// CHECK: [[STRUCT2_RESULT:@.*]] = private {{.*}} constant [[STRUCT2_TYPE:%.*]] { i32 0, i8 0, i8 undef, i8 0, i32 0, i32 0 }
 
 /*****************************************************************************/
 /****************************** PARAMETER ABIS *******************************/
@@ -162,8 +160,8 @@ typedef struct {
 } struct_2;
 TEST(struct_2);
 // CHECK-LABEL: define{{.*}} swiftcc { i64, i64 } @return_struct_2() {{.*}}{
-// CHECK:   [[RET:%.*]] = alloca [[STRUCT2_TYPE]], align 4
-// CHECK:   call void @llvm.memcpy{{.*}}({{.*}}[[RET]], {{.*}}[[STRUCT2_RESULT]]
+// CHECK:   [[RET:%.*]] = alloca [[STRUCT2:%.*]], align 4
+// CHECK:   call void @llvm.memset
 // CHECK:   [[GEP0:%.*]] = getelementptr inbounds { i64, i64 }, ptr [[RET]], i32 0, i32 0
 // CHECK:   [[T0:%.*]] = load i64, ptr [[GEP0]], align 4
 // CHECK:   [[GEP1:%.*]] = getelementptr inbounds { i64, i64 }, ptr [[RET]], i32 0, i32 1
@@ -173,15 +171,15 @@ TEST(struct_2);
 // CHECK:   ret { i64, i64 } [[R1]]
 // CHECK: }
 // CHECK-LABEL: define{{.*}} swiftcc void @take_struct_2(i64 %0, i64 %1) {{.*}}{
-// CHECK:   [[V:%.*]] = alloca [[STRUCT:%.*]], align 4
+// CHECK:   [[V:%.*]] = alloca [[STRUCT2]], align 4
 // CHECK:   [[GEP0:%.*]] = getelementptr inbounds { i64, i64 }, ptr [[V]], i32 0, i32 0
 // CHECK:   store i64 %0, ptr [[GEP0]], align 4
 // CHECK:   [[GEP1:%.*]] = getelementptr inbounds { i64, i64 }, ptr [[V]], i32 0, i32 1
 // CHECK:   store i64 %1, ptr [[GEP1]], align 4
 // CHECK:   ret void
 // CHECK: }
 // CHECK-LABEL: define{{.*}} void @test_struct_2() {{.*}} {
-// CHECK:   [[TMP:%.*]] = alloca [[STRUCT2_TYPE]], align 4
+// CHECK:   [[TMP:%.*]] = alloca [[STRUCT2]], align 4
 // CHECK:   [[CALL:%.*]] = call swiftcc { i64, i64 } @return_struct_2()
 // CHECK:   [[GEP:%.*]] = getelementptr inbounds {{.*}} [[TMP]], i32 0, i32 0
 // CHECK:   [[T0:%.*]] = extractvalue { i64, i64 } [[CALL]], 0
@@ -254,7 +252,7 @@ typedef union {
 TEST(union_het_fp)
 // CHECK-LABEL: define{{.*}} swiftcc i64 @return_union_het_fp()
 // CHECK:  [[RET:%.*]] = alloca [[UNION:%.*]], align 8
-// CHECK:  call void @llvm.memcpy{{.*}}(ptr align 8 [[RET]]
+// CHECK:  call void @llvm.memset{{.*}}(ptr align 8 [[RET]]
 // CHECK:  [[GEP:%.*]] = getelementptr inbounds { i64 }, ptr [[RET]], i32 0, i32 0
 // CHECK:  [[R0:%.*]] = load i64, ptr [[GEP]], align 8
 // CHECK:  ret i64 [[R0]]

clang/test/CodeGen/arm-swiftcall.c

@@ -172,7 +172,7 @@ typedef struct {
 TEST(struct_2);
 // CHECK-LABEL: define{{.*}} @return_struct_2()
 // CHECK:   [[RET:%.*]] = alloca [[REC:%.*]], align 4
-// CHECK:   @llvm.memcpy
+// CHECK:   @llvm.memset
 // CHECK:   [[T0:%.*]] = getelementptr inbounds [[AGG:{ i32, i32, float, float }]], ptr [[RET]], i32 0, i32 0
 // CHECK:   [[FIRST:%.*]] = load i32, ptr [[T0]], align 4
 // CHECK:   [[T0:%.*]] = getelementptr inbounds [[AGG]], ptr [[RET]], i32 0, i32 1
@@ -274,7 +274,7 @@ typedef union {
 TEST(union_het_fp)
 // CHECK-LABEL: define{{.*}} @return_union_het_fp()
 // CHECK:   [[RET:%.*]] = alloca [[REC:%.*]], align {{(4|8)}}
-// CHECK:   @llvm.memcpy
+// CHECK:   @llvm.memset
 // CHECK:   [[T0:%.*]] = getelementptr inbounds [[AGG:{ i32, i32 }]], ptr [[RET]], i32 0, i32 0
 // CHECK:   [[FIRST:%.*]] = load i32, ptr [[T0]], align {{(4|8)}}
 // CHECK:   [[T0:%.*]] = getelementptr inbounds [[AGG]], ptr [[RET]], i32 0, i32 1