Skip to content

Conversation

@tbaederr
Copy link
Contributor

@tbaederr tbaederr commented Mar 1, 2025

When creating descriptor for array element types, we only save the original source, e.g. int[2][2][2]. So later calls to getType() of the element descriptors will also return int[2][2][2], instead of e.g. int[2][2] for the second dimension.
Fix this by explicitly tracking the array types.
The last attached test case used to have an lvalue offset of 32 instead of 24.

We should do this for more descriptor types though and not just composite array, but I'm leaving that to a later patch.

@llvmbot llvmbot added clang Clang issues not falling into any other category clang:frontend Language frontend issues, e.g. anything involving "Sema" labels Mar 1, 2025
@llvmbot
Copy link
Member

llvmbot commented Mar 1, 2025

@llvm/pr-subscribers-clang

Author: Timm Baeder (tbaederr)

Changes

When creating descriptor for array element types, we only save the original source, e.g. int[2][2][2]. So later calls to getType() of the element descriptors will also return int[2][2][2], instead of e.g. int[2][2] for the second dimension.
Fix this by explicitly tracking the array types.
The last attached test case used to have an lvalue offset of 32 instead of 24.

We should do this for more desriptor types though and not just composite array, but I'm leaving that to a later patch.


Full diff: https://github.com/llvm/llvm-project/pull/129376.diff

5 Files Affected:

  • (modified) clang/lib/AST/ByteCode/Descriptor.cpp (+6-2)
  • (modified) clang/lib/AST/ByteCode/Descriptor.h (+4-2)
  • (modified) clang/lib/AST/ByteCode/DynamicAllocator.cpp (+3-1)
  • (modified) clang/lib/AST/ByteCode/Program.cpp (+1-1)
  • (modified) clang/unittests/AST/ByteCode/toAPValue.cpp (+67)
diff --git a/clang/lib/AST/ByteCode/Descriptor.cpp b/clang/lib/AST/ByteCode/Descriptor.cpp
index 6017f6dd61cb3..bcd9f6f3d078a 100644
--- a/clang/lib/AST/ByteCode/Descriptor.cpp
+++ b/clang/lib/AST/ByteCode/Descriptor.cpp
@@ -367,10 +367,12 @@ Descriptor::Descriptor(const DeclTy &D, PrimType Type, MetadataSize MD,
 }
 
 /// Arrays of composite elements.
-Descriptor::Descriptor(const DeclTy &D, const Descriptor *Elem, MetadataSize MD,
+Descriptor::Descriptor(const DeclTy &D, const Type *SourceTy,
+                       const Descriptor *Elem, MetadataSize MD,
                        unsigned NumElems, bool IsConst, bool IsTemporary,
                        bool IsMutable)
-    : Source(D), ElemSize(Elem->getAllocSize() + sizeof(InlineDescriptor)),
+    : Source(D), SourceType(SourceTy),
+      ElemSize(Elem->getAllocSize() + sizeof(InlineDescriptor)),
       Size(ElemSize * NumElems), MDSize(MD.value_or(0)),
       AllocSize(std::max<size_t>(alignof(void *), Size) + MDSize),
       ElemDesc(Elem), IsConst(IsConst), IsMutable(IsMutable),
@@ -410,6 +412,8 @@ Descriptor::Descriptor(const DeclTy &D)
 }
 
 QualType Descriptor::getType() const {
+  if (SourceType)
+    return QualType(SourceType, 0);
   if (const auto *D = asValueDecl())
     return D->getType();
   if (const auto *T = dyn_cast_if_present<TypeDecl>(asDecl()))
diff --git a/clang/lib/AST/ByteCode/Descriptor.h b/clang/lib/AST/ByteCode/Descriptor.h
index 01fa4b198de67..b2e9f1b6bded4 100644
--- a/clang/lib/AST/ByteCode/Descriptor.h
+++ b/clang/lib/AST/ByteCode/Descriptor.h
@@ -124,6 +124,7 @@ struct Descriptor final {
 private:
   /// Original declaration, used to emit the error message.
   const DeclTy Source;
+  const Type *SourceType = nullptr;
   /// Size of an element, in host bytes.
   const unsigned ElemSize;
   /// Size of the storage, in host bytes.
@@ -186,8 +187,9 @@ struct Descriptor final {
              bool IsTemporary, UnknownSize);
 
   /// Allocates a descriptor for an array of composites.
-  Descriptor(const DeclTy &D, const Descriptor *Elem, MetadataSize MD,
-             unsigned NumElems, bool IsConst, bool IsTemporary, bool IsMutable);
+  Descriptor(const DeclTy &D, const Type *SourceTy, const Descriptor *Elem,
+             MetadataSize MD, unsigned NumElems, bool IsConst, bool IsTemporary,
+             bool IsMutable);
 
   /// Allocates a descriptor for an array of composites of unknown size.
   Descriptor(const DeclTy &D, const Descriptor *Elem, MetadataSize MD,
diff --git a/clang/lib/AST/ByteCode/DynamicAllocator.cpp b/clang/lib/AST/ByteCode/DynamicAllocator.cpp
index 3ef8c2e1f3e7c..728bd75d7d141 100644
--- a/clang/lib/AST/ByteCode/DynamicAllocator.cpp
+++ b/clang/lib/AST/ByteCode/DynamicAllocator.cpp
@@ -57,8 +57,10 @@ Block *DynamicAllocator::allocate(const Descriptor *ElementDesc,
   assert(ElementDesc->getMetadataSize() == 0);
   // Create a new descriptor for an array of the specified size and
   // element type.
+  // FIXME: Pass proper element type.
   const Descriptor *D = allocateDescriptor(
-      ElementDesc->asExpr(), ElementDesc, Descriptor::InlineDescMD, NumElements,
+      ElementDesc->asExpr(), nullptr, ElementDesc, Descriptor::InlineDescMD,
+      NumElements,
       /*IsConst=*/false, /*IsTemporary=*/false, /*IsMutable=*/false);
   return allocate(D, EvalID, AllocForm);
 }
diff --git a/clang/lib/AST/ByteCode/Program.cpp b/clang/lib/AST/ByteCode/Program.cpp
index 0754e259b7cb3..cc2cfc9b03b41 100644
--- a/clang/lib/AST/ByteCode/Program.cpp
+++ b/clang/lib/AST/ByteCode/Program.cpp
@@ -419,7 +419,7 @@ Descriptor *Program::createDescriptor(const DeclTy &D, const Type *Ty,
         unsigned ElemSize = ElemDesc->getAllocSize() + sizeof(InlineDescriptor);
         if (std::numeric_limits<unsigned>::max() / ElemSize <= NumElems)
           return {};
-        return allocateDescriptor(D, ElemDesc, MDSize, NumElems, IsConst,
+        return allocateDescriptor(D, Ty, ElemDesc, MDSize, NumElems, IsConst,
                                   IsTemporary, IsMutable);
       }
     }
diff --git a/clang/unittests/AST/ByteCode/toAPValue.cpp b/clang/unittests/AST/ByteCode/toAPValue.cpp
index cd62338ee23c1..c7df97c81b16f 100644
--- a/clang/unittests/AST/ByteCode/toAPValue.cpp
+++ b/clang/unittests/AST/ByteCode/toAPValue.cpp
@@ -254,3 +254,70 @@ TEST(ToAPValue, MemberPointers) {
     ASSERT_EQ(A.getKind(), APValue::MemberPointer);
   }
 }
+
+/// Test the various toAPValue implementations.
+TEST(ToAPValue, Comparison) {
+  constexpr char Code[] =
+      "constexpr int GI = 12;\n"
+      "constexpr const int *PI = &GI;\n"
+      "struct S{int a; };\n"
+      "constexpr S GS[] = {{}, {}};\n"
+      "constexpr const S* OS = GS + 1;\n"
+      "constexpr S DS = *OS;\n"
+      "constexpr int IA[2][2][2] = {{{1, 2}, {3, 4}}, {{5, 6}, {7, 8}}};\n"
+      "constexpr const int *PIA = IA[1][1];\n";
+
+  for (const char *Arg : {"", "-fexperimental-new-constant-interpreter"}) {
+    auto AST = tooling::buildASTFromCodeWithArgs(Code, {Arg});
+
+    // auto &ASTCtx = AST->getASTContext();
+    auto getDecl = [&](const char *Name) -> const VarDecl * {
+      auto Nodes =
+          match(valueDecl(hasName(Name)).bind("var"), AST->getASTContext());
+      assert(Nodes.size() == 1);
+      const auto *D = Nodes[0].getNodeAs<ValueDecl>("var");
+      assert(D);
+      return cast<VarDecl>(D);
+    };
+
+    {
+      const VarDecl *GIDecl = getDecl("GI");
+      const APValue *V = GIDecl->evaluateValue();
+      ASSERT_TRUE(V->isInt());
+    }
+
+    {
+      const VarDecl *GIDecl = getDecl("GI");
+      const VarDecl *PIDecl = getDecl("PI");
+      const APValue *V = PIDecl->evaluateValue();
+      ASSERT_TRUE(V->isLValue());
+      ASSERT_TRUE(V->hasLValuePath());
+      ASSERT_EQ(V->getLValueBase().get<const ValueDecl *>(), GIDecl);
+      ASSERT_EQ(V->getLValuePath().size(), 0u);
+    }
+
+    {
+      const APValue *V = getDecl("OS")->evaluateValue();
+      ASSERT_TRUE(V->isLValue());
+      ASSERT_EQ(V->getLValueOffset().getQuantity(), (unsigned)sizeof(int));
+      ASSERT_TRUE(V->hasLValuePath());
+      ASSERT_EQ(V->getLValuePath().size(), 1u);
+      ASSERT_EQ(V->getLValuePath()[0].getAsArrayIndex(), 1u);
+    }
+
+    {
+      const APValue *V = getDecl("DS")->evaluateValue();
+      ASSERT_TRUE(V->isStruct());
+    }
+    {
+      const APValue *V = getDecl("PIA")->evaluateValue();
+      ASSERT_TRUE(V->isLValue());
+      ASSERT_TRUE(V->hasLValuePath());
+      ASSERT_EQ(V->getLValuePath().size(), 3u);
+      ASSERT_EQ(V->getLValuePath()[0].getAsArrayIndex(), 1u);
+      ASSERT_EQ(V->getLValuePath()[1].getAsArrayIndex(), 1u);
+      ASSERT_EQ(V->getLValuePath()[2].getAsArrayIndex(), 0u);
+      ASSERT_EQ(V->getLValueOffset().getQuantity(), (unsigned)sizeof(int) * 6);
+    }
+  }
+}

@tbaederr tbaederr changed the title [clang][bytecode] Explicitly composite array descriptor types [clang][bytecode] Explicit composite array descriptor types Mar 2, 2025
When creating descriptor for array element types, we only save the
original source, e.g. int[2][2][2]. So later calls to getType() of the
element descriptors will also return int[2][2][2], instead of e.g.
int[2][2] for the second dimension.
Fix this by explicitly tracking the array types.
The last attached test case used to have an lvalue offset of 32 instead
of 24.

We should do this for more desriptor types though and not just composite
array, but I'm leaving that to a later patch.
@tbaederr tbaederr merged commit 2c1e9f1 into llvm:main Mar 2, 2025
11 checks passed
@llvm-ci
Copy link
Collaborator

llvm-ci commented Mar 2, 2025

LLVM Buildbot has detected a new failure on builder clang-s390x-linux running on systemz-1 while building clang at step 5 "ninja check 1".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/42/builds/3497

Here is the relevant piece of the build log for the reference
Step 5 (ninja check 1) failure: stage 1 checked (failure)
******************** TEST 'libFuzzer-s390x-default-Linux :: fuzzer-timeout.test' FAILED ********************
Exit Code: 1

Command Output (stderr):
--
RUN: at line 1: /home/uweigand/sandbox/buildbot/clang-s390x-linux/stage1/./bin/clang    -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta   --driver-mode=g++ -O2 -gline-tables-only -fsanitize=address,fuzzer -I/home/uweigand/sandbox/buildbot/clang-s390x-linux/llvm/compiler-rt/lib/fuzzer  /home/uweigand/sandbox/buildbot/clang-s390x-linux/llvm/compiler-rt/test/fuzzer/TimeoutTest.cpp -o /home/uweigand/sandbox/buildbot/clang-s390x-linux/stage1/runtimes/runtimes-bins/compiler-rt/test/fuzzer/S390XDefaultLinuxConfig/Output/fuzzer-timeout.test.tmp-TimeoutTest
+ /home/uweigand/sandbox/buildbot/clang-s390x-linux/stage1/./bin/clang -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta --driver-mode=g++ -O2 -gline-tables-only -fsanitize=address,fuzzer -I/home/uweigand/sandbox/buildbot/clang-s390x-linux/llvm/compiler-rt/lib/fuzzer /home/uweigand/sandbox/buildbot/clang-s390x-linux/llvm/compiler-rt/test/fuzzer/TimeoutTest.cpp -o /home/uweigand/sandbox/buildbot/clang-s390x-linux/stage1/runtimes/runtimes-bins/compiler-rt/test/fuzzer/S390XDefaultLinuxConfig/Output/fuzzer-timeout.test.tmp-TimeoutTest
RUN: at line 2: /home/uweigand/sandbox/buildbot/clang-s390x-linux/stage1/./bin/clang    -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta   --driver-mode=g++ -O2 -gline-tables-only -fsanitize=address,fuzzer -I/home/uweigand/sandbox/buildbot/clang-s390x-linux/llvm/compiler-rt/lib/fuzzer  /home/uweigand/sandbox/buildbot/clang-s390x-linux/llvm/compiler-rt/test/fuzzer/TimeoutEmptyTest.cpp -o /home/uweigand/sandbox/buildbot/clang-s390x-linux/stage1/runtimes/runtimes-bins/compiler-rt/test/fuzzer/S390XDefaultLinuxConfig/Output/fuzzer-timeout.test.tmp-TimeoutEmptyTest
+ /home/uweigand/sandbox/buildbot/clang-s390x-linux/stage1/./bin/clang -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta --driver-mode=g++ -O2 -gline-tables-only -fsanitize=address,fuzzer -I/home/uweigand/sandbox/buildbot/clang-s390x-linux/llvm/compiler-rt/lib/fuzzer /home/uweigand/sandbox/buildbot/clang-s390x-linux/llvm/compiler-rt/test/fuzzer/TimeoutEmptyTest.cpp -o /home/uweigand/sandbox/buildbot/clang-s390x-linux/stage1/runtimes/runtimes-bins/compiler-rt/test/fuzzer/S390XDefaultLinuxConfig/Output/fuzzer-timeout.test.tmp-TimeoutEmptyTest
RUN: at line 3: not  /home/uweigand/sandbox/buildbot/clang-s390x-linux/stage1/runtimes/runtimes-bins/compiler-rt/test/fuzzer/S390XDefaultLinuxConfig/Output/fuzzer-timeout.test.tmp-TimeoutTest -timeout=1 2>&1 | FileCheck /home/uweigand/sandbox/buildbot/clang-s390x-linux/llvm/compiler-rt/test/fuzzer/fuzzer-timeout.test --check-prefix=TimeoutTest
+ not /home/uweigand/sandbox/buildbot/clang-s390x-linux/stage1/runtimes/runtimes-bins/compiler-rt/test/fuzzer/S390XDefaultLinuxConfig/Output/fuzzer-timeout.test.tmp-TimeoutTest -timeout=1
+ FileCheck /home/uweigand/sandbox/buildbot/clang-s390x-linux/llvm/compiler-rt/test/fuzzer/fuzzer-timeout.test --check-prefix=TimeoutTest
RUN: at line 12: not  /home/uweigand/sandbox/buildbot/clang-s390x-linux/stage1/runtimes/runtimes-bins/compiler-rt/test/fuzzer/S390XDefaultLinuxConfig/Output/fuzzer-timeout.test.tmp-TimeoutTest -timeout=1 /home/uweigand/sandbox/buildbot/clang-s390x-linux/llvm/compiler-rt/test/fuzzer/hi.txt 2>&1 | FileCheck /home/uweigand/sandbox/buildbot/clang-s390x-linux/llvm/compiler-rt/test/fuzzer/fuzzer-timeout.test --check-prefix=SingleInputTimeoutTest
+ FileCheck /home/uweigand/sandbox/buildbot/clang-s390x-linux/llvm/compiler-rt/test/fuzzer/fuzzer-timeout.test --check-prefix=SingleInputTimeoutTest
+ not /home/uweigand/sandbox/buildbot/clang-s390x-linux/stage1/runtimes/runtimes-bins/compiler-rt/test/fuzzer/S390XDefaultLinuxConfig/Output/fuzzer-timeout.test.tmp-TimeoutTest -timeout=1 /home/uweigand/sandbox/buildbot/clang-s390x-linux/llvm/compiler-rt/test/fuzzer/hi.txt
RUN: at line 16: /home/uweigand/sandbox/buildbot/clang-s390x-linux/stage1/runtimes/runtimes-bins/compiler-rt/test/fuzzer/S390XDefaultLinuxConfig/Output/fuzzer-timeout.test.tmp-TimeoutTest -timeout=1 -timeout_exitcode=0
+ /home/uweigand/sandbox/buildbot/clang-s390x-linux/stage1/runtimes/runtimes-bins/compiler-rt/test/fuzzer/S390XDefaultLinuxConfig/Output/fuzzer-timeout.test.tmp-TimeoutTest -timeout=1 -timeout_exitcode=0
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 2269359701
INFO: Loaded 1 modules   (13 inline 8-bit counters): 13 [0x2aa172d0e80, 0x2aa172d0e8d), 
INFO: Loaded 1 PC tables (13 PCs): 13 [0x2aa172d0e90,0x2aa172d0f60), 
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: A corpus is not provided, starting from an empty corpus
#2	INITED cov: 2 ft: 2 corp: 1/1b exec/s: 0 rss: 31Mb
#544	NEW    cov: 3 ft: 3 corp: 2/2b lim: 8 exec/s: 0 rss: 32Mb L: 1/1 MS: 2 ChangeByte-ChangeBit-
#551	NEW    cov: 4 ft: 4 corp: 3/4b lim: 8 exec/s: 0 rss: 32Mb L: 2/2 MS: 2 CrossOver-ShuffleBytes-
#42808	NEW    cov: 5 ft: 5 corp: 4/45b lim: 421 exec/s: 0 rss: 35Mb L: 41/41 MS: 2 CopyPart-InsertRepeatedBytes-
#42819	REDUCE cov: 5 ft: 5 corp: 4/37b lim: 421 exec/s: 0 rss: 35Mb L: 33/33 MS: 1 EraseBytes-
#42862	REDUCE cov: 5 ft: 5 corp: 4/26b lim: 421 exec/s: 0 rss: 35Mb L: 22/22 MS: 3 ChangeByte-ChangeBit-EraseBytes-
#42998	REDUCE cov: 5 ft: 5 corp: 4/16b lim: 421 exec/s: 0 rss: 35Mb L: 12/12 MS: 1 EraseBytes-
#43424	REDUCE cov: 5 ft: 5 corp: 4/10b lim: 421 exec/s: 0 rss: 35Mb L: 6/6 MS: 1 EraseBytes-
#43596	REDUCE cov: 5 ft: 5 corp: 4/7b lim: 421 exec/s: 0 rss: 35Mb L: 3/3 MS: 2 ChangeBinInt-EraseBytes-
#43657	REDUCE cov: 6 ft: 6 corp: 5/9b lim: 421 exec/s: 0 rss: 35Mb L: 2/3 MS: 1 EraseBytes-
ALARM: working on the last Unit for 1 seconds
       and the timeout value is 1 (use -timeout=N to change)
MS: 3 CrossOver-ShuffleBytes-InsertByte-; base unit: 94dd9e08c129c785f7f256e82fbe0a30e6d1ae40
0x48,0x69,0x21,
Hi!
artifact_prefix='./'; Test unit written to ./timeout-c0a0ad26a634840c67a210fefdda76577b03a111
Base64: SGkh
==3730256== ERROR: libFuzzer: timeout after 1 seconds
AddressSanitizer:DEADLYSIGNAL
=================================================================
AddressSanitizer:DEADLYSIGNAL
=================================================================
AddressSanitizer: CHECK failed: asan_report.cpp:199 "((current_error_.kind)) == ((kErrorKindInvalid))" (0x1, 0x0) (tid=3730256)
    <empty stack>

MS: 3 CrossOver-ShuffleBytes-InsertByte-; base unit: 94dd9e08c129c785f7f256e82fbe0a30e6d1ae40
0x48,0x69,0x21,
...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

clang:frontend Language frontend issues, e.g. anything involving "Sema" clang Clang issues not falling into any other category

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants