[CIR] Add X86 prefetch builtins #168051

HendrikHuebner · 2025-11-14T12:35:16Z

Upstreams the sse2 prefetch builtin and adds two more builtins not present in the incubator repo (prefetchh and prefetchw)

llvmbot · 2025-11-14T12:35:47Z

@llvm/pr-subscribers-clang

@llvm/pr-subscribers-clangir

Author: Hendrik Hübner (HendrikHuebner)

Changes

Upstreams the sse2 prefetch builtin and adds two more builtins not present in the incubator repo (prefetchh and prefetchw)

Full diff: https://github.com/llvm/llvm-project/pull/168051.diff

5 Files Affected:

(modified) clang/include/clang/CIR/Dialect/IR/CIROps.td (+2-2)
(modified) clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp (+30)
(added) clang/test/CIR/CodeGen/X86/prefetchw-builtin.c (+36)
(modified) clang/test/CIR/CodeGen/X86/sse-builtins.c (+18)
(modified) clang/test/CIR/CodeGen/X86/sse2-builtins.c (-1)

diff --git a/clang/include/clang/CIR/Dialect/IR/CIROps.td b/clang/include/clang/CIR/Dialect/IR/CIROps.td
index 2124b1dc62a81..219846ec7a884 100644
--- a/clang/include/clang/CIR/Dialect/IR/CIROps.td
+++ b/clang/include/clang/CIR/Dialect/IR/CIROps.td
@@ -4116,7 +4116,7 @@ def CIR_PrefetchOp : CIR_Op<"prefetch"> {
     $locality is a temporal locality specifier ranging from (0) - no locality,
     to (3) - extremely local, keep in cache. If $locality is not present, the
     default value is 3.
-    
+
     $isWrite specifies whether the prefetch is for a 'read' or 'write'. If
     $isWrite is not specified, it means that prefetch is prepared for 'read'.
   }];
@@ -4150,7 +4150,7 @@ def CIR_ObjSizeOp : CIR_Op<"objsize", [Pure]> {
     When the `min` attribute is present, the operation returns the minimum
     guaranteed accessible size. When absent (max mode), it returns the maximum
     possible object size. Corresponds to `llvm.objectsize`'s `min` argument.
-    
+
     The `dynamic` attribute determines if the value should be evaluated at
     runtime. Corresponds to `llvm.objectsize`'s `dynamic` argument.
 
diff --git a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
index 2d6cf30fa2ded..6f23f2be79e91 100644
--- a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
+++ b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
@@ -21,6 +21,11 @@
 using namespace clang;
 using namespace clang::CIRGen;
 
+/// Get integer from a mlir::Value that is an int constant or a constant op.
+static int64_t getIntValueFromConstOp(mlir::Value val) {
+  return val.getDefiningOp<cir::ConstantOp>().getIntValue().getSExtValue();
+}
+
 template <typename... Operands>
 static mlir::Value emitIntrinsicCallOp(CIRGenFunction &cgf, const CallExpr *e,
                                        const std::string &str,
@@ -34,6 +39,28 @@ static mlir::Value emitIntrinsicCallOp(CIRGenFunction &cgf, const CallExpr *e,
       .getResult();
 }
 
+static mlir::Value emitPrefetch(CIRGenFunction &cgf, unsigned builtinID,
+                                const CallExpr *e,
+                                mlir::Value &addr, int64_t hint) {
+  CIRGenBuilderTy &builder = cgf.getBuilder();
+  mlir::Location location = cgf.getLoc(e->getExprLoc());
+  mlir::Type voidTy = builder.getVoidTy();
+  mlir::Value address = builder.createPtrBitcast(addr, voidTy);
+  bool isWrite{};
+  int locality{};
+
+  if (builtinID == X86::BI_mm_prefetch) {
+    isWrite = (hint >> 2) & 0x1;
+    locality = hint & 0x3;
+  } else {
+    isWrite = (builtinID == X86::BI_m_prefetchw);
+    locality = 0x3;
+  }
+
+  cir::PrefetchOp::create(builder, location, address, locality, isWrite);
+  return {};
+}
+
 mlir::Value CIRGenFunction::emitX86BuiltinExpr(unsigned builtinID,
                                                const CallExpr *e) {
   if (builtinID == Builtin::BI__builtin_cpu_is) {
@@ -87,6 +114,9 @@ mlir::Value CIRGenFunction::emitX86BuiltinExpr(unsigned builtinID,
   case X86::BI_mm_sfence:
     return emitIntrinsicCallOp(*this, e, "x86.sse.sfence", voidTy);
   case X86::BI_mm_prefetch:
+  case X86::BI_m_prefetch:
+  case X86::BI_m_prefetchw:
+      return emitPrefetch(*this, builtinID, e, ops[0], getIntValueFromConstOp(ops[1]));
   case X86::BI__rdtsc:
   case X86::BI__builtin_ia32_rdtscp:
   case X86::BI__builtin_ia32_lzcnt_u16:
diff --git a/clang/test/CIR/CodeGen/X86/prefetchw-builtin.c b/clang/test/CIR/CodeGen/X86/prefetchw-builtin.c
new file mode 100644
index 0000000000000..fbf7894ba69b2
--- /dev/null
+++ b/clang/test/CIR/CodeGen/X86/prefetchw-builtin.c
@@ -0,0 +1,36 @@
+
+// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +sse -fclangir -emit-cir -o %t.cir -Wall -Werror
+// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s
+// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +sse -fclangir -emit-llvm -o %t.ll -Wall -Werror
+// RUN: FileCheck --check-prefixes=LLVM --input-file=%t.ll %s
+
+// RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +sse -fno-signed-char -fclangir -emit-cir -o %t.cir -Wall -Werror
+// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s
+// RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +sse -fclangir -emit-llvm -o %t.ll -Wall -Werror
+// RUN: FileCheck --check-prefixes=LLVM --input-file=%t.ll %s
+
+// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +sse -emit-llvm -o - -Wall -Werror | FileCheck %s -check-prefix=OGCG
+// RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +sse -emit-llvm -o - -Wall -Werror | FileCheck %s -check-prefix=OGCG
+
+
+#include <x86intrin.h>
+
+void test_m_prefetch(void *p) {
+  // CIR-LABEL: test_m_prefetch
+  // LLVM-LABEL: test_m_prefetch
+  // OGCG-LABEL: test_m_prefetch
+  return _m_prefetch(p);
+  // CIR: cir.prefetch read locality(0) %{{.*}} : !cir.ptr<!void>
+  // LLVM: call void @llvm.prefetch.p0(ptr {{.*}}, i32 0, i32 0, i32 1)
+  // OGCG: call void @llvm.prefetch.p0(ptr {{.*}}, i32 0, i32 0, i32 1)
+}
+
+void test_m_prefetch_w(void *p) {
+  // CIR-LABEL: test_m_prefetch_w
+  // LLVM-LABEL: test_m_prefetch_w
+  // OGCG-LABEL: test_m_prefetch_w
+  return _m_prefetchw(p);
+  // CIR: cir.prefetch write locality(0) %{{.*}} : !cir.ptr<!void>
+  // LLVM: call void @llvm.prefetch.p0(ptr {{.*}}, i32 0, i32 0, i32 1)
+  // OGCG: call void @llvm.prefetch.p0(ptr {{.*}}, i32 0, i32 0, i32 1)
+}
diff --git a/clang/test/CIR/CodeGen/X86/sse-builtins.c b/clang/test/CIR/CodeGen/X86/sse-builtins.c
index 3a61018741958..07c586d7c0b8c 100644
--- a/clang/test/CIR/CodeGen/X86/sse-builtins.c
+++ b/clang/test/CIR/CodeGen/X86/sse-builtins.c
@@ -26,3 +26,21 @@ void test_mm_sfence(void) {
   // LLVM: call void @llvm.x86.sse.sfence()
   // OGCG: call void @llvm.x86.sse.sfence()
 }
+
+void test_mm_prefetch(char const* p) {
+  // CIR-LABEL: test_mm_prefetch
+  // LLVM-LABEL: test_mm_prefetch
+  _mm_prefetch(p, 0);
+  // CIR: cir.prefetch read locality(0) %{{.*}} : !cir.ptr<!void>
+  // LLVM: call void @llvm.prefetch.p0(ptr {{.*}}, i32 0, i32 0, i32 1)
+  // OGCG: call void @llvm.prefetch.p0(ptr {{.*}}, i32 0, i32 0, i32 1)
+}
+
+void test_mm_prefetch_local(char const* p) {
+  // CIR-LABEL: test_mm_prefetch_local
+  // LLVM-LABEL: test_mm_prefetch_local
+  _mm_prefetch(p, 3);
+  // CIR: cir.prefetch read locality(3) %{{.*}} : !cir.ptr<!void>
+  // LLVM: call void @llvm.prefetch.p0(ptr {{.*}}, i32 0, i32 3, i32 1)
+  // OGCG: call void @llvm.prefetch.p0(ptr {{.*}}, i32 0, i32 3, i32 1)
+}
diff --git a/clang/test/CIR/CodeGen/X86/sse2-builtins.c b/clang/test/CIR/CodeGen/X86/sse2-builtins.c
index 144ca143fbf15..3429eee093f6a 100644
--- a/clang/test/CIR/CodeGen/X86/sse2-builtins.c
+++ b/clang/test/CIR/CodeGen/X86/sse2-builtins.c
@@ -16,7 +16,6 @@
 
 #include <immintrin.h>
 
-
 void test_mm_clflush(void* A) {
   // CIR-LABEL: test_mm_clflush
   // LLVM-LABEL: test_mm_clflush

HendrikHuebner · 2025-11-14T12:36:26Z

clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp

+  }
+
+  cir::PrefetchOp::create(builder, location, address, locality, isWrite);
+  return {};


Here I'm not sure if this is correct: Can I somehow convert the PrefetchOp into an mlir::Value and return it?

I requested this change from the incubator because I thought we should be using CIR-specific operations when we have them, and in this case the CIR operation is exactly equivalent to the LLVM intrinsic.

I see that our handling of BI__builtin_prefetch returns RValue::get(nullptr) from emitBuiltinExpr, which is what we'd get by returning {} here.

You'd have to add a result to the operation. Incubator has support for case X86::BI_mm_prefetch: which should be somewhat similar and shared here, it's using cir.llvm.intrinsic directly, which has a result. If there are reasons to deviate here they should also be replayed in the incubator.

Just returning {} a.k.a nullptr does not work here I think because the caller assumes the builtin was not emitted and throws NYI. Instead, how about we make it return an optional<mlir::Value>? If the optional is present the builtin was generated, and if the value is nullptr, there is no return value. I think void type values are also a thing in mlir, so maybe we could use that as well? If you prefer that, how would I construct one?

I don't think this is going to work because we return {} for all of the cases where we have emitted an NYI diagnostic and for the default, where we have done nothing and don't intend to do anything.

I replaced the return type of the caller with std::optional<mlir::Value> to handle this case. I tested it locally and it still emits the NYI message for missing builtins, and as you can see the tests are passing.

github-actions · 2025-11-14T12:37:00Z

✅ With the latest revision this PR passed the C/C++ code formatter.

clang/include/clang/CIR/Dialect/IR/CIROps.td

bcardosolopes · 2025-11-14T22:53:53Z

clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp

+  }
+
+  cir::PrefetchOp::create(builder, location, address, locality, isWrite);
+  return {};


You'd have to add a result to the operation. Incubator has support for case X86::BI_mm_prefetch: which should be somewhat similar and shared here, it's using cir.llvm.intrinsic directly, which has a result. If there are reasons to deviate here they should also be replayed in the incubator.

andykaylor · 2025-11-14T21:34:12Z

clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp

+  }
+
+  cir::PrefetchOp::create(builder, location, address, locality, isWrite);
+  return {};


I requested this change from the incubator because I thought we should be using CIR-specific operations when we have them, and in this case the CIR operation is exactly equivalent to the LLVM intrinsic.

I see that our handling of BI__builtin_prefetch returns RValue::get(nullptr) from emitBuiltinExpr, which is what we'd get by returning {} here.

clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp

clang/test/CIR/CodeGen/X86/prefetchw-builtin.c

clang/test/CIR/CodeGen/X86/sse-builtins.c

github-actions · 2025-11-17T23:33:19Z

🐧 Linux x64 Test Results

112048 tests passed
4077 tests skipped

bcardosolopes · 2025-11-18T19:09:07Z

clang/lib/CIR/CodeGen/CIRGenBuiltin.cpp


  // Now see if we can emit a target-specific builtin.
-  if (mlir::Value v = emitTargetBuiltinExpr(builtinID, e, returnValue)) {
+  std::optional<mlir::Value> valueOpt =


This is just working around the fact that emitTargetBuiltinExpr is not returning an RValue already. It's a bit more work but you should change emitTargetBuiltinExpr to return an RValue instead. You can then change evalKind below and use isScalar(), etc methods from RValue.

Also, whenever you need to emit RValue::get(nullptr), please use RValue::getIgnored() instead.

I updated the code. I would still like to return the optional for emitX86BuiltinExpr, because this keeps the code a little cleaner since we don't have to call RValue::get for each of the cases in the big switch statement.

HendrikHuebner requested review from andykaylor, bcardosolopes, lanza and xlauko as code owners November 14, 2025 12:35

llvmbot added clang Clang issues not falling into any other category ClangIR Anything related to the ClangIR project labels Nov 14, 2025

HendrikHuebner commented Nov 14, 2025

View reviewed changes

bcardosolopes requested changes Nov 14, 2025

View reviewed changes

andykaylor reviewed Nov 15, 2025

View reviewed changes

HendrikHuebner added 4 commits November 15, 2025 14:56

Add prefetch builtins

02cf87c

Format

84f845d

Feedback

fd266d4

Formatting

30e6801

HendrikHuebner force-pushed the upstream-prefetch branch from 93578ca to 30e6801 Compare November 15, 2025 16:16

revert formatting changes

e8d557f

HendrikHuebner requested a review from bcardosolopes November 17, 2025 22:44

HendrikHuebner added 2 commits November 17, 2025 23:45

revert formatting changes

fd000c8

add OGCG label

34a531d

HendrikHuebner requested a review from andykaylor November 17, 2025 22:47

bcardosolopes reviewed Nov 18, 2025

View reviewed changes

HendrikHuebner and others added 5 commits November 19, 2025 01:55

feedback

58af624

Merge branch 'main' into upstream-prefetch

8e56a36

Fix merge issue

e8c7671

formatting

13d5276

format

2efbd35

HendrikHuebner requested a review from bcardosolopes November 20, 2025 10:50

[CIR] Add X86 prefetch builtins #168051

Are you sure you want to change the base?

[CIR] Add X86 prefetch builtins #168051

Conversation

HendrikHuebner commented Nov 14, 2025

Uh oh!

llvmbot commented Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HendrikHuebner Nov 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HendrikHuebner Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🐧 Linux x64 Test Results

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

llvmbot commented Nov 14, 2025 •

edited

Loading

HendrikHuebner Nov 15, 2025 •

edited

Loading

HendrikHuebner Nov 18, 2025 •

edited

Loading

github-actions bot commented Nov 14, 2025 •

edited

Loading

github-actions bot commented Nov 17, 2025 •

edited

Loading