Wrapping test need to be done only if runtime checks present #149856

annamthomas · 2025-07-21T17:31:45Z

JFI - the failing tests are all because of the change in order of the checks: we are now checking if loop can vectorize in presence of memory before checking early-exit vectorization.

Is this something that can be done in interim until we improve the code for evaluatePtrAddRecAtMaxBTCWillNotWrap?

Am I missing any other reason to have that API?

Code form for llvm#128061 (comment).

llvmbot · 2025-07-21T17:32:17Z

@llvm/pr-subscribers-llvm-transforms
@llvm/pr-subscribers-vectorizers

@llvm/pr-subscribers-llvm-analysis

Author: None (annamthomas)

Changes

Code form for
#128061 (comment).

JFI - the failing tests are all because of the change in order of the checks: we are now checking if loop can vectorize in presence of memory before checking early-exit vectorization.

Is this something that can be done in interim until we improve the code for evaluatePtrAddRecAtMaxBTCWillNotWrap?

Am I missing any other reason to have that API?

Full diff: https://github.com/llvm/llvm-project/pull/149856.diff

6 Files Affected:

(modified) llvm/include/llvm/Analysis/Loads.h (+4-2)
(modified) llvm/include/llvm/Analysis/LoopAccessAnalysis.h (+1-1)
(modified) llvm/include/llvm/Transforms/Vectorize/LoopVectorizationLegality.h (+1-1)
(modified) llvm/lib/Analysis/Loads.cpp (+9-5)
(modified) llvm/lib/Analysis/LoopAccessAnalysis.cpp (+4-3)
(modified) llvm/lib/Transforms/Vectorize/LoopVectorizationLegality.cpp (+13-10)

diff --git a/llvm/include/llvm/Analysis/Loads.h b/llvm/include/llvm/Analysis/Loads.h
index 84564563de8e3..8d44f20b7a7d3 100644
--- a/llvm/include/llvm/Analysis/Loads.h
+++ b/llvm/include/llvm/Analysis/Loads.h
@@ -83,13 +83,15 @@ LLVM_ABI bool isSafeToLoadUnconditionally(
 LLVM_ABI bool isDereferenceableAndAlignedInLoop(
     LoadInst *LI, Loop *L, ScalarEvolution &SE, DominatorTree &DT,
     AssumptionCache *AC = nullptr,
-    SmallVectorImpl<const SCEVPredicate *> *Predicates = nullptr);
+    SmallVectorImpl<const SCEVPredicate *> *Predicates = nullptr,
+    bool ShouldCheckWrapping = true);
 
 /// Return true if the loop \p L cannot fault on any iteration and only
 /// contains read-only memory accesses.
 LLVM_ABI bool isDereferenceableReadOnlyLoop(
     Loop *L, ScalarEvolution *SE, DominatorTree *DT, AssumptionCache *AC,
-    SmallVectorImpl<const SCEVPredicate *> *Predicates = nullptr);
+    SmallVectorImpl<const SCEVPredicate *> *Predicates = nullptr,
+    bool ShouldCheckWrapping = true);
 
 /// Return true if we know that executing a load from this value cannot trap.
 ///
diff --git a/llvm/include/llvm/Analysis/LoopAccessAnalysis.h b/llvm/include/llvm/Analysis/LoopAccessAnalysis.h
index 73bfe1aabb4e0..05280eee3881c 100644
--- a/llvm/include/llvm/Analysis/LoopAccessAnalysis.h
+++ b/llvm/include/llvm/Analysis/LoopAccessAnalysis.h
@@ -922,7 +922,7 @@ LLVM_ABI std::pair<const SCEV *, const SCEV *> getStartAndEndForAccess(
     const Loop *Lp, const SCEV *PtrExpr, Type *AccessTy, const SCEV *BTC,
     const SCEV *MaxBTC, ScalarEvolution *SE,
     DenseMap<std::pair<const SCEV *, Type *>,
-             std::pair<const SCEV *, const SCEV *>> *PointerBounds);
+             std::pair<const SCEV *, const SCEV *>> *PointerBounds, bool ShouldCheckWrapping = true);
 
 class LoopAccessInfoManager {
   /// The cache.
diff --git a/llvm/include/llvm/Transforms/Vectorize/LoopVectorizationLegality.h b/llvm/include/llvm/Transforms/Vectorize/LoopVectorizationLegality.h
index cba37363d0474..1e63859356e09 100644
--- a/llvm/include/llvm/Transforms/Vectorize/LoopVectorizationLegality.h
+++ b/llvm/include/llvm/Transforms/Vectorize/LoopVectorizationLegality.h
@@ -542,7 +542,7 @@ class LoopVectorizationLegality {
   /// The list above is not based on theoretical limitations of vectorization,
   /// but simply a statement that more work is needed to support these
   /// additional cases safely.
-  bool isVectorizableEarlyExitLoop();
+  bool isVectorizableEarlyExitLoop(const bool NeedRuntimeChecks);
 
   /// Return true if all of the instructions in the block can be speculatively
   /// executed, and record the loads/stores that require masking.
diff --git a/llvm/lib/Analysis/Loads.cpp b/llvm/lib/Analysis/Loads.cpp
index 393f2648de3c9..f7f512bf712b2 100644
--- a/llvm/lib/Analysis/Loads.cpp
+++ b/llvm/lib/Analysis/Loads.cpp
@@ -288,7 +288,8 @@ static bool AreEquivalentAddressValues(const Value *A, const Value *B) {
 
 bool llvm::isDereferenceableAndAlignedInLoop(
     LoadInst *LI, Loop *L, ScalarEvolution &SE, DominatorTree &DT,
-    AssumptionCache *AC, SmallVectorImpl<const SCEVPredicate *> *Predicates) {
+    AssumptionCache *AC, SmallVectorImpl<const SCEVPredicate *> *Predicates,
+    bool ShouldCheckWrapping) {
   const Align Alignment = LI->getAlign();
   auto &DL = LI->getDataLayout();
   Value *Ptr = LI->getPointerOperand();
@@ -341,8 +342,9 @@ bool llvm::isDereferenceableAndAlignedInLoop(
             ? SE.getPredicatedConstantMaxBackedgeTakenCount(L, *Predicates)
             : SE.getConstantMaxBackedgeTakenCount(L);
   }
-  const auto &[AccessStart, AccessEnd] = getStartAndEndForAccess(
-      L, PtrScev, LI->getType(), BECount, MaxBECount, &SE, nullptr);
+  const auto &[AccessStart, AccessEnd] =
+      getStartAndEndForAccess(L, PtrScev, LI->getType(), BECount, MaxBECount,
+                              &SE, nullptr, ShouldCheckWrapping);
   if (isa<SCEVCouldNotCompute>(AccessStart) ||
       isa<SCEVCouldNotCompute>(AccessEnd))
     return false;
@@ -850,11 +852,13 @@ bool llvm::canReplacePointersIfEqual(const Value *From, const Value *To,
 
 bool llvm::isDereferenceableReadOnlyLoop(
     Loop *L, ScalarEvolution *SE, DominatorTree *DT, AssumptionCache *AC,
-    SmallVectorImpl<const SCEVPredicate *> *Predicates) {
+    SmallVectorImpl<const SCEVPredicate *> *Predicates,
+    bool ShouldCheckWrapping) {
   for (BasicBlock *BB : L->blocks()) {
     for (Instruction &I : *BB) {
       if (auto *LI = dyn_cast<LoadInst>(&I)) {
-        if (!isDereferenceableAndAlignedInLoop(LI, L, *SE, *DT, AC, Predicates))
+        if (!isDereferenceableAndAlignedInLoop(LI, L, *SE, *DT, AC, Predicates,
+                                               ShouldCheckWrapping))
           return false;
       } else if (I.mayReadFromMemory() || I.mayWriteToMemory() || I.mayThrow())
         return false;
diff --git a/llvm/lib/Analysis/LoopAccessAnalysis.cpp b/llvm/lib/Analysis/LoopAccessAnalysis.cpp
index f3a32d3055edb..32b580668e68c 100644
--- a/llvm/lib/Analysis/LoopAccessAnalysis.cpp
+++ b/llvm/lib/Analysis/LoopAccessAnalysis.cpp
@@ -273,7 +273,8 @@ std::pair<const SCEV *, const SCEV *> llvm::getStartAndEndForAccess(
     const Loop *Lp, const SCEV *PtrExpr, Type *AccessTy, const SCEV *BTC,
     const SCEV *MaxBTC, ScalarEvolution *SE,
     DenseMap<std::pair<const SCEV *, Type *>,
-             std::pair<const SCEV *, const SCEV *>> *PointerBounds) {
+             std::pair<const SCEV *, const SCEV *>> *PointerBounds,
+    bool ShouldCheckWrapping) {
   std::pair<const SCEV *, const SCEV *> *PtrBoundsPair;
   if (PointerBounds) {
     auto [Iter, Ins] = PointerBounds->insert(
@@ -308,8 +309,8 @@ std::pair<const SCEV *, const SCEV *> llvm::getStartAndEndForAccess(
       // sets ScEnd to the maximum unsigned value for the type. Note that LAA
       // separately checks that accesses cannot not wrap, so unsigned max
       // represents an upper bound.
-      if (evaluatePtrAddRecAtMaxBTCWillNotWrap(AR, MaxBTC, EltSizeSCEV, *SE,
-                                               DL)) {
+      if (!ShouldCheckWrapping || evaluatePtrAddRecAtMaxBTCWillNotWrap(
+                                      AR, MaxBTC, EltSizeSCEV, *SE, DL)) {
         ScEnd = AR->evaluateAtIteration(MaxBTC, *SE);
       } else {
         ScEnd = SE->getAddExpr(
diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorizationLegality.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorizationLegality.cpp
index 969d225c6ef2e..8225177d05786 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorizationLegality.cpp
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorizationLegality.cpp
@@ -1643,7 +1643,8 @@ bool LoopVectorizationLegality::canVectorizeLoopNestCFG(
   return Result;
 }
 
-bool LoopVectorizationLegality::isVectorizableEarlyExitLoop() {
+bool LoopVectorizationLegality::isVectorizableEarlyExitLoop(
+    const bool NeedRuntimeChecks) {
   BasicBlock *LatchBB = TheLoop->getLoopLatch();
   if (!LatchBB) {
     reportVectorizationFailure("Loop does not have a latch",
@@ -1851,6 +1852,16 @@ bool LoopVectorizationLegality::canVectorize(bool UseVPlanNativePath) {
       return false;
   }
 
+  // Go over each instruction and look at memory deps.
+  if (!canVectorizeMemory()) {
+    LLVM_DEBUG(dbgs() << "LV: Can't vectorize due to memory conflicts\n");
+    if (DoExtraAnalysis)
+      Result = false;
+    else
+      return false;
+  }
+
+  auto NeedRuntimeChecks = LAI->getRuntimePointerChecking()->Need;
   if (isa<SCEVCouldNotCompute>(PSE.getBackedgeTakenCount())) {
     if (TheLoop->getExitingBlock()) {
       reportVectorizationFailure("Cannot vectorize uncountable loop",
@@ -1860,7 +1871,7 @@ bool LoopVectorizationLegality::canVectorize(bool UseVPlanNativePath) {
       else
         return false;
     } else {
-      if (!isVectorizableEarlyExitLoop()) {
+      if (!isVectorizableEarlyExitLoop(NeedRuntimeChecks)) {
         UncountableEdge = std::nullopt;
         if (DoExtraAnalysis)
           Result = false;
@@ -1870,14 +1881,6 @@ bool LoopVectorizationLegality::canVectorize(bool UseVPlanNativePath) {
     }
   }
 
-  // Go over each instruction and look at memory deps.
-  if (!canVectorizeMemory()) {
-    LLVM_DEBUG(dbgs() << "LV: Can't vectorize due to memory conflicts\n");
-    if (DoExtraAnalysis)
-      Result = false;
-    else
-      return false;
-  }
 
   if (Result) {
     LLVM_DEBUG(dbgs() << "LV: We can vectorize this loop"

github-actions · 2025-07-21T17:35:15Z

⚠️ C/C++ code formatter, clang-format found issues in your code. ⚠️

You can test this locally with the following command:

git-clang-format --diff HEAD~1 HEAD --extensions cpp,h -- llvm/include/llvm/Analysis/Loads.h llvm/include/llvm/Analysis/LoopAccessAnalysis.h llvm/include/llvm/Transforms/Vectorize/LoopVectorizationLegality.h llvm/lib/Analysis/Loads.cpp llvm/lib/Analysis/LoopAccessAnalysis.cpp llvm/lib/Transforms/Vectorize/LoopVectorizationLegality.cpp

View the diff from clang-format here.

diff --git a/llvm/include/llvm/Analysis/LoopAccessAnalysis.h b/llvm/include/llvm/Analysis/LoopAccessAnalysis.h
index 05280eee3..67c5a7ea1 100644
--- a/llvm/include/llvm/Analysis/LoopAccessAnalysis.h
+++ b/llvm/include/llvm/Analysis/LoopAccessAnalysis.h
@@ -922,7 +922,8 @@ LLVM_ABI std::pair<const SCEV *, const SCEV *> getStartAndEndForAccess(
     const Loop *Lp, const SCEV *PtrExpr, Type *AccessTy, const SCEV *BTC,
     const SCEV *MaxBTC, ScalarEvolution *SE,
     DenseMap<std::pair<const SCEV *, Type *>,
-             std::pair<const SCEV *, const SCEV *>> *PointerBounds, bool ShouldCheckWrapping = true);
+             std::pair<const SCEV *, const SCEV *>> *PointerBounds,
+    bool ShouldCheckWrapping = true);
 
 class LoopAccessInfoManager {
   /// The cache.
diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorizationLegality.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorizationLegality.cpp
index 8225177d0..136b056f9 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorizationLegality.cpp
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorizationLegality.cpp
@@ -1881,7 +1881,6 @@ bool LoopVectorizationLegality::canVectorize(bool UseVPlanNativePath) {
     }
   }
 
-
   if (Result) {
     LLVM_DEBUG(dbgs() << "LV: We can vectorize this loop"
                       << (LAI->getRuntimePointerChecking()->Need

annamthomas · 2025-07-21T20:44:51Z

Response to original comment shows this may not end up being correct. Closing this off.

Wrapping test need to be done only if runtime checks present

93736a3

Code form for llvm#128061 (comment).

annamthomas requested review from fhahn and david-arm July 21, 2025 17:31

llvmbot added vectorizers llvm:analysis Includes value tracking, cost tables and constant folding llvm:transforms labels Jul 21, 2025

annamthomas closed this Jul 21, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Wrapping test need to be done only if runtime checks present #149856

Wrapping test need to be done only if runtime checks present #149856

Uh oh!

annamthomas commented Jul 21, 2025

Uh oh!

llvmbot commented Jul 21, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jul 21, 2025

Uh oh!

annamthomas commented Jul 21, 2025

Uh oh!

Uh oh!

Wrapping test need to be done only if runtime checks present #149856

Wrapping test need to be done only if runtime checks present #149856

Uh oh!

Conversation

annamthomas commented Jul 21, 2025

Uh oh!

llvmbot commented Jul 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 21, 2025

Uh oh!

annamthomas commented Jul 21, 2025

Uh oh!

Uh oh!

llvmbot commented Jul 21, 2025 •

edited

Loading