[LoopVectorize] Make collectInLoopReductions more efficient

[david-arm]

We call collectInLoopReductions in multiple places asking
the same question with exactly the same answer. For
example, this was being called from a loop in
calculateRegisterUsage and this patch hoists the call out
to above the loop. In addition I've changed
collectInLoopReductions so that it bails out if we've
already built up a list.

[llvmbot]

@llvm/pr-subscribers-vectorizers

Author: David Sherwood (david-arm)

Changes

We call collectInLoopReductions in multiple places asking
the same question with exactly the same answer. For
example, this was being called from a loop in
calculateRegisterUsage and this patch hoists the call out
to above the loop. In addition I've changed
collectInLoopReductions so that it bails out if we've
already built up a list.

---

Full diff: https://github.com/llvm/llvm-project/pull/126769.diff

1 Files Affected:

• (modified) llvm/lib/Transforms/Vectorize/LoopVectorize.cpp (+6-2)

diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
index f2241be60ce05..a25bde75b465d 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
@@ -5260,6 +5260,8 @@ LoopVectorizationCostModel::calculateRegisterUsage(ArrayRef<ElementCount> VFs) {
     return TTICapture.getRegUsageForType(VectorType::get(Ty, VF));
   };
 
+  collectInLoopReductions();
+
   for (unsigned int Idx = 0, Sz = IdxToInstr.size(); Idx < Sz; ++Idx) {
     Instruction *I = IdxToInstr[Idx];
 
@@ -5276,8 +5278,6 @@ LoopVectorizationCostModel::calculateRegisterUsage(ArrayRef<ElementCount> VFs) {
     if (ValuesToIgnore.count(I))
       continue;
 
-    collectInLoopReductions();
-
     // For each VF find the maximum usage of registers.
     for (unsigned J = 0, E = VFs.size(); J < E; ++J) {
       // Count the number of registers used, per register class, given all open
@@ -7008,6 +7008,10 @@ void LoopVectorizationCostModel::collectValuesToIgnore() {
 }
 
 void LoopVectorizationCostModel::collectInLoopReductions() {
+  // Avoid duplicating work finding in-loop reductions.
+  if (!InLoopReductions.empty())
+    return;
+
   for (const auto &Reduction : Legal->getReductionVars()) {
     PHINode *Phi = Reduction.first;
     const RecurrenceDescriptor &RdxDesc = Reduction.second;

[llvmbot]

@llvm/pr-subscribers-llvm-transforms

Author: David Sherwood (david-arm)

Changes

We call collectInLoopReductions in multiple places asking
the same question with exactly the same answer. For
example, this was being called from a loop in
calculateRegisterUsage and this patch hoists the call out
to above the loop. In addition I've changed
collectInLoopReductions so that it bails out if we've
already built up a list.

---

Full diff: https://github.com/llvm/llvm-project/pull/126769.diff

1 Files Affected:

• (modified) llvm/lib/Transforms/Vectorize/LoopVectorize.cpp (+6-2)

diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
index f2241be60ce05..a25bde75b465d 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
@@ -5260,6 +5260,8 @@ LoopVectorizationCostModel::calculateRegisterUsage(ArrayRef<ElementCount> VFs) {
     return TTICapture.getRegUsageForType(VectorType::get(Ty, VF));
   };
 
+  collectInLoopReductions();
+
   for (unsigned int Idx = 0, Sz = IdxToInstr.size(); Idx < Sz; ++Idx) {
     Instruction *I = IdxToInstr[Idx];
 
@@ -5276,8 +5278,6 @@ LoopVectorizationCostModel::calculateRegisterUsage(ArrayRef<ElementCount> VFs) {
     if (ValuesToIgnore.count(I))
       continue;
 
-    collectInLoopReductions();
-
     // For each VF find the maximum usage of registers.
     for (unsigned J = 0, E = VFs.size(); J < E; ++J) {
       // Count the number of registers used, per register class, given all open
@@ -7008,6 +7008,10 @@ void LoopVectorizationCostModel::collectValuesToIgnore() {
 }
 
 void LoopVectorizationCostModel::collectInLoopReductions() {
+  // Avoid duplicating work finding in-loop reductions.
+  if (!InLoopReductions.empty())
+    return;
+
   for (const auto &Reduction : Legal->getReductionVars()) {
     PHINode *Phi = Reduction.first;
     const RecurrenceDescriptor &RdxDesc = Reduction.second;

[fhahn]

LGTM thanks