Skip to content

Commit 79cb34c

Browse files
committed
[LV] Ignore some costs when loop gets fully unrolled
When VF has a fixed width and equals the number of iterations, and we are not tail folding by masking, comparison instruction and induction operation will be DCEed later. Ignoring the costs of these instructions improves the cost model.
1 parent af31aa4 commit 79cb34c

File tree

1 file changed

+20
-0
lines changed

1 file changed

+20
-0
lines changed

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7248,6 +7248,26 @@ LoopVectorizationPlanner::precomputeCosts(VPlan &Plan, ElementCount VF,
72487248
continue;
72497249
IVInsts.push_back(CI);
72507250
}
7251+
7252+
// If the given VF loop gets fully unrolled, ignore the costs of comparison
7253+
// and increment instruction, as they'll get simplified away
7254+
auto TC = CM.PSE.getSE()->getSmallConstantTripCount(OrigLoop);
7255+
auto *Cmp = OrigLoop->getLatchCmpInst();
7256+
if (Cmp && VF.isFixed() && VF.getFixedValue() == TC) {
7257+
CostCtx.SkipCostComputation.insert(Cmp);
7258+
for (Instruction *IVInst : IVInsts) {
7259+
bool IsSimplifiedAway = true;
7260+
for (auto *UIV : IVInst->users()) {
7261+
if (!Legal->isInductionVariable(UIV) && UIV != Cmp) {
7262+
IsSimplifiedAway = false;
7263+
break;
7264+
}
7265+
}
7266+
if (IsSimplifiedAway)
7267+
CostCtx.SkipCostComputation.insert(IVInst);
7268+
}
7269+
}
7270+
72517271
for (Instruction *IVInst : IVInsts) {
72527272
if (CostCtx.skipCostComputation(IVInst, VF.isVector()))
72537273
continue;

0 commit comments

Comments
 (0)