Skip to content

Commit d7a1335

Browse files
committed
[LV] Ignore some costs when loop gets fully unrolled
When VF has a fixed width and equals the number of iterations, and we are not tail folding by masking, comparison instruction and induction operation will be DCEed later. Ignoring the costs of these instructions improves the cost model.
1 parent 6168739 commit d7a1335

File tree

1 file changed

+20
-0
lines changed

1 file changed

+20
-0
lines changed

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7281,6 +7281,26 @@ LoopVectorizationPlanner::precomputeCosts(VPlan &Plan, ElementCount VF,
72817281
continue;
72827282
IVInsts.push_back(CI);
72837283
}
7284+
7285+
// If the given VF loop gets fully unrolled, ignore the costs of comparison
7286+
// and increment instruction, as they'll get simplified away
7287+
auto TC = CM.PSE.getSE()->getSmallConstantTripCount(OrigLoop);
7288+
auto *Cmp = OrigLoop->getLatchCmpInst();
7289+
if (Cmp && VF.isFixed() && VF.getFixedValue() == TC) {
7290+
CostCtx.SkipCostComputation.insert(Cmp);
7291+
for (Instruction *IVInst : IVInsts) {
7292+
bool IsSimplifiedAway = true;
7293+
for (auto *UIV : IVInst->users()) {
7294+
if (!Legal->isInductionVariable(UIV) && UIV != Cmp) {
7295+
IsSimplifiedAway = false;
7296+
break;
7297+
}
7298+
}
7299+
if (IsSimplifiedAway)
7300+
CostCtx.SkipCostComputation.insert(IVInst);
7301+
}
7302+
}
7303+
72847304
for (Instruction *IVInst : IVInsts) {
72857305
if (CostCtx.skipCostComputation(IVInst, VF.isVector()))
72867306
continue;

0 commit comments

Comments
 (0)