Skip to content

Commit 6cb5a1e

Browse files
committed
[LV] Ignore some costs when loop gets fully unrolled
When VF has a fixed width and equals the number of iterations, and we are not tail folding by masking, comparison instruction and induction operation will be DCEed later. Ignoring the costs of these instructions improves the cost model.
1 parent 0bb7bd4 commit 6cb5a1e

File tree

1 file changed

+20
-0
lines changed

1 file changed

+20
-0
lines changed

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7281,6 +7281,26 @@ LoopVectorizationPlanner::precomputeCosts(VPlan &Plan, ElementCount VF,
72817281
continue;
72827282
IVInsts.push_back(CI);
72837283
}
7284+
7285+
// If the given VF loop gets fully unrolled, ignore the costs of comparison
7286+
// and increment instruction, as they'll get simplified away
7287+
auto TC = CM.PSE.getSE()->getSmallConstantTripCount(OrigLoop);
7288+
auto *Cmp = OrigLoop->getLatchCmpInst();
7289+
if (Cmp && VF.isFixed() && VF.getFixedValue() == TC) {
7290+
CostCtx.SkipCostComputation.insert(Cmp);
7291+
for (Instruction *IVInst : IVInsts) {
7292+
bool IsSimplifiedAway = true;
7293+
for (auto *UIV : IVInst->users()) {
7294+
if (!Legal->isInductionVariable(UIV) && UIV != Cmp) {
7295+
IsSimplifiedAway = false;
7296+
break;
7297+
}
7298+
}
7299+
if (IsSimplifiedAway)
7300+
CostCtx.SkipCostComputation.insert(IVInst);
7301+
}
7302+
}
7303+
72847304
for (Instruction *IVInst : IVInsts) {
72857305
if (CostCtx.skipCostComputation(IVInst, VF.isVector()))
72867306
continue;

0 commit comments

Comments
 (0)