Skip to content

Commit 671072e

Browse files
authored
[AArch64] Unrolling of loops with vector instructions. (#147420)
This patch permits loops with vector instructions to be unrolled. Today there is an early exit in `getUnrollingPreferences()` of AArch64 targets if a vector instruction is observed in any of the loop blocks. This patch fixes that so common loops like this one get a chance to be unrolled: void saxpy (float * dst, const float * src, const float a, const int len) { float32x4_t * vdst = (float32x4_t *)dst; float32x4_t * vsrc = (float32x4_t *)src; float32x4_t vk = vdupq_n_f32(a); for (int i = 0; i < (len >> 2); i++) { vdst[i] = vaddq_f32(vdst[i], vmulq_f32(vsrc[i], vk)); } } Auto-vectorized loops are still not unrolled, unless they were not interleaved when vectorized. The provided test case shows the enhancement on top of runtime/partial unrolling, depending on the CPU. PR: #147420
1 parent 641ff6d commit 671072e

File tree

2 files changed

+501
-6
lines changed

2 files changed

+501
-6
lines changed

llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -4906,15 +4906,14 @@ void AArch64TTIImpl::getUnrollingPreferences(
49064906
// Disable partial & runtime unrolling on -Os.
49074907
UP.PartialOptSizeThreshold = 0;
49084908

4909+
// No need to unroll auto-vectorized loops
4910+
if (findStringMetadataForLoop(L, "llvm.loop.isvectorized"))
4911+
return;
4912+
49094913
// Scan the loop: don't unroll loops with calls as this could prevent
4910-
// inlining. Don't unroll vector loops either, as they don't benefit much from
4911-
// unrolling.
4914+
// inlining.
49124915
for (auto *BB : L->getBlocks()) {
49134916
for (auto &I : *BB) {
4914-
// Don't unroll vectorised loop.
4915-
if (I.getType()->isVectorTy())
4916-
return;
4917-
49184917
if (isa<CallBase>(I)) {
49194918
if (isa<CallInst>(I) || isa<InvokeInst>(I))
49204919
if (const Function *F = cast<CallBase>(I).getCalledFunction())

0 commit comments

Comments
 (0)