[ConstantFolding] Fold scalable get_active_lane_masks #156659

MDevereau · 2025-09-03T12:53:18Z

Scalable get_active_lane_mask intrinsics with a range of 0 can be lowered to zeroinitializer. This helps remove no-op scalable masked stores and loads.

Scalable get_active_lane_mask intrinsics with a range of 0 can be lowered to zeroinitializer. This helps remove no-op scalable masked stores and loads. When the second operand is 0, this cannot be done (see llvm#152140)

david-arm · 2025-09-03T12:55:00Z

llvm/lib/Analysis/ConstantFolding.cpp


    return ConstantInt::getFalse(SVTy);
  }
+  case Intrinsic::get_active_lane_mask: {


I think we probably want to do this for fixed-width as well.

As it stands, ConstantFolding can already handle this for fixed-width. Scalable seems to have been left behind: https://godbolt.org/z/snd7M5oer

I see yeah, and as it stands the code in ConstantFoldFixedVectorCall is broken because it should return poison if Op1 is zero. Once the LangRef is fixed we can always revisit this.

david-arm

LGTM!

david-arm · 2025-09-03T14:56:51Z

llvm/lib/Analysis/ConstantFolding.cpp


    return ConstantInt::getFalse(SVTy);
  }
+  case Intrinsic::get_active_lane_mask: {


I see yeah, and as it stands the code in ConstantFoldFixedVectorCall is broken because it should return poison if Op1 is zero. Once the LangRef is fixed we can always revisit this.

davemgreen · 2025-09-04T06:46:37Z

llvm/lib/Analysis/ConstantFolding.cpp

+  case Intrinsic::get_active_lane_mask: {
+    auto Op0 = cast<ConstantInt>(Operands[0])->getValue();
+    auto Op1 = cast<ConstantInt>(Operands[1])->getValue();
+    if ((Op0.uge(Op1) && (!Op1.isZero())))


I don't think this needs a check for Op1.isZero. It is perfectly valid to refine poison to zero.

It would be converting the whilelo -> get_active_lane_mask that would be invalid, if zero produced poison, as it converts from zero->poison.

(This also has many more brackets than necessary, you can drop a few).

Doesn't that mean that if add nsw i64 %v1, %v2 or getelementptr inbounds ..., etc. are known to produce poison we can just return any other value that we think makes sense too? At the IR level the choice of a zero value here is completely arbitrary (since the LangRef explicitly says the result is poison, not zero), but what if a completely different IR pass decides it can prove Op1 is zero via other means (via computed bits analysis, etc) and decides to return another completely arbitrary value following the same logic, e.g. 1? Wouldn't we then be in a situation where two calls to get.active.lane.mask with an operand of 0 return different non-poison results?

Yep, poison propagates so if it is not frozen it can have different values at different places.
https://llvm.org/docs/LangRef.html#poison-values
Producing all-ones would be valid (if it was poison), but not as useful.

Makes sense, I've removed the Op1.isZero check

Fair enough, thanks for explaining @davemgreen!

I'm not sure the poison conversation is necessarily correct. Whilst you are allowed to remove sources of poison I think when the IR is statically known to be poison the poison should be propagated.

@nikic / @fhahn : Can either of you confirm one way or the other?

@MDevereau: I've just landed #152140, which removes the poison corner case.

@paulwalker-arm Thanks. What I gather from that is it's now OK to assume active_lane_mask(0, 0) is an all false predicate just as active_lane_mask(3, 3) is and that no further changes are needed in this PR.

llvmbot · 2025-09-04T11:59:46Z

@llvm/pr-subscribers-llvm-analysis

@llvm/pr-subscribers-llvm-transforms

Author: Matthew Devereau (MDevereau)

Changes

Scalable get_active_lane_mask intrinsics with a range of 0 can be lowered to zeroinitializer. This helps remove no-op scalable masked stores and loads.

When the second operand is 0, this cannot be done (see #152140)

Full diff: https://github.com/llvm/llvm-project/pull/156659.diff

2 Files Affected:

(modified) llvm/lib/Analysis/ConstantFolding.cpp (+7)
(modified) llvm/test/Transforms/InstSimplify/ConstProp/active-lane-mask.ll (+33)

diff --git a/llvm/lib/Analysis/ConstantFolding.cpp b/llvm/lib/Analysis/ConstantFolding.cpp
index 2148431c1acce..67e6be5b70cb6 100644
--- a/llvm/lib/Analysis/ConstantFolding.cpp
+++ b/llvm/lib/Analysis/ConstantFolding.cpp
@@ -4238,6 +4238,13 @@ static Constant *ConstantFoldScalableVectorCall(
 
     return ConstantInt::getFalse(SVTy);
   }
+  case Intrinsic::get_active_lane_mask: {
+    auto Op0 = cast<ConstantInt>(Operands[0])->getValue();
+    auto Op1 = cast<ConstantInt>(Operands[1])->getValue();
+    if (Op0.uge(Op1))
+      return ConstantVector::getNullValue(SVTy);
+    break;
+  }
   default:
     break;
   }
diff --git a/llvm/test/Transforms/InstSimplify/ConstProp/active-lane-mask.ll b/llvm/test/Transforms/InstSimplify/ConstProp/active-lane-mask.ll
index a904e697cc975..ed26deb58eae4 100644
--- a/llvm/test/Transforms/InstSimplify/ConstProp/active-lane-mask.ll
+++ b/llvm/test/Transforms/InstSimplify/ConstProp/active-lane-mask.ll
@@ -307,6 +307,39 @@ entry:
   ret <4 x float> %var33
 }
 
+define <vscale x 4 x i1> @nxv4i1_12_12() {
+; CHECK-LABEL: @nxv4i1_12_12(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    ret <vscale x 4 x i1> zeroinitializer
+;
+entry:
+  %mask = call <vscale x 4 x i1> @llvm.get.active.lane.mask.nxv4i1.i32(i32 12, i32 12)
+  ret <vscale x 4 x i1> %mask
+}
+
+define <vscale x 4 x i1> @nxv4i1_8_4() {
+; CHECK-LABEL: @nxv4i1_8_4(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    ret <vscale x 4 x i1> zeroinitializer
+;
+entry:
+  %mask = call <vscale x 4 x i1> @llvm.get.active.lane.mask.nxv4i1.i32(i32 8, i32 4)
+  ret <vscale x 4 x i1> %mask
+}
+
+define <vscale x 16 x i1> @nxv16i1_0_0() {
+; CHECK-LABEL: @nxv16i1_0_0(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    ret <vscale x 16 x i1> zeroinitializer
+;
+entry:
+  %mask = call <vscale x 16 x i1> @llvm.get.active.lane.mask.nxv16i1.i64(i64 0, i64 0)
+  ret <vscale x 16 x i1> %mask
+}
+
 declare <4 x i1> @llvm.get.active.lane.mask.v4i1.i32(i32, i32)
 declare <8 x i1> @llvm.get.active.lane.mask.v8i1.i32(i32, i32)
 declare <16 x i1> @llvm.get.active.lane.mask.v16i1.i32(i32, i32)
+
+declare <vscale x 4 x i1> @llvm.get.active.lane.mask.nxv4i1.i32(i32, i32)
+declare <vscale x 16 x i1> @llvm.get.active.lane.mask.nxv4i1.i64(i64, i64)

Check if operands are ConstantInt to avoid crashing on constant expression after #156659.

fhahn · 2025-09-11T22:19:14Z

llvm/lib/Analysis/ConstantFolding.cpp

+    auto Op0 = cast<ConstantInt>(Operands[0])->getValue();
+    auto Op1 = cast<ConstantInt>(Operands[1])->getValue();


This is currently causing crashes when the operands are constant expressions. I added checks to. make sure the operands are actually ConstantInts in 607a813 to avoid a revert.

Thanks for that, much appreciated.

…active_lane_m. Check if operands are ConstantInt to avoid crashing on constant expression after llvm/llvm-project#156659.

[ConstantFolding] Fold scalable get_active_lane_masks

49833a2

Scalable get_active_lane_mask intrinsics with a range of 0 can be lowered to zeroinitializer. This helps remove no-op scalable masked stores and loads. When the second operand is 0, this cannot be done (see llvm#152140)

MDevereau requested review from davemgreen and david-arm September 3, 2025 12:53

david-arm reviewed Sep 3, 2025

View reviewed changes

david-arm approved these changes Sep 3, 2025

View reviewed changes

davemgreen reviewed Sep 4, 2025

View reviewed changes

Remove Op1.isZero() condition

042b029

llvmbot added llvm:instcombine Covers the InstCombine, InstSimplify and AggressiveInstCombine passes llvm:analysis Includes value tracking, cost tables and constant folding llvm:transforms labels Sep 4, 2025

MDevereau merged commit 75099c2 into llvm:main Sep 11, 2025
12 checks passed

fhahn added a commit that referenced this pull request Sep 11, 2025

[ConstFold] Don't crash on ConstantExprs when folding get_active_lane_m.

607a813

Check if operands are ConstantInt to avoid crashing on constant expression after #156659.

fhahn reviewed Sep 11, 2025

View reviewed changes

		auto Op0 = cast<ConstantInt>(Operands[0])->getValue();
		auto Op1 = cast<ConstantInt>(Operands[1])->getValue();

[ConstantFolding] Fold scalable get_active_lane_masks #156659

[ConstantFolding] Fold scalable get_active_lane_masks #156659

Uh oh!

Conversation

MDevereau commented Sep 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

david-arm left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

llvmbot commented Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

MDevereau commented Sep 3, 2025 •

edited

Loading

llvmbot commented Sep 4, 2025 •

edited

Loading