[SCEV] Preserve divisibility info when creating UMax/SMax expressions. #160012

fhahn · 2025-09-21T20:41:08Z

Currently we generate (S|U)Max(1, Op) for Op >= 1. This may discard divisibility info of Op. This patch rewrites such SMax/UMax expressions to use the lowest common multiplier for all non-constant operands.

llvmbot · 2025-09-21T20:41:37Z

@llvm/pr-subscribers-llvm-transforms

@llvm/pr-subscribers-llvm-analysis

Author: Florian Hahn (fhahn)

Changes

Currently we generate (S|U)Max(1, Op) for Op >= 1. This may discard divisibility info of Op. This patch rewrites such SMax/UMax expressions to use the lowest common multiplier for all non-constant operands.

Full diff: https://github.com/llvm/llvm-project/pull/160012.diff

2 Files Affected:

(modified) llvm/lib/Analysis/ScalarEvolution.cpp (+22-2)
(modified) llvm/test/Analysis/ScalarEvolution/trip-count-minmax.ll (+2-2)

diff --git a/llvm/lib/Analysis/ScalarEvolution.cpp b/llvm/lib/Analysis/ScalarEvolution.cpp
index b08399b381f34..ee1f92a4197e8 100644
--- a/llvm/lib/Analysis/ScalarEvolution.cpp
+++ b/llvm/lib/Analysis/ScalarEvolution.cpp
@@ -15850,12 +15850,17 @@ void ScalarEvolution::LoopGuards::collectFromBlock(
         To = SE.getUMaxExpr(FromRewritten, RHS);
         if (auto *UMin = dyn_cast<SCEVUMinExpr>(FromRewritten))
           EnqueueOperands(UMin);
+        if (RHS->isOne())
+          ExprsToRewrite.push_back(From);
         break;
       case CmpInst::ICMP_SGT:
       case CmpInst::ICMP_SGE:
         To = SE.getSMaxExpr(FromRewritten, RHS);
-        if (auto *SMin = dyn_cast<SCEVSMinExpr>(FromRewritten))
+        if (auto *SMin = dyn_cast<SCEVSMinExpr>(FromRewritten)) {
           EnqueueOperands(SMin);
+        }
+        if (RHS->isOne())
+          ExprsToRewrite.push_back(From);
         break;
       case CmpInst::ICMP_EQ:
         if (isa<SCEVConstant>(RHS))
@@ -15986,7 +15991,22 @@ void ScalarEvolution::LoopGuards::collectFromBlock(
     for (const SCEV *Expr : ExprsToRewrite) {
       const SCEV *RewriteTo = Guards.RewriteMap[Expr];
       Guards.RewriteMap.erase(Expr);
-      Guards.RewriteMap.insert({Expr, Guards.rewrite(RewriteTo)});
+      const SCEV *Rewritten = Guards.rewrite(RewriteTo);
+
+      // Try to strengthen divisibility of SMax/UMax expressions coming from >=
+      // 1 conditions.
+      if (auto *SMax = dyn_cast<SCEVSMaxExpr>(Rewritten)) {
+        unsigned MinTrailingZeros = SE.getMinTrailingZeros(SMax->getOperand(1));
+        for (const SCEV *Op : drop_begin(SMax->operands(), 2))
+          MinTrailingZeros =
+              std::min(MinTrailingZeros, SE.getMinTrailingZeros(Op));
+        if (MinTrailingZeros != 0)
+          Rewritten = SE.getSMaxExpr(
+              SE.getConstant(APInt(SMax->getType()->getScalarSizeInBits(), 1)
+                                 .shl(MinTrailingZeros)),
+              SMax);
+      }
+      Guards.RewriteMap.insert({Expr, Rewritten});
     }
   }
 }
diff --git a/llvm/test/Analysis/ScalarEvolution/trip-count-minmax.ll b/llvm/test/Analysis/ScalarEvolution/trip-count-minmax.ll
index 8d091a00ed4b9..d38010403dad7 100644
--- a/llvm/test/Analysis/ScalarEvolution/trip-count-minmax.ll
+++ b/llvm/test/Analysis/ScalarEvolution/trip-count-minmax.ll
@@ -61,7 +61,7 @@ define void @umin(i32 noundef %a, i32 noundef %b) {
 ; CHECK-NEXT:  Loop %for.body: backedge-taken count is (-1 + ((2 * %a) umin (4 * %b)))
 ; CHECK-NEXT:  Loop %for.body: constant max backedge-taken count is i32 2147483646
 ; CHECK-NEXT:  Loop %for.body: symbolic max backedge-taken count is (-1 + ((2 * %a) umin (4 * %b)))
-; CHECK-NEXT:  Loop %for.body: Trip multiple is 1
+; CHECK-NEXT:  Loop %for.body: Trip multiple is 2
 ;
 ; void umin(unsigned a, unsigned b) {
 ;   a *= 2;
@@ -157,7 +157,7 @@ define void @smin(i32 noundef %a, i32 noundef %b) {
 ; CHECK-NEXT:  Loop %for.body: backedge-taken count is (-1 + ((2 * %a)<nsw> smin (4 * %b)<nsw>))
 ; CHECK-NEXT:  Loop %for.body: constant max backedge-taken count is i32 2147483646
 ; CHECK-NEXT:  Loop %for.body: symbolic max backedge-taken count is (-1 + ((2 * %a)<nsw> smin (4 * %b)<nsw>))
-; CHECK-NEXT:  Loop %for.body: Trip multiple is 1
+; CHECK-NEXT:  Loop %for.body: Trip multiple is 2
 ;
 ; void smin(signed a, signed b) {
 ;   a *= 2;

When re-writing SCEVAddExprs to apply information from guards, check if we have information for the expression itself. If so, apply it. When we have an expression of the form (Const + A), check if we have have guard info for (Const + 1 + A) and use it. This is needed to avoid regressions in a few cases, where we have BTCs with a subtracted constant. Rewriting expressions could cause regressions, e.g. when comparing 2 SCEV expressions where we are only able to rewrite one side, but I could not find any cases where this happens more with this patch in practice. Depends on llvm#160012 (included in PR) Proofs for some of the test changes: https://alive2.llvm.org/ce/z/RPX6t_

fhahn · 2025-09-22T12:05:31Z

No differences on llvm-opt-benchmark (dtcxzyw/llvm-opt-benchmark#2846), but there are a few changes on large C/C++ corpus with unrolling and vectorization enabled.

nikic

The idea makes sense to me, but TBH I'm pretty lost in the code structure here. Why are we fixing this up after the fact rather than creating the umax/umin with the larger value from the start?

Also would it make sense to do something more generic during SCEV construction here? Or do we expect this to only be useful for guards?

llvm/lib/Analysis/ScalarEvolution.cpp

Add test for SCEVUMaxExpr handling in #160012.

fhahn

The idea makes sense to me, but TBH I'm pretty lost in the code structure here. Why are we fixing this up after the fact rather than creating the umax/umin with the larger value from the start?

The benefit of delaying is that we can delay the re-write until we collected information from all loop guards, making the code independent of the order of guards.

We could have 3 guards, establishing

umax(%a, %b) > 0
%a multiple of 2
%b multiple of 2

When we construct umax(1, %a, %b) for the first condition, we may not yet have the information available that %a and %b are multiple of 2.

But once we collected all information, we can rewrite umax(1, %a, %b) to something like umax(1, 2 * %a / 2, 2* %b / 2) and get the common multiple using the info from the guards.

Not sure if there's a nicer way to keep things independent of the guard order.

Also would it make sense to do something more generic during SCEV construction here? Or do we expect this to only be useful for guards?

Hmm, I think the current code relies on the fact that the UMax/SMax with the constant is coming from a compare w/o the constant part on the left side.

Are there any particluar cases you are thinking of on construction?

llvm/lib/Analysis/ScalarEvolution.cpp

…nfo from guard Add test for SCEVUMaxExpr handling in llvm/llvm-project#160012.

nikic · 2025-09-22T20:53:19Z

Hmm, I think the current code relies on the fact that the UMax/SMax with the constant is coming from a compare w/o the constant part on the left side.

Are there any particluar cases you are thinking of on construction?

What I meant is that we can generally fold umax(C1, C2*x) with C2>C1 and x!=0 to umax(C2, C2*x). But I guess x!=0 is the critical part here -- we generally do not know that outside the guard context (or rather, can't ignore it outside the guard context).

fhahn · 2025-09-23T09:26:06Z

Hmm, I think the current code relies on the fact that the UMax/SMax with the constant is coming from a compare w/o the constant part on the left side.
Are there any particluar cases you are thinking of on construction?

What I meant is that we can generally fold umax(C1, C2*x) with C2>C1 and x!=0 to umax(C2, C2*x). But I guess x!=0 is the critical part here -- we generally do not know that outside the guard context (or rather, can't ignore it outside the guard context).

Yep, I can try to see if this would also trigger in practice at construction, but we would still need the guard-specific logic w/o the != 0 check

preames · 2025-09-23T15:25:03Z

Yep, I can try to see if this would also trigger in practice at construction, but we would still need the guard-specific logic w/o the != 0 check

Your wording here triggered a thought. When phrased like this, this sounds a lot like SCEV construction under an assumption (or predicate) that x != 0. We have a bunch of logic of this variety in PredicatedScalarEvolution, and our assumption handling already, is there a possibility for code sharing here?

(This may not be worth the work to actually do immediately. Not a blocking comment by any means.)

When collecting information from loop guards, use UMax(1, %b - %a) for ICMP NE %a, %b, if neither are constant. This improves results in some cases, and will be even more useful together with * llvm#160012 * llvm#159942 https://alive2.llvm.org/ce/z/YyBvoT

fhahn · 2025-09-24T17:49:04Z

Yep, I can try to see if this would also trigger in practice at construction, but we would still need the guard-specific logic w/o the != 0 check

Your wording here triggered a thought. When phrased like this, this sounds a lot like SCEV construction under an assumption (or predicate) that x != 0. We have a bunch of logic of this variety in PredicatedScalarEvolution, and our assumption handling already, is there a possibility for code sharing here?

(This may not be worth the work to actually do immediately. Not a blocking comment by any means.)

Thanks, I need to think about this a bit more. With both PredicatedScalarEvolution and loop guards we rewrite SCEV expressions given extra information (runtime predicates and information from guards respectively), but currently there does't seem much overlap in the types of expressions we rewrite, with PSE mostly focused on extends and forced AddRecs.

fhahn · 2025-09-29T18:26:16Z

ping :)

When collecting information from loop guards, use UMax(1, %b - %a) for ICMP NE %a, %b, if neither are constant. This improves results in some cases, and will be even more useful together with * llvm#160012 * llvm#159942 https://alive2.llvm.org/ce/z/YyBvoT

nikic · 2025-10-08T21:45:13Z

llvm/lib/Analysis/ScalarEvolution.cpp

+      case CmpInst::ICMP_UGE: {
+        const SCEV *OpAlignedUp =
+            DividesBy ? GetNextSCEVDividesByDivisor(RHS, DividesBy) : RHS;
+        To = SE.getUMaxExpr(FromRewritten, OpAlignedUp);


Actually, are the changes here necessary at all? It looks like we are already doing these next/prev divisor adjustments for RHS in the switch above this. Maybe the generalization of the divisor logic is sufficient?

Yep it looks like we get all the benfits from the patch with just the switch to getConstantMultiple: #162617

Simplify and generalize the code to get a common constant multiple for expressions when collecting guards, replacing the manual implementation. Split off from llvm#160012.

…s. (#162617) Simplify and generalize the code to get a common constant multiple for expressions when collecting guards, replacing the manual implementation. Split off from #160012. PR: #162617

… from guards. (#162617) Simplify and generalize the code to get a common constant multiple for expressions when collecting guards, replacing the manual implementation. Split off from llvm/llvm-project#160012. PR: llvm/llvm-project#162617

fhahn · 2025-10-09T10:22:59Z

Simplified version in #162617 handles all cases, closing for now

When re-writing SCEVAddExprs to apply information from guards, check if we have information for the expression itself. If so, apply it. When we have an expression of the form (Const + A), check if we have have guard info for (Const + 1 + A) and use it. This is needed to avoid regressions in a few cases, where we have BTCs with a subtracted constant. Rewriting expressions could cause regressions, e.g. when comparing 2 SCEV expressions where we are only able to rewrite one side, but I could not find any cases where this happens more with this patch in practice. Depends on llvm#160012 (included in PR) Proofs for some of the test changes: https://alive2.llvm.org/ce/z/RPX6t_

…s. (#162617) Simplify and generalize the code to get a common constant multiple for expressions when collecting guards, replacing the manual implementation. Split off from #160012. PR: #162617

When re-writing SCEVAddExprs to apply information from guards, check if we have information for the expression itself. If so, apply it. When we have an expression of the form (Const + A), check if we have have guard info for (Const + 1 + A) and use it. This is needed to avoid regressions in a few cases, where we have BTCs with a subtracted constant. Rewriting expressions could cause regressions, e.g. when comparing 2 SCEV expressions where we are only able to rewrite one side, but I could not find any cases where this happens more with this patch in practice. Depends on llvm#160012 (included in PR) Proofs for some of the test changes: https://alive2.llvm.org/ce/z/RPX6t_

…s. (llvm#162617) Simplify and generalize the code to get a common constant multiple for expressions when collecting guards, replacing the manual implementation. Split off from llvm#160012. PR: llvm#162617

When re-writing SCEVAddExprs to apply information from guards, check if we have information for the expression itself. If so, apply it. When we have an expression of the form (Const + A), check if we have have guard info for (Const + 1 + A) and use it. This is needed to avoid regressions in a few cases, where we have BTCs with a subtracted constant. Rewriting expressions could cause regressions, e.g. when comparing 2 SCEV expressions where we are only able to rewrite one side, but I could not find any cases where this happens more with this patch in practice. Depends on llvm#160012 (included in PR) Proofs for some of the test changes: https://alive2.llvm.org/ce/z/RPX6t_

When collecting information from loop guards, use UMax(1, %b - %a) for ICMP NE %a, %b, if neither are constant. This improves results in some cases, and will be even more useful together with * llvm#160012 * llvm#159942 https://alive2.llvm.org/ce/z/YyBvoT

…s. (llvm#162617) Simplify and generalize the code to get a common constant multiple for expressions when collecting guards, replacing the manual implementation. Split off from llvm#160012. PR: llvm#162617

When collecting information from loop guards, use UMax(1, %b - %a) for ICMP NE %a, %b, if neither are constant. This improves results in some cases, and will be even more useful together with * llvm#160012 * llvm#159942 https://alive2.llvm.org/ce/z/YyBvoT

When re-writing SCEVAddExprs to apply information from guards, check if we have information for the expression itself. If so, apply it. When we have an expression of the form (Const + A), check if we have have guard info for (Const + 1 + A) and use it. This is needed to avoid regressions in a few cases, where we have BTCs with a subtracted constant. Rewriting expressions could cause regressions, e.g. when comparing 2 SCEV expressions where we are only able to rewrite one side, but I could not find any cases where this happens more with this patch in practice. Depends on llvm#160012 (included in PR) Proofs for some of the test changes: https://alive2.llvm.org/ce/z/RPX6t_

When collecting information from loop guards, use UMax(1, %b - %a) for ICMP NE %a, %b, if neither are constant. This improves results in some cases, and will be even more useful together with * llvm#160012 * llvm#159942 https://alive2.llvm.org/ce/z/YyBvoT

When collecting information from loop guards, use UMax(1, %b - %a) for ICMP NE %a, %b, if neither are constant. This improves results in some cases, and will be even more useful together with * #160012 * #159942 https://alive2.llvm.org/ce/z/YyBvoT PR: #160500

…500) When collecting information from loop guards, use UMax(1, %b - %a) for ICMP NE %a, %b, if neither are constant. This improves results in some cases, and will be even more useful together with * llvm/llvm-project#160012 * llvm/llvm-project#159942 https://alive2.llvm.org/ce/z/YyBvoT PR: llvm/llvm-project#160500

…s. (llvm#162617) Simplify and generalize the code to get a common constant multiple for expressions when collecting guards, replacing the manual implementation. Split off from llvm#160012. PR: llvm#162617

When collecting information from loop guards, use UMax(1, %b - %a) for ICMP NE %a, %b, if neither are constant. This improves results in some cases, and will be even more useful together with * llvm#160012 * llvm#159942 https://alive2.llvm.org/ce/z/YyBvoT PR: llvm#160500

When re-writing SCEVAddExprs to apply information from guards, check if we have information for the expression itself. If so, apply it. When we have an expression of the form (Const + A), check if we have have guard info for (Const + 1 + A) and use it. This is needed to avoid regressions in a few cases, where we have BTCs with a subtracted constant. Rewriting expressions could cause regressions, e.g. when comparing 2 SCEV expressions where we are only able to rewrite one side, but I could not find any cases where this happens more with this patch in practice. Depends on llvm#160012 (included in PR) Proofs for some of the test changes: https://alive2.llvm.org/ce/z/RPX6t_

fhahn requested review from efriedma-quic, nikic and preames September 21, 2025 20:41

llvmbot added the llvm:analysis Includes value tracking, cost tables and constant folding label Sep 21, 2025

This was referenced Sep 21, 2025

[SCEV] Rewrite more SCEVAddExpr when applying guards. #159942

Open

Task submission dtcxzyw/llvm-opt-benchmark#1312

Open

zyw-bot mentioned this pull request Sep 22, 2025

pre-commit: PR160012 dtcxzyw/llvm-opt-benchmark#2846

Closed

nikic reviewed Sep 22, 2025

View reviewed changes

llvm/lib/Analysis/ScalarEvolution.cpp Outdated Show resolved Hide resolved

fhahn added a commit that referenced this pull request Sep 22, 2025

[LV] Add test showing missed optimization due to missing info from guard

129c683

Add test for SCEVUMaxExpr handling in #160012.

fhahn force-pushed the scev-guards-smax-umax-divisibility branch from 7d05774 to 941d620 Compare September 22, 2025 20:03

llvmbot added the llvm:transforms label Sep 22, 2025

fhahn commented Sep 22, 2025

View reviewed changes

llvm/lib/Analysis/ScalarEvolution.cpp Outdated Show resolved Hide resolved

llvm-sync bot pushed a commit to arm/arm-toolchain that referenced this pull request Sep 22, 2025

Automerge: [LV] Add test showing missed optimization due to missing i…

ac3e50e

…nfo from guard Add test for SCEVUMaxExpr handling in llvm/llvm-project#160012.

fhahn mentioned this pull request Sep 24, 2025

[SCEV] Collect guard info for ICMP NE w/o constants. #160500

Merged

fhahn force-pushed the scev-guards-smax-umax-divisibility branch from 941d620 to 59a8e0f Compare September 29, 2025 18:26

fhahn force-pushed the scev-guards-smax-umax-divisibility branch from 59a8e0f to 8e50ec5 Compare October 6, 2025 14:43

fhahn added 4 commits October 8, 2025 21:30

!fixup also rewrite SCEVUMaxExpr, use getConstantMultiple.

be1eefe

!fixup Simplify and handle via DividesBy.

0fbf7d0

!fixup remove unneeded code

cc57a12

!fixup remove code, just use getConstantMultiple.

258fb8f

fhahn force-pushed the scev-guards-smax-umax-divisibility branch from 502cef4 to 258fb8f Compare October 8, 2025 20:37

nikic reviewed Oct 8, 2025

View reviewed changes

fhahn mentioned this pull request Oct 9, 2025

[SCEV] Use getConstantMultiple in to get divisibility info from guards. #162617

Merged

fhahn closed this Oct 9, 2025

fhahn deleted the scev-guards-smax-umax-divisibility branch October 9, 2025 10:23

[SCEV] Preserve divisibility info when creating UMax/SMax expressions. #160012

[SCEV] Preserve divisibility info when creating UMax/SMax expressions. #160012

Uh oh!

Conversation

fhahn commented Sep 21, 2025

Uh oh!

llvmbot commented Sep 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fhahn commented Sep 22, 2025

Uh oh!

nikic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

fhahn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nikic commented Sep 22, 2025

Uh oh!

fhahn commented Sep 23, 2025

Uh oh!

preames commented Sep 23, 2025

Uh oh!

fhahn commented Sep 24, 2025

Uh oh!

fhahn commented Sep 29, 2025

Uh oh!

nikic Oct 8, 2025

Choose a reason for hiding this comment

Uh oh!

fhahn Oct 9, 2025

Choose a reason for hiding this comment

Uh oh!

fhahn commented Oct 9, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

llvmbot commented Sep 21, 2025 •

edited

Loading