[ConstraintElimination] Add additional facts for bitwise AND OR #132124
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
|
@llvm/pr-subscribers-llvm-transforms Author: Lee Wei (leewei05) ChangesRelated to #118114 bitwise OR Alive2: https://alive2.llvm.org/ce/z/Y-vZBx This patch was also tested against yarpgen with an extensive amount of time. @regehr Full diff: https://github.com/llvm/llvm-project/pull/132124.diff 3 Files Affected:
diff --git a/llvm/lib/Transforms/Scalar/ConstraintElimination.cpp b/llvm/lib/Transforms/Scalar/ConstraintElimination.cpp
index 267eb319a5616..01f4a6a06645b 100644
--- a/llvm/lib/Transforms/Scalar/ConstraintElimination.cpp
+++ b/llvm/lib/Transforms/Scalar/ConstraintElimination.cpp
@@ -1605,6 +1605,28 @@ void ConstraintInfo::addFact(CmpInst::Predicate Pred, Value *A, Value *B,
unsigned NumIn, unsigned NumOut,
SmallVectorImpl<StackEntry> &DFSInStack) {
addFactImpl(Pred, A, B, NumIn, NumOut, DFSInStack, false);
+
+ Value *LHS;
+ Value *RHS;
+ if (match(A, m_Or(m_Value(LHS), m_Value(RHS)))) {
+ // (LHS | RHS >= 0) => LHS >= 0 && RHS >= 0
+ // (LHS | RHS > -1) => LHS >= 0 && RHS >= 0
+ if ((match(B, m_Zero()) && Pred == CmpInst::ICMP_SGE) || (match(B, m_AllOnes()) && Pred == CmpInst::ICMP_SGT)) {
+ addFact(CmpInst::ICMP_SGE, LHS, ConstantInt::get(LHS->getType(), 0),
+ NumIn, NumOut, DFSInStack);
+ addFact(CmpInst::ICMP_SGE, RHS, ConstantInt::get(RHS->getType(), 0),
+ NumIn, NumOut, DFSInStack);
+ }
+ } else if (match(A, m_And(m_Value(LHS), m_Value(RHS)))) {
+ // (LHS & RHS < 0) => LHS < 0 && RHS < 0
+ // (LHS & RHS <= -1) => LHS < 0 && RHS < 0
+ if ((match(B, m_Zero()) && Pred == CmpInst::ICMP_SLT) || (match (B, m_AllOnes()) && Pred == CmpInst::ICMP_SLE)) {
+ addFact(CmpInst::ICMP_SLT, LHS, ConstantInt::get(LHS->getType(), 0),
+ NumIn, NumOut, DFSInStack);
+ addFact(CmpInst::ICMP_SLT, RHS, ConstantInt::get(RHS->getType(), 0),
+ NumIn, NumOut, DFSInStack);
+ }
+ }
// If the Pred is eq/ne, also add the fact to signed system.
if (CmpInst::isEquality(Pred))
addFactImpl(Pred, A, B, NumIn, NumOut, DFSInStack, true);
diff --git a/llvm/test/Transforms/ConstraintElimination/and.ll b/llvm/test/Transforms/ConstraintElimination/and.ll
index f9824df3975e9..80273aea03ea5 100644
--- a/llvm/test/Transforms/ConstraintElimination/and.ll
+++ b/llvm/test/Transforms/ConstraintElimination/and.ll
@@ -603,3 +603,129 @@ exit:
ret i1 %r.10
}
+
+define void @test_decompose_bitwise_and(i4 %x, i4 %y) {
+; CHECK-LABEL: @test_decompose_bitwise_and(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: [[TMP0:%.*]] = and i4 [[Y:%.*]], [[X:%.*]]
+; CHECK-NEXT: [[AND:%.*]] = icmp slt i4 [[TMP0]], 0
+; CHECK-NEXT: br i1 [[AND]], label [[BB1:%.*]], label [[EXIT:%.*]]
+; CHECK: bb1:
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: ret void
+; CHECK: exit:
+; CHECK-NEXT: ret void
+;
+entry:
+ %1 = and i4 %y, %x
+ %and = icmp slt i4 %1, 0
+ br i1 %and, label %bb1, label %exit
+
+bb1:
+ %f.1 = icmp slt i4 %x, 0
+ %f.2 = icmp slt i4 %y, 0
+ call void @use(i1 %f.1)
+ call void @use(i1 %f.2)
+ ret void
+
+exit:
+ ret void
+}
+
+define i1 @test_decompose_bitwise_and2(i4 %x, i4 %y) {
+; CHECK-LABEL: @test_decompose_bitwise_and2(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: [[TMP0:%.*]] = and i4 [[X:%.*]], [[Y:%.*]]
+; CHECK-NEXT: [[AND_NOT:%.*]] = icmp sgt i4 [[TMP0]], -1
+; CHECK-NEXT: br i1 [[AND_NOT]], label [[END:%.*]], label [[THEN:%.*]]
+; CHECK: then:
+; CHECK-NEXT: ret i1 true
+; CHECK: end:
+; CHECK-NEXT: ret i1 false
+;
+entry:
+ %0 = and i4 %x, %y
+ %and.not = icmp sgt i4 %0, -1
+ br i1 %and.not, label %end, label %then
+
+then:
+ %cmp = icmp slt i4 %x, 0
+ ret i1 %cmp
+
+end:
+ ret i1 false
+}
+
+define void @test_decompose_nested_bitwise_and(i4 %x, i4 %y, i4 %z, i4 %w) {
+; CHECK-LABEL: @test_decompose_nested_bitwise_and(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: [[TMP0:%.*]] = and i4 [[Y:%.*]], [[X:%.*]]
+; CHECK-NEXT: [[TMP1:%.*]] = and i4 [[TMP0]], [[Z:%.*]]
+; CHECK-NEXT: [[TMP2:%.*]] = and i4 [[TMP1]], [[W:%.*]]
+; CHECK-NEXT: [[AND:%.*]] = icmp slt i4 [[TMP2]], 0
+; CHECK-NEXT: br i1 [[AND]], label [[BB1:%.*]], label [[EXIT:%.*]]
+; CHECK: bb1:
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: ret void
+; CHECK: exit:
+; CHECK-NEXT: ret void
+;
+entry:
+ %1 = and i4 %y, %x
+ %2 = and i4 %1, %z
+ %3 = and i4 %2, %w
+ %and = icmp slt i4 %3, 0
+ br i1 %and, label %bb1, label %exit
+
+bb1:
+ %f.1 = icmp slt i4 %x, 0
+ %f.2 = icmp slt i4 %y, 0
+ %f.3 = icmp slt i4 %z, 0
+ %f.4 = icmp slt i4 %w, 0
+ call void @use(i1 %f.1)
+ call void @use(i1 %f.2)
+ call void @use(i1 %f.3)
+ call void @use(i1 %f.4)
+ ret void
+
+exit:
+ ret void
+}
+
+define void @test_decompose_nested_bitwise_and2(i4 %x, i4 %y, i4 %z) {
+; CHECK-LABEL: @test_decompose_nested_bitwise_and2(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: [[TMP0:%.*]] = and i4 [[X:%.*]], [[Y:%.*]]
+; CHECK-NEXT: [[TMP1:%.*]] = and i4 [[TMP0]], [[Z:%.*]]
+; CHECK-NEXT: [[AND_2_NOT:%.*]] = icmp sgt i4 [[TMP1]], -1
+; CHECK-NEXT: br i1 [[AND_2_NOT]], label [[F:%.*]], label [[T:%.*]]
+; CHECK: t:
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: ret void
+; CHECK: f:
+; CHECK-NEXT: ret void
+;
+entry:
+ %0 = and i4 %x, %y
+ %1 = and i4 %0, %z
+ %and.2.not = icmp sgt i4 %1, -1
+ br i1 %and.2.not, label %f, label %t
+
+t:
+ %cmp = icmp slt i4 %x, 0
+ call void @use(i1 %cmp)
+ %cmp.2 = icmp slt i4 %y, 0
+ call void @use(i1 %cmp.2)
+ %cmp.3 = icmp slt i4 %z, 0
+ call void @use(i1 %cmp.3)
+ ret void
+
+f:
+ ret void
+}
diff --git a/llvm/test/Transforms/ConstraintElimination/or.ll b/llvm/test/Transforms/ConstraintElimination/or.ll
index b401d6f181369..002f2fffebb56 100644
--- a/llvm/test/Transforms/ConstraintElimination/or.ll
+++ b/llvm/test/Transforms/ConstraintElimination/or.ll
@@ -808,3 +808,129 @@ end: ; preds = %entry
ret void
}
+
+define void @test_decompose_bitwise_or(i4 %x, i4 %y) {
+; CHECK-LABEL: @test_decompose_bitwise_or(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: [[TMP0:%.*]] = or i4 [[Y:%.*]], [[X:%.*]]
+; CHECK-NEXT: [[OR:%.*]] = icmp slt i4 [[TMP0]], 0
+; CHECK-NEXT: br i1 [[OR]], label [[BB1:%.*]], label [[EXIT:%.*]]
+; CHECK: bb1:
+; CHECK-NEXT: ret void
+; CHECK: exit:
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: ret void
+;
+entry:
+ %0 = or i4 %y, %x
+ %or = icmp slt i4 %0, 0
+ br i1 %or, label %bb1, label %exit
+
+bb1:
+ ret void
+
+exit:
+ %f.3 = icmp sge i4 %x, 0
+ %f.4 = icmp sge i4 %y, 0
+ call void @use(i1 %f.3)
+ call void @use(i1 %f.4)
+ ret void
+}
+
+define i1 @test_decompose_bitwise_or2(i4 %x, i4 %y) {
+; CHECK-LABEL: @test_decompose_bitwise_or2(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: [[TMP0:%.*]] = or i4 [[X:%.*]], [[Y:%.*]]
+; CHECK-NEXT: [[OR_NOT:%.*]] = icmp sgt i4 [[TMP0]], -1
+; CHECK-NEXT: br i1 [[OR_NOT]], label [[END:%.*]], label [[THEN:%.*]]
+; CHECK: then:
+; CHECK-NEXT: ret i1 false
+; CHECK: end:
+; CHECK-NEXT: ret i1 true
+;
+entry:
+ %0 = or i4 %x, %y
+ %or.not = icmp sgt i4 %0, -1
+ br i1 %or.not, label %end, label %then
+
+then:
+ ret i1 false
+
+end:
+ %cmp = icmp sgt i4 %x, -1
+ ret i1 %cmp
+}
+
+define void @test_decompose_nested_bitwise_or(i4 %x, i4 %y, i4 %z, i4 %w) {
+; CHECK-LABEL: @test_decompose_nested_bitwise_or(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: [[TMP0:%.*]] = or i4 [[Y:%.*]], [[X:%.*]]
+; CHECK-NEXT: [[TMP1:%.*]] = or i4 [[TMP0]], [[Z:%.*]]
+; CHECK-NEXT: [[TMP2:%.*]] = or i4 [[TMP1]], [[W:%.*]]
+; CHECK-NEXT: [[OR:%.*]] = icmp slt i4 [[TMP2]], 0
+; CHECK-NEXT: br i1 [[OR]], label [[BB1:%.*]], label [[EXIT:%.*]]
+; CHECK: bb1:
+; CHECK-NEXT: ret void
+; CHECK: exit:
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: ret void
+;
+entry:
+ %1 = or i4 %y, %x
+ %2 = or i4 %1, %z
+ %3 = or i4 %2, %w
+ %or = icmp slt i4 %3, 0
+ br i1 %or, label %bb1, label %exit
+
+bb1:
+ ret void
+
+exit:
+ %f.4 = icmp sge i4 %x, 0
+ %f.5 = icmp sge i4 %y, 0
+ %f.6 = icmp sge i4 %z, 0
+ %f.7 = icmp sge i4 %w, 0
+ call void @use(i1 %f.4)
+ call void @use(i1 %f.5)
+ call void @use(i1 %f.6)
+ call void @use(i1 %f.7)
+ ret void
+}
+
+define void @test_decompose_nested_bitwise_or2(i4 %x, i4 %y, i4 %z) {
+; CHECK-LABEL: @test_decompose_nested_bitwise_or2(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: [[TMP0:%.*]] = or i4 [[X:%.*]], [[Y:%.*]]
+; CHECK-NEXT: [[TMP1:%.*]] = or i4 [[TMP0]], [[Z:%.*]]
+; CHECK-NEXT: [[OR_2_NOT:%.*]] = icmp sgt i4 [[TMP1]], -1
+; CHECK-NEXT: br i1 [[OR_2_NOT]], label [[F:%.*]], label [[T:%.*]]
+; CHECK: t:
+; CHECK-NEXT: ret void
+; CHECK: f:
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: call void @use(i1 true)
+; CHECK-NEXT: ret void
+;
+entry:
+ %0 = or i4 %x, %y
+ %1 = or i4 %0, %z
+ %or.2.not = icmp sgt i4 %1, -1
+ br i1 %or.2.not, label %f, label %t
+
+t: ; preds = %entry
+ ret void
+
+f: ; preds = %entry
+ %cmp.1 = icmp sgt i4 %x, -1
+ call void @use(i1 %cmp.1)
+ %cmp.2 = icmp sgt i4 %y, -1
+ call void @use(i1 %cmp.2)
+ %cmp.3 = icmp sgt i4 %z, -1
+ call void @use(i1 %cmp.3)
+ ret void
+}
|
Contributor
Author
|
@dtcxzyw Can you also run this patch with llvm-opt-benchmark? I'm curious on how much this optimization would benefit in real life program. Thanks! |
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
leewei05
force-pushed
the
add-ltz-constraint
branch
from
0b2798f to
9f4e642
Compare
This was referenced Mar 20, 2025
leewei05
force-pushed
the
add-ltz-constraint
branch
from
9f4e642 to
e4a8b21
Compare
dtcxzyw
reviewed
Mar 20, 2025
| Value *LHS; | ||
| Value *RHS; | ||
| if (match(A, m_Or(m_Value(LHS), m_Value(RHS)))) { | ||
| // (LHS | RHS >= 0) => LHS >= 0 && RHS >= 0 |
Member
There was a problem hiding this comment.
X <=/>= 0 are not canonical forms. Do we need to handle them?
Contributor
Author
There was a problem hiding this comment.
@dtcxzyw I wanted to handle as many patterns as possible. Also, constraint elimination adds certain facts with ICMP_SGE and I want to make sure that those facts can be handled as well. Even if X <=/>= 0 are not canonical forms, can these patterns exist after running InstCombine?
Btw, is there a documentation for these canonical forms? Or these canonical forms can only be found in InstCombine code.
Contributor
There was a problem hiding this comment.
it looks like instcombine reliably canonicalizes these, so I don't think you need to match the non-canonical forms
Contributor
Author
There was a problem hiding this comment.
Yes, we need to handle <=/>= despite them not being canonical, as ConstraintElimination can still use them for optimizations. The pass doesn’t limit facts to canonical forms—e.g., in the example below, the end block adds a fact from an inverted predicate (%1 <= -1). This applies to both AND and OR cases.
define void @test_decompose_bitwise_and(i4 %x, i4 %y) {
entry:
%1 = and i4 %y, %x
; canonicalize form after InstCombine:
; %and = icmp sgt i4 %1, -1
%and = icmp sge i4 %1, 0
br i1 %and, label %then, label %end
then:
; fact: %and > -1
ret void
end:
; fact: %and <= -1
%f.1 = icmp slt i4 %x, 0
%f.2 = icmp slt i4 %y, 0
call void @use(i1 %f.1)
call void @use(i1 %f.2)
ret void
}
AND case: https://llvm.godbolt.org/z/7h86PqacE
AND Alive2: https://alive2.llvm.org/ce/z/9FY5Tt
OR case: https://llvm.godbolt.org/z/fdzrbTn4M
OR Alive2: https://alive2.llvm.org/ce/z/Cog6xS
I hope it makes sense. I'll update the test cases accordingly as well.
| ; CHECK-NEXT: ret void | ||
| ; | ||
| entry: | ||
| %1 = and i4 %y, %x |
|
|
||
| ret i1 %r.10 | ||
| } | ||
|
|
Member
There was a problem hiding this comment.
Can you add some negative tests?
fhahn
reviewed
Mar 20, 2025
Contributor
fhahn
left a comment
fhahn
left a comment
There was a problem hiding this comment.
Thanks for working on this!
| addFact(CmpInst::ICMP_SGE, LHS, ConstantInt::get(LHS->getType(), 0), | ||
| NumIn, NumOut, DFSInStack); | ||
| addFact(CmpInst::ICMP_SGE, RHS, ConstantInt::get(RHS->getType(), 0), | ||
| NumIn, NumOut, DFSInStack); |
Contributor
There was a problem hiding this comment.
is there benefit of continuing to add the constraint for A, B below or can we return early? Would be good to ensure sufficient test coverage
Contributor
Author
There was a problem hiding this comment.
Yes, keeping the original A, B constraint is beneficial. For OR, it keeps cases where LHS | RHS >= 0 holds, while the derived LHS >= 0 && RHS >= 0 strengthens it. For AND, it matches exactly and ties to the code. The recursive addFact also supports nested OR/AND, adding more facts. I’ll ensure test coverage for single-level, nested, and edge cases. Does this answer your question?
Use named values and rename branches
Contributor
Author
|
Ping |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related to #118114
bitwise OR Alive2: https://alive2.llvm.org/ce/z/Y-vZBx
bitwise AND Alive2: https://alive2.llvm.org/ce/z/xMy4zZ
This patch was also tested against yarpgen with an extensive amount of time.
@regehr