[LVI] Infer non-zero from equality icmp #112838

dtcxzyw · 2024-10-18T05:03:34Z

This following pattern is common in loop headers:

  %101 = sub nuw i64 %78, %98
  %103 = icmp eq i64 %78, %98
  br i1 %103, label %.thread.i.i, label %.preheader.preheader.i.i

.preheader.preheader.i.i:
  %invariant.umin.i.i = call i64 @llvm.umin.i64(i64 %101, i64 9)
  %umax.i = call i64 @llvm.umax.i64(i64 %invariant.umin.i.i, i64 1)
  br label %.preheader.i.i

.preheader.i.i:
  ...
  %116 = add nuw nsw i64 %.011.i.i, 1
  %exitcond.not.i = icmp eq i64 %116, %umax.i
  br i1 %exitcond.not.i, label %.critedge.i.i, label %.preheader.i.i

As %78 is not equal to %98 in BB .preheader.preheader.i.i, we can prove %101 is non-zero. Then we can simplify the loop exit condition.

Addresses regression introduced by #112742.

llvmbot · 2024-10-18T05:04:10Z

@llvm/pr-subscribers-llvm-transforms

Author: Yingwei Zheng (dtcxzyw)

Changes

This following pattern is common in loop headers:

  %101 = sub nuw i64 %78, %98
  %103 = icmp eq i64 %78, %98
  br i1 %103, label %.thread.i.i, label %.preheader.preheader.i.i

.preheader.preheader.i.i:
  %invariant.umin.i.i = call i64 @<!-- -->llvm.umin.i64(i64 %101, i64 9)
  %umax.i = call i64 @<!-- -->llvm.umax.i64(i64 %invariant.umin.i.i, i64 1)
  br label %.preheader.i.i

.preheader.i.i:
  ...
  %116 = add nuw nsw i64 %.011.i.i, 1
  %exitcond.not.i = icmp eq i64 %116, %umax.i
  br i1 %exitcond.not.i, label %.critedge.i.i, label %.preheader.i.i

As %78 is not equal to %98 in BB .preheader.preheader.i.i, we can prove %101 is non-zero. Then we can simplify the loop exit condition.

Addresses regression introduced by #112742.

Full diff: https://github.com/llvm/llvm-project/pull/112838.diff

2 Files Affected:

(modified) llvm/lib/Analysis/LazyValueInfo.cpp (+14)
(added) llvm/test/Transforms/CorrelatedValuePropagation/umax.ll (+58)

diff --git a/llvm/lib/Analysis/LazyValueInfo.cpp b/llvm/lib/Analysis/LazyValueInfo.cpp
index 10ad4708596cb3..f29777a584772d 100644
--- a/llvm/lib/Analysis/LazyValueInfo.cpp
+++ b/llvm/lib/Analysis/LazyValueInfo.cpp
@@ -1127,6 +1127,20 @@ std::optional<ValueLatticeElement> LazyValueInfoImpl::getValueFromICmpCondition(
   if (!Ty->isIntegerTy())
     return ValueLatticeElement::getOverdefined();
 
+  // a - b or ptrtoint(a) - ptrtoint(b) ==/!= 0 if a ==/!= b
+  Value *X, *Y;
+  if (ICI->isEquality() && match(Val, m_Sub(m_Value(X), m_Value(Y)))) {
+    // Peek through ptrtoints
+    match(X, m_PtrToIntSameSize(DL, m_Value(X)));
+    match(Y, m_PtrToIntSameSize(DL, m_Value(Y)));
+    if ((X == LHS && Y == RHS) || (X == RHS && Y == LHS)) {
+      Constant *NullVal = Constant::getNullValue(Val->getType());
+      if (EdgePred == ICmpInst::ICMP_EQ)
+        return ValueLatticeElement::get(NullVal);
+      return ValueLatticeElement::getNot(NullVal);
+    }
+  }
+
   unsigned BitWidth = Ty->getScalarSizeInBits();
   APInt Offset(BitWidth, 0);
   if (matchICmpOperand(Offset, LHS, Val, EdgePred))
diff --git a/llvm/test/Transforms/CorrelatedValuePropagation/umax.ll b/llvm/test/Transforms/CorrelatedValuePropagation/umax.ll
new file mode 100644
index 00000000000000..5cd615e948adbe
--- /dev/null
+++ b/llvm/test/Transforms/CorrelatedValuePropagation/umax.ll
@@ -0,0 +1,58 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 5
+; RUN: opt < %s -passes=correlated-propagation -S | FileCheck %s
+
+target datalayout = "p:32:32"
+
+define i32 @infer_range_from_dom_equality(i32 %x, i32 %y) {
+; CHECK-LABEL: define range(i32 1, 0) i32 @infer_range_from_dom_equality(
+; CHECK-SAME: i32 [[X:%.*]], i32 [[Y:%.*]]) {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    [[COND:%.*]] = icmp eq i32 [[X]], [[Y]]
+; CHECK-NEXT:    [[SUB:%.*]] = sub i32 [[X]], [[Y]]
+; CHECK-NEXT:    br i1 [[COND]], label %[[IF_THEN:.*]], label %[[IF_ELSE:.*]]
+; CHECK:       [[IF_THEN]]:
+; CHECK-NEXT:    ret i32 1
+; CHECK:       [[IF_ELSE]]:
+; CHECK-NEXT:    ret i32 [[SUB]]
+;
+entry:
+  %cond = icmp eq i32 %x, %y
+  %sub = sub i32 %x, %y
+  br i1 %cond, label %if.then, label %if.else
+
+if.then:
+  %max1 = call i32 @llvm.umax.i32(i32 %sub, i32 1)
+  ret i32 %max1
+
+if.else:
+  %max2 = call i32 @llvm.umax.i32(i32 %sub, i32 1)
+  ret i32 %max2
+}
+
+define i32 @infer_range_from_dom_equality_ptrdiff(ptr %x, ptr %y) {
+; CHECK-LABEL: define range(i32 1, 0) i32 @infer_range_from_dom_equality_ptrdiff(
+; CHECK-SAME: ptr [[X:%.*]], ptr [[Y:%.*]]) {
+; CHECK-NEXT:    [[COND:%.*]] = icmp eq ptr [[X]], [[Y]]
+; CHECK-NEXT:    [[XI:%.*]] = ptrtoint ptr [[X]] to i32
+; CHECK-NEXT:    [[YI:%.*]] = ptrtoint ptr [[Y]] to i32
+; CHECK-NEXT:    [[SUB:%.*]] = sub i32 [[XI]], [[YI]]
+; CHECK-NEXT:    br i1 [[COND]], label %[[IF_THEN:.*]], label %[[IF_ELSE:.*]]
+; CHECK:       [[IF_THEN]]:
+; CHECK-NEXT:    ret i32 1
+; CHECK:       [[IF_ELSE]]:
+; CHECK-NEXT:    ret i32 [[SUB]]
+;
+  %cond = icmp eq ptr %x, %y
+  %xi = ptrtoint ptr %x to i32
+  %yi = ptrtoint ptr %y to i32
+  %sub = sub i32 %xi, %yi
+  br i1 %cond, label %if.then, label %if.else
+
+if.then:
+  %max1 = call i32 @llvm.umax.i32(i32 %sub, i32 1)
+  ret i32 %max1
+
+if.else:
+  %max2 = call i32 @llvm.umax.i32(i32 %sub, i32 1)
+  ret i32 %max2
+}

llvmbot · 2024-10-18T05:04:10Z

@llvm/pr-subscribers-llvm-analysis

Author: Yingwei Zheng (dtcxzyw)

Changes

This following pattern is common in loop headers:

  %101 = sub nuw i64 %78, %98
  %103 = icmp eq i64 %78, %98
  br i1 %103, label %.thread.i.i, label %.preheader.preheader.i.i

.preheader.preheader.i.i:
  %invariant.umin.i.i = call i64 @<!-- -->llvm.umin.i64(i64 %101, i64 9)
  %umax.i = call i64 @<!-- -->llvm.umax.i64(i64 %invariant.umin.i.i, i64 1)
  br label %.preheader.i.i

.preheader.i.i:
  ...
  %116 = add nuw nsw i64 %.011.i.i, 1
  %exitcond.not.i = icmp eq i64 %116, %umax.i
  br i1 %exitcond.not.i, label %.critedge.i.i, label %.preheader.i.i

As %78 is not equal to %98 in BB .preheader.preheader.i.i, we can prove %101 is non-zero. Then we can simplify the loop exit condition.

Addresses regression introduced by #112742.

Full diff: https://github.com/llvm/llvm-project/pull/112838.diff

2 Files Affected:

(modified) llvm/lib/Analysis/LazyValueInfo.cpp (+14)
(added) llvm/test/Transforms/CorrelatedValuePropagation/umax.ll (+58)

diff --git a/llvm/lib/Analysis/LazyValueInfo.cpp b/llvm/lib/Analysis/LazyValueInfo.cpp
index 10ad4708596cb3..f29777a584772d 100644
--- a/llvm/lib/Analysis/LazyValueInfo.cpp
+++ b/llvm/lib/Analysis/LazyValueInfo.cpp
@@ -1127,6 +1127,20 @@ std::optional<ValueLatticeElement> LazyValueInfoImpl::getValueFromICmpCondition(
   if (!Ty->isIntegerTy())
     return ValueLatticeElement::getOverdefined();
 
+  // a - b or ptrtoint(a) - ptrtoint(b) ==/!= 0 if a ==/!= b
+  Value *X, *Y;
+  if (ICI->isEquality() && match(Val, m_Sub(m_Value(X), m_Value(Y)))) {
+    // Peek through ptrtoints
+    match(X, m_PtrToIntSameSize(DL, m_Value(X)));
+    match(Y, m_PtrToIntSameSize(DL, m_Value(Y)));
+    if ((X == LHS && Y == RHS) || (X == RHS && Y == LHS)) {
+      Constant *NullVal = Constant::getNullValue(Val->getType());
+      if (EdgePred == ICmpInst::ICMP_EQ)
+        return ValueLatticeElement::get(NullVal);
+      return ValueLatticeElement::getNot(NullVal);
+    }
+  }
+
   unsigned BitWidth = Ty->getScalarSizeInBits();
   APInt Offset(BitWidth, 0);
   if (matchICmpOperand(Offset, LHS, Val, EdgePred))
diff --git a/llvm/test/Transforms/CorrelatedValuePropagation/umax.ll b/llvm/test/Transforms/CorrelatedValuePropagation/umax.ll
new file mode 100644
index 00000000000000..5cd615e948adbe
--- /dev/null
+++ b/llvm/test/Transforms/CorrelatedValuePropagation/umax.ll
@@ -0,0 +1,58 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 5
+; RUN: opt < %s -passes=correlated-propagation -S | FileCheck %s
+
+target datalayout = "p:32:32"
+
+define i32 @infer_range_from_dom_equality(i32 %x, i32 %y) {
+; CHECK-LABEL: define range(i32 1, 0) i32 @infer_range_from_dom_equality(
+; CHECK-SAME: i32 [[X:%.*]], i32 [[Y:%.*]]) {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    [[COND:%.*]] = icmp eq i32 [[X]], [[Y]]
+; CHECK-NEXT:    [[SUB:%.*]] = sub i32 [[X]], [[Y]]
+; CHECK-NEXT:    br i1 [[COND]], label %[[IF_THEN:.*]], label %[[IF_ELSE:.*]]
+; CHECK:       [[IF_THEN]]:
+; CHECK-NEXT:    ret i32 1
+; CHECK:       [[IF_ELSE]]:
+; CHECK-NEXT:    ret i32 [[SUB]]
+;
+entry:
+  %cond = icmp eq i32 %x, %y
+  %sub = sub i32 %x, %y
+  br i1 %cond, label %if.then, label %if.else
+
+if.then:
+  %max1 = call i32 @llvm.umax.i32(i32 %sub, i32 1)
+  ret i32 %max1
+
+if.else:
+  %max2 = call i32 @llvm.umax.i32(i32 %sub, i32 1)
+  ret i32 %max2
+}
+
+define i32 @infer_range_from_dom_equality_ptrdiff(ptr %x, ptr %y) {
+; CHECK-LABEL: define range(i32 1, 0) i32 @infer_range_from_dom_equality_ptrdiff(
+; CHECK-SAME: ptr [[X:%.*]], ptr [[Y:%.*]]) {
+; CHECK-NEXT:    [[COND:%.*]] = icmp eq ptr [[X]], [[Y]]
+; CHECK-NEXT:    [[XI:%.*]] = ptrtoint ptr [[X]] to i32
+; CHECK-NEXT:    [[YI:%.*]] = ptrtoint ptr [[Y]] to i32
+; CHECK-NEXT:    [[SUB:%.*]] = sub i32 [[XI]], [[YI]]
+; CHECK-NEXT:    br i1 [[COND]], label %[[IF_THEN:.*]], label %[[IF_ELSE:.*]]
+; CHECK:       [[IF_THEN]]:
+; CHECK-NEXT:    ret i32 1
+; CHECK:       [[IF_ELSE]]:
+; CHECK-NEXT:    ret i32 [[SUB]]
+;
+  %cond = icmp eq ptr %x, %y
+  %xi = ptrtoint ptr %x to i32
+  %yi = ptrtoint ptr %y to i32
+  %sub = sub i32 %xi, %yi
+  br i1 %cond, label %if.then, label %if.else
+
+if.then:
+  %max1 = call i32 @llvm.umax.i32(i32 %sub, i32 1)
+  ret i32 %max1
+
+if.else:
+  %max2 = call i32 @llvm.umax.i32(i32 %sub, i32 1)
+  ret i32 %max2
+}

nikic

LGTM

llvm/lib/Analysis/LazyValueInfo.cpp

llvm/test/Transforms/CorrelatedValuePropagation/umax.ll

dtcxzyw · 2024-10-18T13:13:52Z

Failed Tests (3):
LLVM :: MC/ELF/warn-newline-in-escaped-string.s
LLVM :: TableGen/x86-fold-tables.td
LLVM :: tools/llvm-rc/tag-html.test

Looks unrelated.

dtcxzyw added 2 commits October 18, 2024 12:44

[CVP] Add pre-commit tests. NFC.

c34c8f9

[LVI] Infer non-zero from equality icmp

cee2b09

dtcxzyw requested review from fhahn, goldsteinn and preames October 18, 2024 05:03

dtcxzyw requested a review from nikic as a code owner October 18, 2024 05:03

llvmbot added llvm:analysis Includes value tracking, cost tables and constant folding llvm:transforms labels Oct 18, 2024

dtcxzyw mentioned this pull request Oct 18, 2024

Task submission dtcxzyw/llvm-opt-benchmark#1312

Open

dtcxzyw mentioned this pull request Oct 18, 2024

pre-commit: PR112838 dtcxzyw/llvm-opt-benchmark#1514

Closed

nikic approved these changes Oct 18, 2024

View reviewed changes

llvm/lib/Analysis/LazyValueInfo.cpp Outdated Show resolved Hide resolved

llvm/test/Transforms/CorrelatedValuePropagation/umax.ll Show resolved Hide resolved

[LVI] Address review comments.

2c3e039

dtcxzyw merged commit c89d731 into llvm:main Oct 18, 2024
6 of 8 checks passed

dtcxzyw deleted the perf/lvi-eq branch October 18, 2024 13:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[LVI] Infer non-zero from equality icmp #112838

[LVI] Infer non-zero from equality icmp #112838

Uh oh!

dtcxzyw commented Oct 18, 2024

Uh oh!

llvmbot commented Oct 18, 2024

Uh oh!

llvmbot commented Oct 18, 2024

Uh oh!

nikic left a comment

Uh oh!

Uh oh!

Uh oh!

dtcxzyw commented Oct 18, 2024

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[LVI] Infer non-zero from equality icmp #112838

[LVI] Infer non-zero from equality icmp #112838

Uh oh!

Conversation

dtcxzyw commented Oct 18, 2024

Uh oh!

llvmbot commented Oct 18, 2024

Uh oh!

llvmbot commented Oct 18, 2024

Uh oh!

nikic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

dtcxzyw commented Oct 18, 2024

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants