Skip to content

[InstCombine][VectorCombine][NFC] Unify uses of lossless inverse cast #156597

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Sep 8, 2025
Merged

[InstCombine][VectorCombine][NFC] Unify uses of lossless inverse cast #156597

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
e279282
[InstCombine][VectorCombine] Unify uses of lossless inverse cast
XChy Sep 3, 2025
3bc3ce2
format
XChy Sep 3, 2025
1b9aa50
Remove old helper functions
XChy Sep 3, 2025
fe59e8b
Add signed/unsigned trunc wrapper
XChy Sep 3, 2025
14a4e42
refine comment
XChy Sep 5, 2025
f774ff8
Merge branch 'main' into invcast-instcombine-vectorcombine
XChy Sep 8, 2025
3b60666
resolve conflict
XChy Sep 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions llvm/include/llvm/Analysis/ConstantFolding.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,18 @@ LLVM_ABI bool isMathLibCallNoop(const CallBase *Call,

LLVM_ABI Constant *ReadByteArrayFromGlobal(const GlobalVariable *GV,
uint64_t Offset);

struct PreservedCastFlags {
bool NNeg = false;
bool NUW = false;
bool NSW = false;
};

/// Try to cast C to InvC losslessly, satisfying CastOp(InvC) == C.
Copy link
Member

@dtcxzyw dtcxzyw Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CastOp(InvC) == C is imprecise (e.g., bitcast <2 x i16> <i16 0, i16 poison> to i32 is refined to 0).

Copy link
Member Author

@XChy XChy Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. What about CastOp(InvC) == C or CastOp(InvC) is one of the possible values of C if C is undefined

/// Will try best to preserve the flags.
LLVM_ABI Constant *getLosslessInvCast(Constant *C, Type *InvCastTo,
unsigned CastOp, const DataLayout &DL,
PreservedCastFlags *Flags = nullptr);
}

#endif
40 changes: 40 additions & 0 deletions llvm/lib/Analysis/ConstantFolding.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4608,4 +4608,44 @@ bool llvm::isMathLibCallNoop(const CallBase *Call,
return false;
}

LLVM_ABI Constant *llvm::getLosslessInvCast(Constant *C, Type *InvCastTo,
unsigned CastOp,
const DataLayout &DL,
PreservedCastFlags *Flags) {
switch (CastOp) {
case Instruction::BitCast:
// Bitcast is always lossless.
return ConstantFoldCastOperand(Instruction::BitCast, C, InvCastTo, DL);
case Instruction::Trunc: {
auto *ZExtC = ConstantFoldCastOperand(Instruction::ZExt, C, InvCastTo, DL);
if (Flags) {
// Truncation back on ZExt value is always NUW.
Flags->NUW = true;
// Test positivity of C.
auto *SExtC =
ConstantFoldCastOperand(Instruction::SExt, C, InvCastTo, DL);
Flags->NSW = ZExtC == SExtC;
}
return ZExtC;
}
case Instruction::SExt:
case Instruction::ZExt: {
auto *InvC = ConstantExpr::getTrunc(C, InvCastTo);
auto *CastInvC = ConstantFoldCastOperand(CastOp, InvC, C->getType(), DL);
// Must satisfy CastOp(InvC) == C.
if (!CastInvC || CastInvC != C)
return nullptr;
if (Flags && CastOp == Instruction::ZExt) {
auto *SExtInvC =
ConstantFoldCastOperand(Instruction::SExt, InvC, C->getType(), DL);
// Test positivity of InvC.
Flags->NNeg = CastInvC == SExtInvC;
}
return InvC;
}
default:
return nullptr;
}
}

void TargetFolder::anchor() {}
7 changes: 5 additions & 2 deletions llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1799,16 +1799,19 @@ static Instruction *foldLogicCastConstant(BinaryOperator &Logic, CastInst *Cast,
// type may provide more information to later folds, and the smaller logic
// instruction may be cheaper (particularly in the case of vectors).
Value *X;
auto &DL = IC.getDataLayout();
if (match(Cast, m_OneUse(m_ZExt(m_Value(X))))) {
if (Constant *TruncC = IC.getLosslessUnsignedTrunc(C, SrcTy)) {
if (Constant *TruncC =
getLosslessInvCast(C, SrcTy, Instruction::ZExt, DL)) {
Copy link
Contributor

@nikic nikic Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also remove the old helper functions, which are presumably unused now?

TBH I find the new API super confusing. This is doing a truncate, but it's phrased in terms of a ZExt.

Copy link
Member Author

@XChy XChy Sep 3, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also remove the old helper functions, which are presumably unused now?

Oh sure. I missed it.

TBH I find the new API super confusing. This is doing a truncate, but it's phrased in terms of a ZExt.

It's doing the inverse cast of ZExt, to find a constant InvC, s.t. ZExt(InvC) == C. Or we still extract the ZExt/SExt part and name it getLosslessTrunc?

Copy link
Collaborator

@RKSimon RKSimon Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its messy, but we could keep the getLosslessUnsignedTrunc/getLosslessSignedTrunc function names and change them to wrappers for getLosslessInvCast?

// LogicOpc (zext X), C --> zext (LogicOpc X, C)
Value *NewOp = IC.Builder.CreateBinOp(LogicOpc, X, TruncC);
return new ZExtInst(NewOp, DestTy);
}
}

if (match(Cast, m_OneUse(m_SExtLike(m_Value(X))))) {
if (Constant *TruncC = IC.getLosslessSignedTrunc(C, SrcTy)) {
if (Constant *TruncC =
getLosslessInvCast(C, SrcTy, Instruction::SExt, DL)) {
// LogicOpc (sext X), C --> sext (LogicOpc X, C)
Value *NewOp = IC.Builder.CreateBinOp(LogicOpc, X, TruncC);
return new SExtInst(NewOp, DestTy);
Expand Down
6 changes: 4 additions & 2 deletions llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1956,7 +1956,8 @@ Instruction *InstCombinerImpl::visitCallInst(CallInst &CI) {
Constant *C;
if (match(I0, m_ZExt(m_Value(X))) && match(I1, m_Constant(C)) &&
I0->hasOneUse()) {
if (Constant *NarrowC = getLosslessUnsignedTrunc(C, X->getType())) {
if (Constant *NarrowC =
getLosslessInvCast(C, X->getType(), Instruction::ZExt, DL)) {
Value *NarrowMaxMin = Builder.CreateBinaryIntrinsic(IID, X, NarrowC);
return CastInst::Create(Instruction::ZExt, NarrowMaxMin, II->getType());
}
Expand Down Expand Up @@ -2006,7 +2007,8 @@ Instruction *InstCombinerImpl::visitCallInst(CallInst &CI) {
Constant *C;
if (match(I0, m_SExt(m_Value(X))) && match(I1, m_Constant(C)) &&
I0->hasOneUse()) {
if (Constant *NarrowC = getLosslessSignedTrunc(C, X->getType())) {
if (Constant *NarrowC =
getLosslessInvCast(C, X->getType(), Instruction::SExt, DL)) {
Value *NarrowMaxMin = Builder.CreateBinaryIntrinsic(IID, X, NarrowC);
return CastInst::Create(Instruction::SExt, NarrowMaxMin, II->getType());
}
Expand Down
7 changes: 5 additions & 2 deletions llvm/lib/Transforms/InstCombine/InstCombineMulDivRem.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1642,10 +1642,12 @@ static Instruction *narrowUDivURem(BinaryOperator &I,
}

Constant *C;
auto &DL = IC.getDataLayout();
if (isa<Instruction>(N) && match(N, m_OneUse(m_ZExt(m_Value(X)))) &&
match(D, m_Constant(C))) {
// If the constant is the same in the smaller type, use the narrow version.
Constant *TruncC = IC.getLosslessUnsignedTrunc(C, X->getType());
Constant *TruncC =
getLosslessInvCast(C, X->getType(), Instruction::ZExt, DL);
if (!TruncC)
return nullptr;

Expand All @@ -1656,7 +1658,8 @@ static Instruction *narrowUDivURem(BinaryOperator &I,
if (isa<Instruction>(D) && match(D, m_OneUse(m_ZExt(m_Value(X)))) &&
match(N, m_Constant(C))) {
// If the constant is the same in the smaller type, use the narrow version.
Constant *TruncC = IC.getLosslessUnsignedTrunc(C, X->getType());
Constant *TruncC =
getLosslessInvCast(C, X->getType(), Instruction::ZExt, DL);
if (!TruncC)
return nullptr;

Expand Down
3 changes: 2 additions & 1 deletion llvm/lib/Transforms/InstCombine/InstCombinePHI.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -841,7 +841,8 @@ Instruction *InstCombinerImpl::foldPHIArgZextsIntoPHI(PHINode &Phi) {
NumZexts++;
} else if (auto *C = dyn_cast<Constant>(V)) {
// Make sure that constants can fit in the new type.
Constant *Trunc = getLosslessUnsignedTrunc(C, NarrowType);
Constant *Trunc =
getLosslessInvCast(C, NarrowType, Instruction::ZExt, DL);
if (!Trunc)
return nullptr;
NewIncoming.push_back(Trunc);
Expand Down
2 changes: 1 addition & 1 deletion llvm/lib/Transforms/InstCombine/InstCombineSelect.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2375,7 +2375,7 @@ Instruction *InstCombinerImpl::foldSelectExtConst(SelectInst &Sel) {
// If the constant is the same after truncation to the smaller type and
// extension to the original type, we can narrow the select.
Type *SelType = Sel.getType();
Constant *TruncC = getLosslessTrunc(C, SmallType, ExtOpcode);
Constant *TruncC = getLosslessInvCast(C, SmallType, ExtOpcode, DL);
if (TruncC && ExtInst->hasOneUse()) {
Value *TruncCVal = cast<Value>(TruncC);
if (ExtInst == Sel.getFalseValue())
Expand Down
2 changes: 1 addition & 1 deletion llvm/lib/Transforms/InstCombine/InstructionCombining.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2568,7 +2568,7 @@ Instruction *InstCombinerImpl::narrowMathIfNoOverflow(BinaryOperator &BO) {
Constant *WideC;
if (!Op0->hasOneUse() || !match(Op1, m_Constant(WideC)))
return nullptr;
Constant *NarrowC = getLosslessTrunc(WideC, X->getType(), CastOpc);
Constant *NarrowC = getLosslessInvCast(WideC, X->getType(), CastOpc, DL);
if (!NarrowC)
return nullptr;
Y = NarrowC;
Expand Down
47 changes: 1 addition & 46 deletions llvm/lib/Transforms/Vectorize/VectorCombine.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -938,51 +938,6 @@ bool VectorCombine::foldBitOpOfCastops(Instruction &I) {
return true;
}

struct PreservedCastFlags {
bool NNeg = false;
bool NUW = false;
bool NSW = false;
};

// Try to cast C to InvC losslessly, satisfying CastOp(InvC) == C.
// Will try best to preserve the flags.
static Constant *getLosslessInvCast(Constant *C, Type *InvCastTo,
Instruction::CastOps CastOp,
const DataLayout &DL,
PreservedCastFlags &Flags) {
switch (CastOp) {
case Instruction::BitCast:
// Bitcast is always lossless.
return ConstantFoldCastOperand(Instruction::BitCast, C, InvCastTo, DL);
case Instruction::Trunc: {
auto *ZExtC = ConstantFoldCastOperand(Instruction::ZExt, C, InvCastTo, DL);
auto *SExtC = ConstantFoldCastOperand(Instruction::SExt, C, InvCastTo, DL);
// Truncation back on ZExt value is always NUW.
Flags.NUW = true;
// Test positivity of C.
Flags.NSW = ZExtC == SExtC;
return ZExtC;
}
case Instruction::SExt:
case Instruction::ZExt: {
auto *InvC = ConstantExpr::getTrunc(C, InvCastTo);
auto *CastInvC = ConstantFoldCastOperand(CastOp, InvC, C->getType(), DL);
// Must satisfy CastOp(InvC) == C.
if (!CastInvC || CastInvC != C)
return nullptr;
if (CastOp == Instruction::ZExt) {
auto *SExtInvC =
ConstantFoldCastOperand(Instruction::SExt, InvC, C->getType(), DL);
// Test positivity of InvC.
Flags.NNeg = CastInvC == SExtInvC;
}
return InvC;
}
default:
return nullptr;
}
}

/// Match:
// bitop(castop(x), C) ->
// bitop(castop(x), castop(InvC)) ->
Expand Down Expand Up @@ -1025,7 +980,7 @@ bool VectorCombine::foldBitOpOfCastConstant(Instruction &I) {

// Find the constant InvC, such that castop(InvC) equals to C.
PreservedCastFlags RHSFlags;
Constant *InvC = getLosslessInvCast(C, SrcVecTy, CastOpcode, *DL, RHSFlags);
Constant *InvC = getLosslessInvCast(C, SrcVecTy, CastOpcode, *DL, &RHSFlags);
if (!InvC)
return false;

Expand Down
Loading