Skip to content

[CGP][PAC] Flip PHI and blends when all immediate modifiers are the same #150226

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: users/atrosinenko/pauth-imm-modifier-other
Choose a base branch
from
+217 −0
Open

[CGP][PAC] Flip PHI and blends when all immediate modifiers are the same #150226

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions llvm/lib/CodeGen/CodeGenPrepare.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -444,6 +444,7 @@ class CodeGenPrepare {
bool optimizeSwitchPhiConstants(SwitchInst *SI);
bool optimizeSwitchInst(SwitchInst *SI);
bool optimizeExtractElementInst(Instruction *Inst);
bool optimizePtrauthInst(Instruction *Inst, Value *Disc);
bool dupRetToEnableTailCallOpts(BasicBlock *BB, ModifyDT &ModifiedDT);
bool fixupDbgVariableRecord(DbgVariableRecord &I);
bool fixupDbgVariableRecordsOnInst(Instruction &I);
Expand Down Expand Up @@ -2765,6 +2766,12 @@ bool CodeGenPrepare::optimizeCallInst(CallInst *CI, ModifyDT &ModifiedDT) {
return optimizeGatherScatterInst(II, II->getArgOperand(0));
case Intrinsic::masked_scatter:
return optimizeGatherScatterInst(II, II->getArgOperand(1));
case Intrinsic::ptrauth_auth:
case Intrinsic::ptrauth_sign:
return optimizePtrauthInst(II, II->getArgOperand(2));
case Intrinsic::ptrauth_resign:
return optimizePtrauthInst(II, II->getArgOperand(2)) ||
optimizePtrauthInst(II, II->getArgOperand(4));
}

SmallVector<Value *, 2> PtrOps;
Expand Down Expand Up @@ -8309,6 +8316,74 @@ bool CodeGenPrepare::optimizeExtractElementInst(Instruction *Inst) {
return false;
}

// Given the instruction Inst, rewrite its discriminator operand Disc if it is
// a PHI node with all incoming values being @llvm.ptrauth.blend(addr, imm)
// with the same immediate modifier.
bool CodeGenPrepare::optimizePtrauthInst(Instruction *Inst, Value *Disc) {
Copy link
Contributor

@ojhunt ojhunt Jul 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This matching is fine for now, but the llvm intrinsic does not require the parameter to be a pointer. I think the best path forward would be to simply make the intrinsic match the expected interface? In principle no one should be emitting @llvm.ptrauth.blend(i64, (i64)pointer) (reversing the normal blend parameters), but maybe we should simply make it not an option to construct the intrinsic in that order?

That will require clang fixes to remove casts in the codegen, but also would presumably simplify things as it's removing those casts?

Maybe @ahmedbougacha or @rjmccall know why it is set up like this?

// GVN PRE, SimplifyCFG and possibly other passes may hoist or sink the call
// to @llvm.ptrauth.blend intrinsic, introducing multiple duplicate call
// instructions hidden behind a PHI node.
//
// To enforce the specific immediate modifier being blended into the
// discriminator even if the other, address component was reloaded from the
// stack, the AArch64 backend defines a number of pseudo instructions
// representing auth, sign and other operations. These pseudo instructions
// have separate register and immediate operands absorbing the arguments of
// blend intrinsic in case of a common `op(ptr, key_id, blend(addr, imm))`
// code pattern.
//
// To help the instruction selector, this function detects the discriminators
// represented by a PHI node with all incoming values being `blend`s with
// the same integer operand - each such discriminator is rewritten as a single
// blend, whose address operand is a PHI node.

PHINode *P = dyn_cast<PHINode>(Disc);
if (!P)
return false;

// Checks if V is blend(something, imm), returns imm if it is.
auto GetImmModifier = [](Value *V) -> std::optional<uint64_t> {
auto *II = dyn_cast<IntrinsicInst>(V);
if (!II || II->getIntrinsicID() != Intrinsic::ptrauth_blend)
return std::nullopt;
auto *ImmModifier = dyn_cast<ConstantInt>(II->getArgOperand(1));
if (!ImmModifier)
Copy link
Contributor

@ojhunt ojhunt Jul 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was about to ask "when is there no second argument" because apparently reading dyn_cast was too hard for me :D

but is this limitation strictly required? is the concern spilling a variable extra discriminator along the lines of?
(pseudo ir)

...
%a = call i64 @llvm.ptrauth.blend(i64 %addr.1, i64 %v1)
...
%b=  call i64 @llvm.ptrauth.blend(i64 %addr.2, i64 %v1)
...
%r = phi(%a,%b)

getting turned into

%v1 = ...
; spill %v1
...
; load %v1
%x = phi(%a, %b)
%r = call i64 @llvm.ptrauth.blend(i64 %x, i64 %v1)

return std::nullopt;
return ImmModifier->getZExtValue();
};

// Check that all incoming values of P are blends with the same immediate
// modifier.
std::optional<uint64_t> ImmModifier = GetImmModifier(*P->op_begin());
if (!ImmModifier ||
!llvm::all_of(P->operands(), [&ImmModifier, GetImmModifier](Value *V) {
return ImmModifier == GetImmModifier(V);
}))
return false;

// At this point, P is a PHI node, with all the incoming values being a blend
// operations with the same immediate modifier, but possibly different address
// modifiers. Replace Disc argument of Inst with a single ptrauth_blend
// whose address modifier if provided by a PHI node.

IRBuilder<> Builder(Inst->getContext());
Type *Int64Ty = Builder.getInt64Ty();

Builder.SetInsertPoint(P);
PHINode *AddrDiscPhi = Builder.CreatePHI(Int64Ty, P->getNumIncomingValues());
for (auto [Val, BB] : llvm::zip_equal(P->operands(), P->blocks())) {
Value *AddrDisc = cast<IntrinsicInst>(Val)->getArgOperand(0);
AddrDiscPhi->addIncoming(AddrDisc, BB);
}

Builder.SetInsertPoint(Inst);
Value *ImmModifValue = ConstantInt::get(Int64Ty, *ImmModifier);
Value *Blend = Builder.CreateIntrinsic(Intrinsic::ptrauth_blend,
{AddrDiscPhi, ImmModifValue});
Inst->replaceUsesOfWith(Disc, Blend);
return true;
}

/// For the instruction sequence of store below, F and I values
/// are bundled together as an i64 value before being stored into memory.
/// Sometimes it is more efficient to generate separate stores for F and I,
Expand Down
142 changes: 142 additions & 0 deletions llvm/test/Transforms/CodeGenPrepare/AArch64/ptrauth.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: -p --version 5
Copy link
Contributor

@ojhunt ojhunt Jul 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests look fine but I'd like a counter example with non-constant discriminator to track that behavior.

; RUN: opt -mtriple=aarch64-linux-pauthtest -S -passes='require<profile-summary>,function(codegenprepare)' < %s | FileCheck %s
; RUN: opt -mtriple=arm64e-apple-darwin -S -passes='require<profile-summary>,function(codegenprepare)' < %s | FileCheck %s

define i64 @choose_disc(i1 %cond, i64 %addr.1, i64 %addr.2, i64 %ptr) {
; CHECK-LABEL: define i64 @choose_disc(i1 %cond, i64 %addr.1, i64 %addr.2, i64 %ptr) {
; CHECK-NEXT: entry:
; CHECK-NEXT: br i1 %cond, label %bb.blend.1, label %bb.blend.2
; CHECK: bb.blend.1:
; CHECK-NEXT: %disc.1 = call i64 @llvm.ptrauth.blend(i64 %addr.1, i64 1234)
; CHECK-NEXT: br label %bb.user
; CHECK: bb.blend.2:
; CHECK-NEXT: %disc.2 = call i64 @llvm.ptrauth.blend(i64 %addr.2, i64 1234)
; CHECK-NEXT: br label %bb.user
; CHECK: bb.user:
; CHECK-NEXT: %0 = phi i64 [ %addr.1, %bb.blend.1 ], [ %addr.2, %bb.blend.2 ]
; CHECK-NEXT: %disc = phi i64 [ %disc.1, %bb.blend.1 ], [ %disc.2, %bb.blend.2 ]
; CHECK-NEXT: %1 = call i64 @llvm.ptrauth.blend(i64 %0, i64 1234)
; CHECK-NEXT: %result = call i64 @llvm.ptrauth.sign(i64 %ptr, i32 0, i64 %1)
; CHECK-NEXT: ret i64 %result
;
entry:
br i1 %cond, label %bb.blend.1, label %bb.blend.2
bb.blend.1:
%disc.1 = call i64 @llvm.ptrauth.blend(i64 %addr.1, i64 1234)
br label %bb.user
bb.blend.2:
%disc.2 = call i64 @llvm.ptrauth.blend(i64 %addr.2, i64 1234)
br label %bb.user
bb.user:
%disc = phi i64 [ %disc.1, %bb.blend.1 ], [ %disc.2, %bb.blend.2 ]
%result = call i64 @llvm.ptrauth.sign(i64 %ptr, i32 0, i64 %disc)
ret i64 %result
}

define i64 @choose_disc_different_imm(i1 %cond, i64 %addr.1, i64 %addr.2, i64 %ptr) {
; CHECK-LABEL: define i64 @choose_disc_different_imm(i1 %cond, i64 %addr.1, i64 %addr.2, i64 %ptr) {
; CHECK-NEXT: entry:
; CHECK-NEXT: br i1 %cond, label %bb.blend.1, label %bb.blend.2
; CHECK: bb.blend.1:
; CHECK-NEXT: %disc.1 = call i64 @llvm.ptrauth.blend(i64 %addr.1, i64 1234)
; CHECK-NEXT: br label %bb.user
; CHECK: bb.blend.2:
; CHECK-NEXT: %disc.2 = call i64 @llvm.ptrauth.blend(i64 %addr.2, i64 42)
; CHECK-NEXT: br label %bb.user
; CHECK: bb.user:
; CHECK-NEXT: %disc = phi i64 [ %disc.1, %bb.blend.1 ], [ %disc.2, %bb.blend.2 ]
; CHECK-NEXT: %result = call i64 @llvm.ptrauth.sign(i64 %ptr, i32 0, i64 %disc)
; CHECK-NEXT: ret i64 %result
;
entry:
br i1 %cond, label %bb.blend.1, label %bb.blend.2
bb.blend.1:
%disc.1 = call i64 @llvm.ptrauth.blend(i64 %addr.1, i64 1234)
br label %bb.user
bb.blend.2:
%disc.2 = call i64 @llvm.ptrauth.blend(i64 %addr.2, i64 42)
br label %bb.user
bb.user:
%disc = phi i64 [ %disc.1, %bb.blend.1 ], [ %disc.2, %bb.blend.2 ]
%result = call i64 @llvm.ptrauth.sign(i64 %ptr, i32 0, i64 %disc)
ret i64 %result
}

define i64 @choose_disc_multi_edge(i32 %arg, i64 %addr.1, i64 %addr.2, i64 %ptr) {
; CHECK-LABEL: define i64 @choose_disc_multi_edge(i32 %arg, i64 %addr.1, i64 %addr.2, i64 %ptr) {
; CHECK-NEXT: entry:
; CHECK-NEXT: %cond = icmp eq i32 %arg, 0
; CHECK-NEXT: br i1 %cond, label %bb.blend.1, label %bb.blend.2
; CHECK: bb.blend.1:
; CHECK-NEXT: %disc.1 = call i64 @llvm.ptrauth.blend(i64 %addr.1, i64 1234)
; CHECK-NEXT: switch i32 %arg, label %bb.blend.2 [
; CHECK-NEXT: i32 0, label %bb.user
; CHECK-NEXT: i32 3, label %bb.user
; CHECK-NEXT: ]
; CHECK: bb.blend.2:
; CHECK-NEXT: %disc.2 = call i64 @llvm.ptrauth.blend(i64 %addr.2, i64 1234)
; CHECK-NEXT: br label %bb.user
; CHECK: bb.user:
; CHECK-NEXT: %0 = phi i64 [ %addr.1, %bb.blend.1 ], [ %addr.1, %bb.blend.1 ], [ %addr.2, %bb.blend.2 ]
; CHECK-NEXT: %disc = phi i64 [ %disc.1, %bb.blend.1 ], [ %disc.1, %bb.blend.1 ], [ %disc.2, %bb.blend.2 ]
; CHECK-NEXT: %1 = call i64 @llvm.ptrauth.blend(i64 %0, i64 1234)
; CHECK-NEXT: %result = call i64 @llvm.ptrauth.sign(i64 %ptr, i32 0, i64 %1)
; CHECK-NEXT: ret i64 %result
;
entry:
%cond = icmp eq i32 %arg, 0
br i1 %cond, label %bb.blend.1, label %bb.blend.2
bb.blend.1:
%disc.1 = call i64 @llvm.ptrauth.blend(i64 %addr.1, i64 1234)
switch i32 %arg, label %bb.blend.2 [
i32 0, label %bb.user
i32 2, label %bb.blend.2
i32 3, label %bb.user
]
bb.blend.2:
%disc.2 = call i64 @llvm.ptrauth.blend(i64 %addr.2, i64 1234)
br label %bb.user
bb.user:
%disc = phi i64 [ %disc.1, %bb.blend.1 ], [ %disc.1, %bb.blend.1 ], [ %disc.2, %bb.blend.2 ]
%result = call i64 @llvm.ptrauth.sign(i64 %ptr, i32 0, i64 %disc)
ret i64 %result
}

define i64 @choose_disc_phi_placement(i1 %cond, i64 %addr.1, i64 %addr.2, i64 %ptr) {
; CHECK-LABEL: define i64 @choose_disc_phi_placement(i1 %cond, i64 %addr.1, i64 %addr.2, i64 %ptr) {
; CHECK-NEXT: entry:
; CHECK-NEXT: br i1 %cond, label %bb.blend.1, label %bb.blend.2
; CHECK: bb.blend.1:
; CHECK-NEXT: %disc.1 = call i64 @llvm.ptrauth.blend(i64 %addr.1, i64 1234)
; CHECK-NEXT: br label %bb.phi
; CHECK: bb.blend.2:
; CHECK-NEXT: %disc.2 = call i64 @llvm.ptrauth.blend(i64 %addr.2, i64 1234)
; CHECK-NEXT: br label %bb.phi
; CHECK: bb.phi:
; CHECK-NEXT: %0 = phi i64 [ %addr.1, %bb.blend.1 ], [ %addr.2, %bb.blend.2 ]
; CHECK-NEXT: %disc = phi i64 [ %disc.1, %bb.blend.1 ], [ %disc.2, %bb.blend.2 ]
; CHECK-NEXT: br i1 %cond, label %bb.user, label %bb.ret0
; CHECK: bb.user:
; CHECK-NEXT: %1 = call i64 @llvm.ptrauth.blend(i64 %0, i64 1234)
; CHECK-NEXT: %result = call i64 @llvm.ptrauth.sign(i64 %ptr, i32 0, i64 %1)
; CHECK-NEXT: ret i64 %result
; CHECK: bb.ret0:
; CHECK-NEXT: ret i64 0
;
entry:
br i1 %cond, label %bb.blend.1, label %bb.blend.2
bb.blend.1:
%disc.1 = call i64 @llvm.ptrauth.blend(i64 %addr.1, i64 1234)
br label %bb.phi
bb.blend.2:
%disc.2 = call i64 @llvm.ptrauth.blend(i64 %addr.2, i64 1234)
br label %bb.phi
bb.phi:
%disc = phi i64 [ %disc.1, %bb.blend.1 ], [ %disc.2, %bb.blend.2 ]
br i1 %cond, label %bb.user, label %bb.ret0
bb.user:
%result = call i64 @llvm.ptrauth.sign(i64 %ptr, i32 0, i64 %disc)
ret i64 %result
bb.ret0:
ret i64 0
}
Loading