-
Notifications
You must be signed in to change notification settings - Fork 15.4k
[AIX] Implement the ifunc attribute. #153049
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
@llvm/pr-subscribers-clang-driver @llvm/pr-subscribers-backend-powerpc Author: Wael Yehia (w2yehia) ChangesFull diff: https://github.com/llvm/llvm-project/pull/153049.diff 10 Files Affected:
diff --git a/clang/include/clang/Basic/TargetInfo.h b/clang/include/clang/Basic/TargetInfo.h
index ce4677e540226..ffeab27b67911 100644
--- a/clang/include/clang/Basic/TargetInfo.h
+++ b/clang/include/clang/Basic/TargetInfo.h
@@ -1545,6 +1545,8 @@ class TargetInfo : public TransferrableTargetInfo,
return true;
if (getTriple().getArch() == llvm::Triple::ArchType::avr)
return true;
+ if (getTriple().isOSAIX())
+ return true;
return getTriple().isOSBinFormatELF() &&
((getTriple().isOSLinux() && !getTriple().isMusl()) ||
getTriple().isOSFreeBSD());
diff --git a/llvm/include/llvm/IR/IntrinsicsPowerPC.td b/llvm/include/llvm/IR/IntrinsicsPowerPC.td
index 7dd9ff7f08b8b..75b64fba6e3ca 100644
--- a/llvm/include/llvm/IR/IntrinsicsPowerPC.td
+++ b/llvm/include/llvm/IR/IntrinsicsPowerPC.td
@@ -2090,3 +2090,13 @@ let TargetPrefix = "ppc" in {
Intrinsic<[], [llvm_i64_ty, llvm_i64_ty, llvm_ptr_ty],
[IntrArgMemOnly, IntrWriteMem, NoCapture<ArgIndex<2>>]>;
}
+
+
+//===----------------------------------------------------------------------===//
+// XCOFF Intrinsics
+let TargetPrefix = "ppc" in {
+ def int_ppc_get_function_entry :
+ DefaultAttrsIntrinsic<[llvm_ptr_ty], [llvm_ptr_ty], [IntrNoMem]>;
+ def int_ppc_get_function_descriptor :
+ DefaultAttrsIntrinsic<[llvm_ptr_ty], [llvm_ptr_ty], [IntrNoMem]>;
+}
diff --git a/llvm/include/llvm/Transforms/Utils/ModuleUtils.h b/llvm/include/llvm/Transforms/Utils/ModuleUtils.h
index 4036c4e947c75..6be7ac80dd428 100644
--- a/llvm/include/llvm/Transforms/Utils/ModuleUtils.h
+++ b/llvm/include/llvm/Transforms/Utils/ModuleUtils.h
@@ -159,6 +159,13 @@ LLVM_ABI bool
lowerGlobalIFuncUsersAsGlobalCtor(Module &M,
ArrayRef<GlobalIFunc *> IFuncsToLower = {});
+/// AIX specific lowering of ifuncs where we convert an ifunc to a regular
+/// function with the following implementation:
+/// Check if the function's descriptor still points to itself (true on first
+/// entry), if so then call the resolver function and atomically store the
+/// resulting function pointer into the descriptor. Make an indirect call
+/// through the function pointer in the descriptor.
+LLVM_ABI void lowerIFuncsOnAIX(Module &M);
} // End llvm namespace
#endif // LLVM_TRANSFORMS_UTILS_MODULEUTILS_H
diff --git a/llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp b/llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp
index 2ab2c147be0ec..957b2b3e7555d 100644
--- a/llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp
+++ b/llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp
@@ -763,6 +763,8 @@ static MCSymbol *getMCSymbolForTOCPseudoMO(const MachineOperand &MO,
return AP.GetJTISymbol(MO.getIndex());
case MachineOperand::MO_BlockAddress:
return AP.GetBlockAddressSymbol(MO.getBlockAddress());
+ case MachineOperand::MO_MCSymbol:
+ return MO.getMCSymbol();
default:
llvm_unreachable("Unexpected operand type to get symbol.");
}
@@ -792,6 +794,8 @@ getTOCEntryTypeForMO(const MachineOperand &MO) {
return PPCAsmPrinter::TOCType_JumpTable;
case MachineOperand::MO_BlockAddress:
return PPCAsmPrinter::TOCType_BlockAddress;
+ case MachineOperand::MO_MCSymbol:
+ return PPCAsmPrinter::TOCType_GlobalExternal; // TODO
default:
llvm_unreachable("Unexpected operand type to get TOC type.");
}
@@ -1043,7 +1047,7 @@ void PPCAsmPrinter::emitInstruction(const MachineInstr *MI) {
TmpInst.setOpcode(PPC::LWZ);
const MachineOperand &MO = MI->getOperand(1);
- assert((MO.isGlobal() || MO.isCPI() || MO.isJTI() || MO.isBlockAddress()) &&
+ assert((MO.isGlobal() || MO.isCPI() || MO.isJTI() || MO.isBlockAddress() || MO.isMCSymbol()) &&
"Invalid operand for LWZtoc.");
// Map the operand to its corresponding MCSymbol.
@@ -1127,7 +1131,7 @@ void PPCAsmPrinter::emitInstruction(const MachineInstr *MI) {
TmpInst.setOpcode(PPC::LD);
const MachineOperand &MO = MI->getOperand(1);
- assert((MO.isGlobal() || MO.isCPI() || MO.isJTI() || MO.isBlockAddress()) &&
+ assert((MO.isGlobal() || MO.isCPI() || MO.isJTI() || MO.isBlockAddress() || MO.isMCSymbol()) &&
"Invalid operand!");
// Map the operand to its corresponding MCSymbol.
diff --git a/llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp b/llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp
index 415164fc9e2cb..921cb72a83030 100644
--- a/llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp
+++ b/llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp
@@ -6158,6 +6158,10 @@ void PPCDAGToDAGISel::Select(SDNode *N) {
replaceWith(PPC::ADDItoc8, N, MVT::i64);
return;
}
+ if (N->getOperand(0).getOpcode() == ISD::MCSymbol) {
+ replaceWith(PPC::LDtoc, N, MVT::i64);
+ return;
+ }
// Break if it doesn't have toc data attribute. Proceed with common
// SelectCode.
break;
diff --git a/llvm/lib/Target/PowerPC/PPCISelLowering.cpp b/llvm/lib/Target/PowerPC/PPCISelLowering.cpp
index 74ae8502dccea..bbdd573f4a81b 100644
--- a/llvm/lib/Target/PowerPC/PPCISelLowering.cpp
+++ b/llvm/lib/Target/PowerPC/PPCISelLowering.cpp
@@ -11167,6 +11167,32 @@ SDValue PPCTargetLowering::LowerINTRINSIC_WO_CHAIN(SDValue Op,
SDLoc dl(Op);
switch (IntrinsicID) {
+ case Intrinsic::ppc_get_function_descriptor:
+ return Op.getOperand(1);
+ case Intrinsic::ppc_get_function_entry: {
+ SDValue Op1 = Op.getOperand(1);
+ if (auto *G = dyn_cast<GlobalAddressSDNode>(Op1)) {
+ const GlobalValue *GV = G->getGlobal();
+ assert(isFunctionGlobalAddress(GV));
+ assert(Subtarget.isAIXABI());
+ assert(!isa<GlobalIFunc>(GV) && "IFunc is not supported on AIX.");
+ const TargetMachine &TM = Subtarget.getTargetMachine();
+ const TargetLoweringObjectFile *TLOF = TM.getObjFileLowering();
+ MCSymbolXCOFF *S =
+ cast<MCSymbolXCOFF>(TLOF->getFunctionEntryPointSymbol(GV, TM));
+
+ MVT PtrVT = DAG.getTargetLoweringInfo().getPointerTy(DAG.getDataLayout());
+ SDValue EntryPointSym = DAG.getMCSymbol(S, PtrVT);
+ return getTOCEntry(DAG, dl, EntryPointSym);
+ }
+ assert (Op1.getOpcode() == ISD::CopyFromReg);
+ SDLoc dl(Op);
+ EVT PtrVT = getPointerTy(DAG.getDataLayout());
+ SDValue Result = DAG.getLoad(
+ PtrVT, dl, DAG.getEntryNode(), Op1, MachinePointerInfo(), DAG.getDataLayout().getPointerABIAlignment(0),
+ MachineMemOperand::MODereferenceable);
+ return Result;
+ }
case Intrinsic::thread_pointer:
// Reads the thread pointer register, used for __builtin_thread_pointer.
if (Subtarget.isPPC64())
diff --git a/llvm/lib/Target/PowerPC/PPCTargetMachine.cpp b/llvm/lib/Target/PowerPC/PPCTargetMachine.cpp
index b5c6ac111dff0..d8e840912bc37 100644
--- a/llvm/lib/Target/PowerPC/PPCTargetMachine.cpp
+++ b/llvm/lib/Target/PowerPC/PPCTargetMachine.cpp
@@ -36,6 +36,7 @@
#include "llvm/InitializePasses.h"
#include "llvm/MC/TargetRegistry.h"
#include "llvm/Pass.h"
+#include "llvm/Passes/PassBuilder.h"
#include "llvm/Support/CodeGen.h"
#include "llvm/Support/CommandLine.h"
#include "llvm/Support/Compiler.h"
@@ -43,6 +44,7 @@
#include "llvm/Target/TargetOptions.h"
#include "llvm/TargetParser/Triple.h"
#include "llvm/Transforms/Scalar.h"
+#include "llvm/Transforms/Utils/LowerIFunc.h"
#include <cassert>
#include <memory>
#include <optional>
@@ -452,6 +454,15 @@ class PPCPassConfig : public TargetPassConfig {
} // end anonymous namespace
+void PPCTargetMachine::registerPassBuilderCallbacks(PassBuilder &PB) {
+ if (getTargetTriple().isOSAIX())
+ PB.registerOptimizerLastEPCallback(
+ [](ModulePassManager &PM, OptimizationLevel Level, ThinOrFullLTOPhase Phase) {
+ if (Phase == ThinOrFullLTOPhase::None)
+ PM.addPass(LowerIFuncPass());
+ });
+}
+
TargetPassConfig *PPCTargetMachine::createPassConfig(PassManagerBase &PM) {
return new PPCPassConfig(*this, PM);
}
diff --git a/llvm/lib/Target/PowerPC/PPCTargetMachine.h b/llvm/lib/Target/PowerPC/PPCTargetMachine.h
index cb02b446fadb3..608750f7a3f2b 100644
--- a/llvm/lib/Target/PowerPC/PPCTargetMachine.h
+++ b/llvm/lib/Target/PowerPC/PPCTargetMachine.h
@@ -54,6 +54,8 @@ class PPCTargetMachine final : public CodeGenTargetMachineImpl {
// Pass Pipeline Configuration
TargetPassConfig *createPassConfig(PassManagerBase &PM) override;
+ void registerPassBuilderCallbacks(PassBuilder &PB) override;
+
TargetTransformInfo getTargetTransformInfo(const Function &F) const override;
TargetLoweringObjectFile *getObjFileLowering() const override {
diff --git a/llvm/lib/Transforms/Utils/LowerIFunc.cpp b/llvm/lib/Transforms/Utils/LowerIFunc.cpp
index 18ae0bbe2e731..991890aa56b83 100644
--- a/llvm/lib/Transforms/Utils/LowerIFunc.cpp
+++ b/llvm/lib/Transforms/Utils/LowerIFunc.cpp
@@ -13,6 +13,7 @@
#include "llvm/Transforms/Utils/LowerIFunc.h"
#include "llvm/IR/Module.h"
#include "llvm/Pass.h"
+#include "llvm/TargetParser/Triple.h"
#include "llvm/Transforms/Utils/ModuleUtils.h"
using namespace llvm;
@@ -22,6 +23,10 @@ PreservedAnalyses LowerIFuncPass::run(Module &M, ModuleAnalysisManager &AM) {
if (M.ifunc_empty())
return PreservedAnalyses::all();
- lowerGlobalIFuncUsersAsGlobalCtor(M, {});
+ Triple TargetTriple(M.getTargetTriple());
+ if (TargetTriple.isOSAIX())
+ lowerIFuncsOnAIX(M);
+ else
+ lowerGlobalIFuncUsersAsGlobalCtor(M, {});
return PreservedAnalyses::none();
}
diff --git a/llvm/lib/Transforms/Utils/ModuleUtils.cpp b/llvm/lib/Transforms/Utils/ModuleUtils.cpp
index 596849ecab742..67da3e66cf16a 100644
--- a/llvm/lib/Transforms/Utils/ModuleUtils.cpp
+++ b/llvm/lib/Transforms/Utils/ModuleUtils.cpp
@@ -16,6 +16,8 @@
#include "llvm/IR/DerivedTypes.h"
#include "llvm/IR/Function.h"
#include "llvm/IR/IRBuilder.h"
+#include "llvm/IR/Intrinsics.h"
+#include "llvm/IR/IntrinsicsPowerPC.h"
#include "llvm/IR/MDBuilder.h"
#include "llvm/IR/Module.h"
#include "llvm/Support/Casting.h"
@@ -397,6 +399,85 @@ void llvm::embedBufferInModule(Module &M, MemoryBufferRef Buf,
appendToCompilerUsed(M, GV);
}
+void llvm::lowerIFuncsOnAIX(Module &M) {
+ for (GlobalIFunc &IFunc : make_early_inc_range(M.ifuncs())) {
+
+ // Let IFunc be:
+ // @foo = ifunc rt-type (type1, type2), ptr @resolver
+ assert(isa<llvm::FunctionType>(IFunc.getValueType()));
+ FunctionType *FTy = cast<llvm::FunctionType>(IFunc.getValueType());
+
+ // Create the regular (non-ifunc) function:
+ // define rt-type @foo(type1 %0, type2 %1)
+ Function *F = Function::Create(FTy, IFunc.getLinkage(), IFunc.getAddressSpace(), IFunc.getName(), &M);
+ LLVMContext &Ctx = F->getContext();
+
+ // entry:
+ BasicBlock *CurBlock = BasicBlock::Create(Ctx, "entry", F);
+ IRBuilder<> Builder(CurBlock);
+
+ PointerType *PtrTy = Builder.getPtrTy();
+ // %DescPtr = call ptr @ppc_get_function_descriptor(ptr noundef @foo)
+ auto *DescPtr = Builder.CreateIntrinsic(/*RetTy*/ PtrTy, Intrinsic::ppc_get_function_descriptor, {F}, {}, "desc.ptr");
+
+ // %DesctAddr = getelementptr inbounds %struct.Desc_t, ptr %DescPtr, i32 0, i32 0
+ StructType *DescriptorType = StructType::get(PtrTy, PtrTy, PtrTy);
+ auto *DesctAddr = Builder.CreateStructGEP(DescriptorType, DescPtr, 0, "desc_t.addr");
+
+ // %AddrInDesc = load ptr, ptr %DesctAddr, align 4
+ auto *AddrInDesc = Builder.CreateAlignedLoad(PtrTy, DesctAddr, DesctAddr->getPointerAlignment(M.getDataLayout()), "addr.in.desc");
+
+ // %OriginalAddr = call ptr @ppc_get_function_entry(ptr noundef @foo) #3
+ auto *OriginalAddr = Builder.CreateIntrinsic(/*RetTy*/ PtrTy, Intrinsic::ppc_get_function_entry, {F}, {}, "original.addr");
+
+ // %cmp = icmp eq ptr %AddrInDesc, %OriginalAddr
+ auto *CMP = Builder.CreateICmpEQ(AddrInDesc, OriginalAddr);
+
+ // br i1 %cmp, label %resolver, label %end
+ CurBlock = BasicBlock::Create(Ctx, "resolver", F);
+ BasicBlock *FinalBlock = BasicBlock::Create(Ctx, "end", F);
+ Builder.CreateCondBr(CMP, CurBlock, FinalBlock);
+
+ // Emit the 'then' (resolver) code:
+ Builder.SetInsertPoint(CurBlock);
+
+ // resolver:
+ // %ResolvedFunc = call ptr @resolver()
+ // %ResolvedAddr = call ptr @ppc_get_function_entry(ptr %ResolvedFunc)
+ // %4 = ptrtoint ptr %ResolvedAddr to i32
+ // store atomic i32 %4, ptr %desc_t.addr release, align 4
+ Function *ResolverFunc = IFunc.getResolverFunction();
+ auto *ResolvedFunc = Builder.CreateCall(ResolverFunc, ArrayRef<Value *>(), "resolved.func");
+ auto *ResolvedAddr = Builder.CreateIntrinsic(/*RetTy*/ PtrTy, Intrinsic::ppc_get_function_entry, {ResolvedFunc}, {}, "resolved.addr");
+ auto *PtrToInt = Builder.CreatePtrToInt(ResolvedAddr, Builder.getIntPtrTy(M.getDataLayout()));
+ // TODO fix alignment
+ Builder.CreateAlignedStore(PtrToInt, DesctAddr, MaybeAlign())->setAtomic(AtomicOrdering::Release);
+
+ // br label %if.end
+ Builder.CreateBr(FinalBlock);
+
+ // Emit the continuation block for code after the if.
+ Builder.SetInsertPoint(FinalBlock);
+
+ // %res = musttail call i32 %DescPtr(i32 noundef %a) #3
+ SmallVector<Value *, 10> Args(make_pointer_range(F->args()));
+ CallInst *Result = Builder.CreateCall(F->getFunctionType(), DescPtr, Args, "res");
+ //Result->setTailCallKind(CallInst::TCK_MustTail);
+
+ // ret i32 %res
+ if (F->getReturnType()->isVoidTy())
+ Builder.CreateRetVoid();
+ else
+ Builder.CreateRet(Result);
+
+ // replace all uses of the ifunc with the newly created function
+ IFunc.replaceAllUsesWith(F);
+
+ std::string name = IFunc.getName().str();
+ IFunc.eraseFromParent();
+ F->setName(name);
+ }
+}
bool llvm::lowerGlobalIFuncUsersAsGlobalCtor(
Module &M, ArrayRef<GlobalIFunc *> FilteredIFuncsToLower) {
SmallVector<GlobalIFunc *, 32> AllIFuncs;
|
|
@llvm/pr-subscribers-llvm-ir Author: Wael Yehia (w2yehia) ChangesFull diff: https://github.com/llvm/llvm-project/pull/153049.diff 10 Files Affected:
diff --git a/clang/include/clang/Basic/TargetInfo.h b/clang/include/clang/Basic/TargetInfo.h
index ce4677e540226..ffeab27b67911 100644
--- a/clang/include/clang/Basic/TargetInfo.h
+++ b/clang/include/clang/Basic/TargetInfo.h
@@ -1545,6 +1545,8 @@ class TargetInfo : public TransferrableTargetInfo,
return true;
if (getTriple().getArch() == llvm::Triple::ArchType::avr)
return true;
+ if (getTriple().isOSAIX())
+ return true;
return getTriple().isOSBinFormatELF() &&
((getTriple().isOSLinux() && !getTriple().isMusl()) ||
getTriple().isOSFreeBSD());
diff --git a/llvm/include/llvm/IR/IntrinsicsPowerPC.td b/llvm/include/llvm/IR/IntrinsicsPowerPC.td
index 7dd9ff7f08b8b..75b64fba6e3ca 100644
--- a/llvm/include/llvm/IR/IntrinsicsPowerPC.td
+++ b/llvm/include/llvm/IR/IntrinsicsPowerPC.td
@@ -2090,3 +2090,13 @@ let TargetPrefix = "ppc" in {
Intrinsic<[], [llvm_i64_ty, llvm_i64_ty, llvm_ptr_ty],
[IntrArgMemOnly, IntrWriteMem, NoCapture<ArgIndex<2>>]>;
}
+
+
+//===----------------------------------------------------------------------===//
+// XCOFF Intrinsics
+let TargetPrefix = "ppc" in {
+ def int_ppc_get_function_entry :
+ DefaultAttrsIntrinsic<[llvm_ptr_ty], [llvm_ptr_ty], [IntrNoMem]>;
+ def int_ppc_get_function_descriptor :
+ DefaultAttrsIntrinsic<[llvm_ptr_ty], [llvm_ptr_ty], [IntrNoMem]>;
+}
diff --git a/llvm/include/llvm/Transforms/Utils/ModuleUtils.h b/llvm/include/llvm/Transforms/Utils/ModuleUtils.h
index 4036c4e947c75..6be7ac80dd428 100644
--- a/llvm/include/llvm/Transforms/Utils/ModuleUtils.h
+++ b/llvm/include/llvm/Transforms/Utils/ModuleUtils.h
@@ -159,6 +159,13 @@ LLVM_ABI bool
lowerGlobalIFuncUsersAsGlobalCtor(Module &M,
ArrayRef<GlobalIFunc *> IFuncsToLower = {});
+/// AIX specific lowering of ifuncs where we convert an ifunc to a regular
+/// function with the following implementation:
+/// Check if the function's descriptor still points to itself (true on first
+/// entry), if so then call the resolver function and atomically store the
+/// resulting function pointer into the descriptor. Make an indirect call
+/// through the function pointer in the descriptor.
+LLVM_ABI void lowerIFuncsOnAIX(Module &M);
} // End llvm namespace
#endif // LLVM_TRANSFORMS_UTILS_MODULEUTILS_H
diff --git a/llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp b/llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp
index 2ab2c147be0ec..957b2b3e7555d 100644
--- a/llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp
+++ b/llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp
@@ -763,6 +763,8 @@ static MCSymbol *getMCSymbolForTOCPseudoMO(const MachineOperand &MO,
return AP.GetJTISymbol(MO.getIndex());
case MachineOperand::MO_BlockAddress:
return AP.GetBlockAddressSymbol(MO.getBlockAddress());
+ case MachineOperand::MO_MCSymbol:
+ return MO.getMCSymbol();
default:
llvm_unreachable("Unexpected operand type to get symbol.");
}
@@ -792,6 +794,8 @@ getTOCEntryTypeForMO(const MachineOperand &MO) {
return PPCAsmPrinter::TOCType_JumpTable;
case MachineOperand::MO_BlockAddress:
return PPCAsmPrinter::TOCType_BlockAddress;
+ case MachineOperand::MO_MCSymbol:
+ return PPCAsmPrinter::TOCType_GlobalExternal; // TODO
default:
llvm_unreachable("Unexpected operand type to get TOC type.");
}
@@ -1043,7 +1047,7 @@ void PPCAsmPrinter::emitInstruction(const MachineInstr *MI) {
TmpInst.setOpcode(PPC::LWZ);
const MachineOperand &MO = MI->getOperand(1);
- assert((MO.isGlobal() || MO.isCPI() || MO.isJTI() || MO.isBlockAddress()) &&
+ assert((MO.isGlobal() || MO.isCPI() || MO.isJTI() || MO.isBlockAddress() || MO.isMCSymbol()) &&
"Invalid operand for LWZtoc.");
// Map the operand to its corresponding MCSymbol.
@@ -1127,7 +1131,7 @@ void PPCAsmPrinter::emitInstruction(const MachineInstr *MI) {
TmpInst.setOpcode(PPC::LD);
const MachineOperand &MO = MI->getOperand(1);
- assert((MO.isGlobal() || MO.isCPI() || MO.isJTI() || MO.isBlockAddress()) &&
+ assert((MO.isGlobal() || MO.isCPI() || MO.isJTI() || MO.isBlockAddress() || MO.isMCSymbol()) &&
"Invalid operand!");
// Map the operand to its corresponding MCSymbol.
diff --git a/llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp b/llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp
index 415164fc9e2cb..921cb72a83030 100644
--- a/llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp
+++ b/llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp
@@ -6158,6 +6158,10 @@ void PPCDAGToDAGISel::Select(SDNode *N) {
replaceWith(PPC::ADDItoc8, N, MVT::i64);
return;
}
+ if (N->getOperand(0).getOpcode() == ISD::MCSymbol) {
+ replaceWith(PPC::LDtoc, N, MVT::i64);
+ return;
+ }
// Break if it doesn't have toc data attribute. Proceed with common
// SelectCode.
break;
diff --git a/llvm/lib/Target/PowerPC/PPCISelLowering.cpp b/llvm/lib/Target/PowerPC/PPCISelLowering.cpp
index 74ae8502dccea..bbdd573f4a81b 100644
--- a/llvm/lib/Target/PowerPC/PPCISelLowering.cpp
+++ b/llvm/lib/Target/PowerPC/PPCISelLowering.cpp
@@ -11167,6 +11167,32 @@ SDValue PPCTargetLowering::LowerINTRINSIC_WO_CHAIN(SDValue Op,
SDLoc dl(Op);
switch (IntrinsicID) {
+ case Intrinsic::ppc_get_function_descriptor:
+ return Op.getOperand(1);
+ case Intrinsic::ppc_get_function_entry: {
+ SDValue Op1 = Op.getOperand(1);
+ if (auto *G = dyn_cast<GlobalAddressSDNode>(Op1)) {
+ const GlobalValue *GV = G->getGlobal();
+ assert(isFunctionGlobalAddress(GV));
+ assert(Subtarget.isAIXABI());
+ assert(!isa<GlobalIFunc>(GV) && "IFunc is not supported on AIX.");
+ const TargetMachine &TM = Subtarget.getTargetMachine();
+ const TargetLoweringObjectFile *TLOF = TM.getObjFileLowering();
+ MCSymbolXCOFF *S =
+ cast<MCSymbolXCOFF>(TLOF->getFunctionEntryPointSymbol(GV, TM));
+
+ MVT PtrVT = DAG.getTargetLoweringInfo().getPointerTy(DAG.getDataLayout());
+ SDValue EntryPointSym = DAG.getMCSymbol(S, PtrVT);
+ return getTOCEntry(DAG, dl, EntryPointSym);
+ }
+ assert (Op1.getOpcode() == ISD::CopyFromReg);
+ SDLoc dl(Op);
+ EVT PtrVT = getPointerTy(DAG.getDataLayout());
+ SDValue Result = DAG.getLoad(
+ PtrVT, dl, DAG.getEntryNode(), Op1, MachinePointerInfo(), DAG.getDataLayout().getPointerABIAlignment(0),
+ MachineMemOperand::MODereferenceable);
+ return Result;
+ }
case Intrinsic::thread_pointer:
// Reads the thread pointer register, used for __builtin_thread_pointer.
if (Subtarget.isPPC64())
diff --git a/llvm/lib/Target/PowerPC/PPCTargetMachine.cpp b/llvm/lib/Target/PowerPC/PPCTargetMachine.cpp
index b5c6ac111dff0..d8e840912bc37 100644
--- a/llvm/lib/Target/PowerPC/PPCTargetMachine.cpp
+++ b/llvm/lib/Target/PowerPC/PPCTargetMachine.cpp
@@ -36,6 +36,7 @@
#include "llvm/InitializePasses.h"
#include "llvm/MC/TargetRegistry.h"
#include "llvm/Pass.h"
+#include "llvm/Passes/PassBuilder.h"
#include "llvm/Support/CodeGen.h"
#include "llvm/Support/CommandLine.h"
#include "llvm/Support/Compiler.h"
@@ -43,6 +44,7 @@
#include "llvm/Target/TargetOptions.h"
#include "llvm/TargetParser/Triple.h"
#include "llvm/Transforms/Scalar.h"
+#include "llvm/Transforms/Utils/LowerIFunc.h"
#include <cassert>
#include <memory>
#include <optional>
@@ -452,6 +454,15 @@ class PPCPassConfig : public TargetPassConfig {
} // end anonymous namespace
+void PPCTargetMachine::registerPassBuilderCallbacks(PassBuilder &PB) {
+ if (getTargetTriple().isOSAIX())
+ PB.registerOptimizerLastEPCallback(
+ [](ModulePassManager &PM, OptimizationLevel Level, ThinOrFullLTOPhase Phase) {
+ if (Phase == ThinOrFullLTOPhase::None)
+ PM.addPass(LowerIFuncPass());
+ });
+}
+
TargetPassConfig *PPCTargetMachine::createPassConfig(PassManagerBase &PM) {
return new PPCPassConfig(*this, PM);
}
diff --git a/llvm/lib/Target/PowerPC/PPCTargetMachine.h b/llvm/lib/Target/PowerPC/PPCTargetMachine.h
index cb02b446fadb3..608750f7a3f2b 100644
--- a/llvm/lib/Target/PowerPC/PPCTargetMachine.h
+++ b/llvm/lib/Target/PowerPC/PPCTargetMachine.h
@@ -54,6 +54,8 @@ class PPCTargetMachine final : public CodeGenTargetMachineImpl {
// Pass Pipeline Configuration
TargetPassConfig *createPassConfig(PassManagerBase &PM) override;
+ void registerPassBuilderCallbacks(PassBuilder &PB) override;
+
TargetTransformInfo getTargetTransformInfo(const Function &F) const override;
TargetLoweringObjectFile *getObjFileLowering() const override {
diff --git a/llvm/lib/Transforms/Utils/LowerIFunc.cpp b/llvm/lib/Transforms/Utils/LowerIFunc.cpp
index 18ae0bbe2e731..991890aa56b83 100644
--- a/llvm/lib/Transforms/Utils/LowerIFunc.cpp
+++ b/llvm/lib/Transforms/Utils/LowerIFunc.cpp
@@ -13,6 +13,7 @@
#include "llvm/Transforms/Utils/LowerIFunc.h"
#include "llvm/IR/Module.h"
#include "llvm/Pass.h"
+#include "llvm/TargetParser/Triple.h"
#include "llvm/Transforms/Utils/ModuleUtils.h"
using namespace llvm;
@@ -22,6 +23,10 @@ PreservedAnalyses LowerIFuncPass::run(Module &M, ModuleAnalysisManager &AM) {
if (M.ifunc_empty())
return PreservedAnalyses::all();
- lowerGlobalIFuncUsersAsGlobalCtor(M, {});
+ Triple TargetTriple(M.getTargetTriple());
+ if (TargetTriple.isOSAIX())
+ lowerIFuncsOnAIX(M);
+ else
+ lowerGlobalIFuncUsersAsGlobalCtor(M, {});
return PreservedAnalyses::none();
}
diff --git a/llvm/lib/Transforms/Utils/ModuleUtils.cpp b/llvm/lib/Transforms/Utils/ModuleUtils.cpp
index 596849ecab742..67da3e66cf16a 100644
--- a/llvm/lib/Transforms/Utils/ModuleUtils.cpp
+++ b/llvm/lib/Transforms/Utils/ModuleUtils.cpp
@@ -16,6 +16,8 @@
#include "llvm/IR/DerivedTypes.h"
#include "llvm/IR/Function.h"
#include "llvm/IR/IRBuilder.h"
+#include "llvm/IR/Intrinsics.h"
+#include "llvm/IR/IntrinsicsPowerPC.h"
#include "llvm/IR/MDBuilder.h"
#include "llvm/IR/Module.h"
#include "llvm/Support/Casting.h"
@@ -397,6 +399,85 @@ void llvm::embedBufferInModule(Module &M, MemoryBufferRef Buf,
appendToCompilerUsed(M, GV);
}
+void llvm::lowerIFuncsOnAIX(Module &M) {
+ for (GlobalIFunc &IFunc : make_early_inc_range(M.ifuncs())) {
+
+ // Let IFunc be:
+ // @foo = ifunc rt-type (type1, type2), ptr @resolver
+ assert(isa<llvm::FunctionType>(IFunc.getValueType()));
+ FunctionType *FTy = cast<llvm::FunctionType>(IFunc.getValueType());
+
+ // Create the regular (non-ifunc) function:
+ // define rt-type @foo(type1 %0, type2 %1)
+ Function *F = Function::Create(FTy, IFunc.getLinkage(), IFunc.getAddressSpace(), IFunc.getName(), &M);
+ LLVMContext &Ctx = F->getContext();
+
+ // entry:
+ BasicBlock *CurBlock = BasicBlock::Create(Ctx, "entry", F);
+ IRBuilder<> Builder(CurBlock);
+
+ PointerType *PtrTy = Builder.getPtrTy();
+ // %DescPtr = call ptr @ppc_get_function_descriptor(ptr noundef @foo)
+ auto *DescPtr = Builder.CreateIntrinsic(/*RetTy*/ PtrTy, Intrinsic::ppc_get_function_descriptor, {F}, {}, "desc.ptr");
+
+ // %DesctAddr = getelementptr inbounds %struct.Desc_t, ptr %DescPtr, i32 0, i32 0
+ StructType *DescriptorType = StructType::get(PtrTy, PtrTy, PtrTy);
+ auto *DesctAddr = Builder.CreateStructGEP(DescriptorType, DescPtr, 0, "desc_t.addr");
+
+ // %AddrInDesc = load ptr, ptr %DesctAddr, align 4
+ auto *AddrInDesc = Builder.CreateAlignedLoad(PtrTy, DesctAddr, DesctAddr->getPointerAlignment(M.getDataLayout()), "addr.in.desc");
+
+ // %OriginalAddr = call ptr @ppc_get_function_entry(ptr noundef @foo) #3
+ auto *OriginalAddr = Builder.CreateIntrinsic(/*RetTy*/ PtrTy, Intrinsic::ppc_get_function_entry, {F}, {}, "original.addr");
+
+ // %cmp = icmp eq ptr %AddrInDesc, %OriginalAddr
+ auto *CMP = Builder.CreateICmpEQ(AddrInDesc, OriginalAddr);
+
+ // br i1 %cmp, label %resolver, label %end
+ CurBlock = BasicBlock::Create(Ctx, "resolver", F);
+ BasicBlock *FinalBlock = BasicBlock::Create(Ctx, "end", F);
+ Builder.CreateCondBr(CMP, CurBlock, FinalBlock);
+
+ // Emit the 'then' (resolver) code:
+ Builder.SetInsertPoint(CurBlock);
+
+ // resolver:
+ // %ResolvedFunc = call ptr @resolver()
+ // %ResolvedAddr = call ptr @ppc_get_function_entry(ptr %ResolvedFunc)
+ // %4 = ptrtoint ptr %ResolvedAddr to i32
+ // store atomic i32 %4, ptr %desc_t.addr release, align 4
+ Function *ResolverFunc = IFunc.getResolverFunction();
+ auto *ResolvedFunc = Builder.CreateCall(ResolverFunc, ArrayRef<Value *>(), "resolved.func");
+ auto *ResolvedAddr = Builder.CreateIntrinsic(/*RetTy*/ PtrTy, Intrinsic::ppc_get_function_entry, {ResolvedFunc}, {}, "resolved.addr");
+ auto *PtrToInt = Builder.CreatePtrToInt(ResolvedAddr, Builder.getIntPtrTy(M.getDataLayout()));
+ // TODO fix alignment
+ Builder.CreateAlignedStore(PtrToInt, DesctAddr, MaybeAlign())->setAtomic(AtomicOrdering::Release);
+
+ // br label %if.end
+ Builder.CreateBr(FinalBlock);
+
+ // Emit the continuation block for code after the if.
+ Builder.SetInsertPoint(FinalBlock);
+
+ // %res = musttail call i32 %DescPtr(i32 noundef %a) #3
+ SmallVector<Value *, 10> Args(make_pointer_range(F->args()));
+ CallInst *Result = Builder.CreateCall(F->getFunctionType(), DescPtr, Args, "res");
+ //Result->setTailCallKind(CallInst::TCK_MustTail);
+
+ // ret i32 %res
+ if (F->getReturnType()->isVoidTy())
+ Builder.CreateRetVoid();
+ else
+ Builder.CreateRet(Result);
+
+ // replace all uses of the ifunc with the newly created function
+ IFunc.replaceAllUsesWith(F);
+
+ std::string name = IFunc.getName().str();
+ IFunc.eraseFromParent();
+ F->setName(name);
+ }
+}
bool llvm::lowerGlobalIFuncUsersAsGlobalCtor(
Module &M, ArrayRef<GlobalIFunc *> FilteredIFuncsToLower) {
SmallVector<GlobalIFunc *, 32> AllIFuncs;
|
You can test this locally with the following command:git-clang-format --diff origin/main HEAD --extensions cpp,h,c -- compiler-rt/lib/builtins/ppc/init_ifuncs.c llvm/lib/Target/PowerPC/PPCPrepareIFuncsOnAIX.cpp clang/include/clang/Basic/TargetInfo.h clang/lib/Driver/ToolChains/AIX.cpp clang/test/CodeGen/attr-ifunc.c clang/test/CodeGen/attr-ifunc.cpp clang/test/CodeGen/ifunc.c clang/test/SemaCXX/ifunc-has-attribute.cpp llvm/include/llvm/CodeGen/AsmPrinter.h llvm/include/llvm/CodeGen/TargetLoweringObjectFileImpl.h llvm/include/llvm/Target/TargetLoweringObjectFile.h llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp llvm/lib/Target/PowerPC/PPC.h llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp llvm/lib/Target/PowerPC/PPCISelLowering.cpp llvm/lib/Target/PowerPC/PPCTargetMachine.cpp --diff_from_common_commit
View the diff from clang-format here.diff --git a/llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp b/llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp
index 5c4f9362c..480a510c6 100644
--- a/llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp
+++ b/llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp
@@ -3444,9 +3444,10 @@ static bool TOCRestoreNeededForCallToImplementation(const GlobalIFunc &GI) {
}
return Res;
}
- // @switch.table.resolve_foo = private unnamed_addr constant [3 x ptr] [ptr @foo_static, ptr @foo_hidden, ptr @foo_protected]
- // %switch.gep = getelementptr inbounds nuw ptr, ptr @switch.table, i64 %2
- // V = load ptr, ptr %switch.gep,
+ // @switch.table.resolve_foo = private unnamed_addr constant [3 x ptr] [ptr
+ // @foo_static, ptr @foo_hidden, ptr @foo_protected] %switch.gep =
+ // getelementptr inbounds nuw ptr, ptr @switch.table, i64 %2 V = load ptr,
+ // ptr %switch.gep,
else if (auto *Op = getPointerOperand(I)) {
while (isa<GEPOperator>(Op))
Op = cast<GEPOperator>(Op)->getPointerOperand();
@@ -3587,7 +3588,8 @@ void PPCAIXAsmPrinter::emitGlobalIFunc(Module &M, const GlobalIFunc &GI) {
if (TOCRestoreNeededForCallToImplementation(GI)) {
reportFatalUsageError(
"unimplemented: TOC register save/restore needed for ifunc \"" +
- Twine(GI.getName()) + "\", because couldn't prove all candidates "
+ Twine(GI.getName()) +
+ "\", because couldn't prove all candidates "
"are static or hidden/protected visibility definitions");
return;
}
|
|
I mistakenly thought ifunc can be forward declared with enough information to indicate it's an ifunc, so that all callsites of ifunc can be converted to indirect calls through the function descriptor (which is something that was planned as an immediate next step after this PR or included in it). I had some offline discussions with @RolandF77 and @mandlebug, and two solutions were proposed:
In both approaches the original ifunc function becomes a wrapper that indirect calls the function pointer in the descriptor (no checks). We would prefer a solution that doesn't require any input from the user (in the form of extra flags or user code changes (such as decorating ifunc declarations with some extra info)). I will evaluate the two approaches and post an update here soon. |
w2yehia
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
part 1 group codereview
| @@ -4,6 +4,8 @@ | |||
| // RUN: %clang_cc1 -triple x86_64-apple-macosx -verify -emit-llvm-only %s | |||
| // RUN: %clang_cc1 -triple aarch64-none-linux-gnu -verify -emit-llvm-only %s | |||
| // RUN: %clang_cc1 -triple aarch64-pc-windows-msvcu -verify -emit-llvm-only %s | |||
| // RUN: %clang_cc1 -triple powerpc64-ibm-aix-xcoff -verify -emit-llvm-only %s | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shud have at least one 32-bit test here or in anothe LIT
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since the test case only test whether the source code is compiled pass or not ? do it matter 32bit or 64bit ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, maybe in this test it's not useful to test 32-bit, but we need some where to test that ifunc is supported in 32-bit AIX.
| @@ -4,6 +4,8 @@ | |||
| // RUN: %clang_cc1 -triple x86_64-apple-macosx -verify -emit-llvm-only %s | |||
| // RUN: %clang_cc1 -triple aarch64-none-linux-gnu -verify -emit-llvm-only %s | |||
| // RUN: %clang_cc1 -triple aarch64-pc-windows-msvcu -verify -emit-llvm-only %s | |||
| // RUN: %clang_cc1 -triple powerpc64-ibm-aix-xcoff -verify -emit-llvm-only %s | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since the test case only test whether the source code is compiled pass or not ? do it matter 32bit or 64bit ?
In PR llvm#153049, we have a use case of attaching the !associated metadata to an ifunc. Since an ifunc is similar to a function declaration, it seems natural to allow metadata on ifuncs. Currently, the metadata API allows adding Metadata to llvm::Values, so the in-memory IR allows for metadata on ifuncs, but the IR reader/writer is not aware of that. Teach the IR parser and writer to support metadata on ifuncs, and update documentation.
hubert-reinterpretcast
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When the target of the call is known to be an ifunc (e.g., when the definition is visible because of LTO), it would make sense to avoid going through the implementation stub and to just generate an indirect call sequence in-line from the call site. Such a call sequence is generated when -mlongcall is specified (but the IR realization of it seems to be tied to the caller and not the callee). LLVM has a "long-call" callee attribute for MIPS.
Teach the IR parser and writer to support metadata on ifuncs, and update documentation. In PR #153049, we have a use case of attaching the `!associated` metadata to an ifunc. Since an ifunc is similar to a function declaration, it seems natural to allow metadata on ifuncs. Currently, the metadata API allows adding Metadata to llvm::GlobalObject, so the in-memory IR allows for metadata on ifuncs, but the IR reader/writer is not aware of that. --------- Co-authored-by: Wael Yehia <[email protected]>
…ion -bdbg:namedsects:ss by default on AIX
…art/stop symbol references when user code does not use ifuncs
also pass -bdbg:namedsects:ss if AIX version is unknown
Load R11 in the ifunc stub
mandlebug
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Partway through my review but though I'd leave some of the initial comments.
| cl::desc("Enable Passing SSP Canary info in Trackback on AIX"), cl::Hidden); | ||
|
|
||
| static cl::list<std::string> IFuncLocal( | ||
| "ifunc-local", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't recall where we ended up in the discussion of this option. I'm not a fan of having a safelist option that takes a list of strings, however I think we came to the conclusion this might be needed because having a compiler only implementation can't sufficiently verify the resolver is always safe?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We didn't settle on whether to keep it or not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I spoke with Sean offline, and he suggested to make it a user error to have resolvers returning non-local functions, and to change the walker in function TOCRestoreNeededForCallToImplementation to flag any violation it can find.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mandlebug please see latest version of TOCRestoreNeededForCallToImplementation.
I changed the walker to return a tristate: IsLocal (all are provably local), IsPreemptible (at least one is provably preemptible), and Unknown (a mix of provablty local and unknown structure).
I added a false-by-default flag -mllvm -ifunc-local-if-proven to control the bias. By default, the bias is local unless proven preemptible. I thought some users might not be comfortable with this as it might lead to silent bad behavior, so they can opt-out by using that flag. The flag is boolean, not a list of functions.
Once we agree that this is an acceptable approach, I'll add a note under the ifunc attribute documentation.
mandlebug
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some minor comments so far.
Currently, the AIX linker and loader do not provide a mechanism to implement ifuncs similar to GNU_ifunc on ELF Linux.
On AIX, we will lower
__attribute__((ifunc("resolver"))to the llvmifuncas other platforms do. The llvmifuncin turn will get lowered at late stages of the optimization pipeline to an AIX-specific implementation. No special linkage or relocations are needed when generating assembly/object output.On AIX, a function
foohas two symbols associated with it: a function descriptor (foo) residing in the.datasection, and an entry point (.foo) residing in the.textsection. The first field of the descriptor is the address of the entry point. Typically, the address field in the descriptor is initialized once: statically, at load time (?), or at runtime if runtime linking is enabled.Here we would like to use the address field in the descriptor to implement the
ifuncsemantics. Specifically, the ifunc function will become a stub that jumps to the entry point in the address field. A constructor function is linked into every linkage module. The constructor walks an array of{descriptor, resolver}pairs, calling the resolver and saving the result in the address field in the descriptor.TODO: