[win][x64] Unwind v2 4/4: Use chaining to split unwind info if needed, and allow dealloc to be elided if setframe + frame pointer is used #159206
+615
−150
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@llvm/pr-subscribers-mlgo @llvm/pr-subscribers-llvm-binary-utilities Author: Daniel Paoliello (dpaoliello) ChangesWindows x64 unwind v2 only permits an offset from the end of the current unwind info frame to the epilog of 4Kb. Since LLVM only uses one frame info per function, the furthest an epilog could be from the end of the function is 4Kb. One way to permit offsets that are further is to use frame info chaining: each of the chained frame infos covers a different portion of the function and all point back to the parent unwind info. LLVM's existing support for chaining frame infos is broken. Windows requires that each frame info (including the parent) is adjacent, but not overlapping, and the whole function is covered (i.e., each instruction maps to exactly one frame info). Therefore, having start/end chaining pseudo instructions doesn't make any sense, as every "end chain" pseudo would need a start pseudo immediately after it or be at the end of function. Instead, I switched to having a "split chain" pseudo instruction that ends the current frame and starts a new chained frame. The x64 unwind v2 lowering will now keep track of the approximate number of instructions between each epilog and, if there are more than 1000 instructions (chosen based on real world testing, this can be overridden with an LLVM flag) between the current epilog and the end of the current frame info, then a split frame info will be inserted. Additionally, if the lowering detects that the current unwind info might have more than 255 bytes of unwind codes (limit per frame info, also overridable via LLVM flag) it will also split the frame info. Patch is 29.03 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/159206.diff 16 Files Affected:
diff --git a/llvm/include/llvm/MC/MCStreamer.h b/llvm/include/llvm/MC/MCStreamer.h
index 79c715e3820a6..2a0095c9b7613 100644
--- a/llvm/include/llvm/MC/MCStreamer.h
+++ b/llvm/include/llvm/MC/MCStreamer.h
@@ -1024,8 +1024,7 @@ class LLVM_ABI MCStreamer {
/// for the frame. We cannot use the End marker, as it is not set at the
/// point of emitting .xdata, in order to indicate that the frame is active.
virtual void emitWinCFIFuncletOrFuncEnd(SMLoc Loc = SMLoc());
- virtual void emitWinCFIStartChained(SMLoc Loc = SMLoc());
- virtual void emitWinCFIEndChained(SMLoc Loc = SMLoc());
+ virtual void emitWinCFISplitChained(SMLoc Loc = SMLoc());
virtual void emitWinCFIPushReg(MCRegister Register, SMLoc Loc = SMLoc());
virtual void emitWinCFISetFrame(MCRegister Register, unsigned Offset,
SMLoc Loc = SMLoc());
diff --git a/llvm/include/llvm/MC/MCWinEH.h b/llvm/include/llvm/MC/MCWinEH.h
index 1bbfb9f59bc4c..be5fdb32a48f9 100644
--- a/llvm/include/llvm/MC/MCWinEH.h
+++ b/llvm/include/llvm/MC/MCWinEH.h
@@ -59,7 +59,7 @@ struct FrameInfo {
uint8_t Version = DefaultVersion;
int LastFrameInst = -1;
- const FrameInfo *ChainedParent = nullptr;
+ FrameInfo *ChainedParent = nullptr;
std::vector<Instruction> Instructions;
struct Epilog {
std::vector<Instruction> Instructions;
@@ -90,9 +90,9 @@ struct FrameInfo {
FrameInfo(const MCSymbol *Function, const MCSymbol *BeginFuncEHLabel)
: Begin(BeginFuncEHLabel), Function(Function) {}
FrameInfo(const MCSymbol *Function, const MCSymbol *BeginFuncEHLabel,
- const FrameInfo *ChainedParent)
+ FrameInfo *ChainedParent)
: Begin(BeginFuncEHLabel), Function(Function),
- ChainedParent(ChainedParent) {}
+ ChainedParent(ChainedParent), Version(ChainedParent->Version) {}
bool empty() const {
if (!Instructions.empty())
diff --git a/llvm/lib/MC/MCAsmStreamer.cpp b/llvm/lib/MC/MCAsmStreamer.cpp
index be8c022f39ad1..bee7badbe0bc9 100644
--- a/llvm/lib/MC/MCAsmStreamer.cpp
+++ b/llvm/lib/MC/MCAsmStreamer.cpp
@@ -375,8 +375,7 @@ class MCAsmStreamer final : public MCStreamer {
void emitWinCFIStartProc(const MCSymbol *Symbol, SMLoc Loc) override;
void emitWinCFIEndProc(SMLoc Loc) override;
void emitWinCFIFuncletOrFuncEnd(SMLoc Loc) override;
- void emitWinCFIStartChained(SMLoc Loc) override;
- void emitWinCFIEndChained(SMLoc Loc) override;
+ void emitWinCFISplitChained(SMLoc Loc) override;
void emitWinCFIPushReg(MCRegister Register, SMLoc Loc) override;
void emitWinCFISetFrame(MCRegister Register, unsigned Offset,
SMLoc Loc) override;
@@ -2175,17 +2174,10 @@ void MCAsmStreamer::emitWinCFIFuncletOrFuncEnd(SMLoc Loc) {
EmitEOL();
}
-void MCAsmStreamer::emitWinCFIStartChained(SMLoc Loc) {
- MCStreamer::emitWinCFIStartChained(Loc);
+void MCAsmStreamer::emitWinCFISplitChained(SMLoc Loc) {
+ MCStreamer::emitWinCFISplitChained(Loc);
- OS << "\t.seh_startchained";
- EmitEOL();
-}
-
-void MCAsmStreamer::emitWinCFIEndChained(SMLoc Loc) {
- MCStreamer::emitWinCFIEndChained(Loc);
-
- OS << "\t.seh_endchained";
+ OS << "\t.seh_splitchained";
EmitEOL();
}
diff --git a/llvm/lib/MC/MCParser/COFFAsmParser.cpp b/llvm/lib/MC/MCParser/COFFAsmParser.cpp
index 5dd79946d8779..b795eca73cce5 100644
--- a/llvm/lib/MC/MCParser/COFFAsmParser.cpp
+++ b/llvm/lib/MC/MCParser/COFFAsmParser.cpp
@@ -80,10 +80,8 @@ class COFFAsmParser : public MCAsmParserExtension {
".seh_endproc");
addDirectiveHandler<&COFFAsmParser::parseSEHDirectiveEndFuncletOrFunc>(
".seh_endfunclet");
- addDirectiveHandler<&COFFAsmParser::parseSEHDirectiveStartChained>(
- ".seh_startchained");
- addDirectiveHandler<&COFFAsmParser::parseSEHDirectiveEndChained>(
- ".seh_endchained");
+ addDirectiveHandler<&COFFAsmParser::parseSEHDirectiveSplitChained>(
+ ".seh_splitchained");
addDirectiveHandler<&COFFAsmParser::parseSEHDirectiveHandler>(
".seh_handler");
addDirectiveHandler<&COFFAsmParser::parseSEHDirectiveHandlerData>(
@@ -143,8 +141,7 @@ class COFFAsmParser : public MCAsmParserExtension {
bool parseSEHDirectiveStartProc(StringRef, SMLoc);
bool parseSEHDirectiveEndProc(StringRef, SMLoc);
bool parseSEHDirectiveEndFuncletOrFunc(StringRef, SMLoc);
- bool parseSEHDirectiveStartChained(StringRef, SMLoc);
- bool parseSEHDirectiveEndChained(StringRef, SMLoc);
+ bool parseSEHDirectiveSplitChained(StringRef, SMLoc);
bool parseSEHDirectiveHandler(StringRef, SMLoc);
bool parseSEHDirectiveHandlerData(StringRef, SMLoc);
bool parseSEHDirectiveAllocStack(StringRef, SMLoc);
@@ -685,15 +682,9 @@ bool COFFAsmParser::parseSEHDirectiveEndFuncletOrFunc(StringRef, SMLoc Loc) {
return false;
}
-bool COFFAsmParser::parseSEHDirectiveStartChained(StringRef, SMLoc Loc) {
+bool COFFAsmParser::parseSEHDirectiveSplitChained(StringRef, SMLoc Loc) {
Lex();
- getStreamer().emitWinCFIStartChained(Loc);
- return false;
-}
-
-bool COFFAsmParser::parseSEHDirectiveEndChained(StringRef, SMLoc Loc) {
- Lex();
- getStreamer().emitWinCFIEndChained(Loc);
+ getStreamer().emitWinCFISplitChained(Loc);
return false;
}
diff --git a/llvm/lib/MC/MCStreamer.cpp b/llvm/lib/MC/MCStreamer.cpp
index bc7398120096e..0773a15b0efc7 100644
--- a/llvm/lib/MC/MCStreamer.cpp
+++ b/llvm/lib/MC/MCStreamer.cpp
@@ -749,13 +749,14 @@ void MCStreamer::emitWinCFIEndProc(SMLoc Loc) {
WinEH::FrameInfo *CurFrame = EnsureValidWinFrameInfo(Loc);
if (!CurFrame)
return;
- if (CurFrame->ChainedParent)
- getContext().reportError(Loc, "Not all chained regions terminated!");
MCSymbol *Label = emitCFILabel();
CurFrame->End = Label;
- if (!CurFrame->FuncletOrFuncEnd)
- CurFrame->FuncletOrFuncEnd = CurFrame->End;
+ const MCSymbol **FuncletOrFuncEndPtr =
+ CurFrame->ChainedParent ? &CurFrame->ChainedParent->FuncletOrFuncEnd
+ : &CurFrame->FuncletOrFuncEnd;
+ if (!*FuncletOrFuncEndPtr)
+ *FuncletOrFuncEndPtr = CurFrame->End;
for (size_t I = CurrentProcWinFrameInfoStartIndex, E = WinFrameInfos.size();
I != E; ++I)
@@ -767,38 +768,38 @@ void MCStreamer::emitWinCFIFuncletOrFuncEnd(SMLoc Loc) {
WinEH::FrameInfo *CurFrame = EnsureValidWinFrameInfo(Loc);
if (!CurFrame)
return;
- if (CurFrame->ChainedParent)
- getContext().reportError(Loc, "Not all chained regions terminated!");
MCSymbol *Label = emitCFILabel();
- CurFrame->FuncletOrFuncEnd = Label;
+ const MCSymbol **FuncletOrFuncEndPtr =
+ CurFrame->ChainedParent ? &CurFrame->ChainedParent->FuncletOrFuncEnd
+ : &CurFrame->FuncletOrFuncEnd;
+ *FuncletOrFuncEndPtr = Label;
}
-void MCStreamer::emitWinCFIStartChained(SMLoc Loc) {
+void MCStreamer::emitWinCFISplitChained(SMLoc Loc) {
WinEH::FrameInfo *CurFrame = EnsureValidWinFrameInfo(Loc);
if (!CurFrame)
return;
- MCSymbol *StartProc = emitCFILabel();
-
- WinFrameInfos.emplace_back(std::make_unique<WinEH::FrameInfo>(
- CurFrame->Function, StartProc, CurFrame));
- CurrentWinFrameInfo = WinFrameInfos.back().get();
- CurrentWinFrameInfo->TextSection = getCurrentSectionOnly();
-}
-
-void MCStreamer::emitWinCFIEndChained(SMLoc Loc) {
- WinEH::FrameInfo *CurFrame = EnsureValidWinFrameInfo(Loc);
- if (!CurFrame)
- return;
- if (!CurFrame->ChainedParent)
+ if (!CurFrame->PrologEnd && !CurFrame->ChainedParent)
return getContext().reportError(
- Loc, "End of a chained region outside a chained region!");
+ Loc, "can't split into a new chained region (.seh_splitchained) in the "
+ "middle of a prolog in " +
+ CurFrame->Function->getName());
MCSymbol *Label = emitCFILabel();
+ // Complete the current frame before starting a new, chained one.
CurFrame->End = Label;
- CurrentWinFrameInfo = const_cast<WinEH::FrameInfo *>(CurFrame->ChainedParent);
+
+ // All chained frames point to the same parent.
+ WinEH::FrameInfo *ChainedParent =
+ CurFrame->ChainedParent ? CurFrame->ChainedParent : CurFrame;
+
+ WinFrameInfos.emplace_back(std::make_unique<WinEH::FrameInfo>(
+ CurFrame->Function, Label, ChainedParent));
+ CurrentWinFrameInfo = WinFrameInfos.back().get();
+ CurrentWinFrameInfo->TextSection = getCurrentSectionOnly();
}
void MCStreamer::emitWinEHHandler(const MCSymbol *Sym, bool Unwind, bool Except,
@@ -994,7 +995,8 @@ void MCStreamer::emitWinCFIBeginEpilogue(SMLoc Loc) {
if (!CurFrame)
return;
- if (!CurFrame->PrologEnd)
+ // Chained unwinds aren't guaranteed to have a prolog.
+ if (!CurFrame->PrologEnd && !CurFrame->ChainedParent)
return getContext().reportError(
Loc, "starting epilogue (.seh_startepilogue) before prologue has ended "
"(.seh_endprologue) in " +
diff --git a/llvm/lib/MC/MCWin64EH.cpp b/llvm/lib/MC/MCWin64EH.cpp
index 8111ccb8bc69c..b5c4d3293e09f 100644
--- a/llvm/lib/MC/MCWin64EH.cpp
+++ b/llvm/lib/MC/MCWin64EH.cpp
@@ -23,7 +23,7 @@ class MCSection;
/// MCExpr that represents the epilog unwind code in an unwind table.
class MCUnwindV2EpilogTargetExpr final : public MCTargetExpr {
const MCSymbol *Function;
- const MCSymbol *FunctionEnd;
+ const MCSymbol *EndLabel;
const MCSymbol *UnwindV2Start;
const MCSymbol *EpilogEnd;
uint8_t EpilogSize;
@@ -32,7 +32,7 @@ class MCUnwindV2EpilogTargetExpr final : public MCTargetExpr {
MCUnwindV2EpilogTargetExpr(const WinEH::FrameInfo &FrameInfo,
const WinEH::FrameInfo::Epilog &Epilog,
uint8_t EpilogSize_)
- : Function(FrameInfo.Function), FunctionEnd(FrameInfo.FuncletOrFuncEnd),
+ : Function(FrameInfo.Function), EndLabel(FrameInfo.End),
UnwindV2Start(Epilog.UnwindV2Start), EpilogEnd(Epilog.End),
EpilogSize(EpilogSize_), Loc(Epilog.Loc) {}
@@ -271,7 +271,7 @@ static void EmitUnwindInfo(MCStreamer &streamer, WinEH::FrameInfo *info) {
// encoding for it. Because of our +1 trick for the size, this will only
// work where that final terminator instruction is 1 byte long.
auto LastEpilogToFuncEnd = GetOptionalAbsDifference(
- OS->getAssembler(), info->FuncletOrFuncEnd, LastEpilog.UnwindV2Start);
+ OS->getAssembler(), info->End, LastEpilog.UnwindV2Start);
LastEpilogIsAtEnd = (LastEpilogToFuncEnd == EpilogSize);
// If we have an odd number of epilog codes, we need to add a padding code.
@@ -384,7 +384,7 @@ bool MCUnwindV2EpilogTargetExpr::evaluateAsRelocatableImpl(
MCValue &Res, const MCAssembler *Asm) const {
// Calculate the offset to this epilog, and validate it's within the allowed
// range.
- auto Offset = GetOptionalAbsDifference(*Asm, FunctionEnd, UnwindV2Start);
+ auto Offset = GetOptionalAbsDifference(*Asm, EndLabel, UnwindV2Start);
if (!Offset) {
Asm->getContext().reportError(
Loc, "Failed to evaluate epilog offset for Unwind v2 in " +
@@ -400,7 +400,7 @@ bool MCUnwindV2EpilogTargetExpr::evaluateAsRelocatableImpl(
return false;
}
- // Sanity check that all epilogs are the same size.
+ // Validate that all epilogs are the same size.
auto Size = GetOptionalAbsDifference(*Asm, EpilogEnd, UnwindV2Start);
if (Size != (EpilogSize - 1)) {
Asm->getContext().reportError(
diff --git a/llvm/lib/Target/X86/X86InstrCompiler.td b/llvm/lib/Target/X86/X86InstrCompiler.td
index af7a33abaf758..941fe8e361561 100644
--- a/llvm/lib/Target/X86/X86InstrCompiler.td
+++ b/llvm/lib/Target/X86/X86InstrCompiler.td
@@ -272,6 +272,12 @@ let isPseudo = 1, isMeta = 1, SchedRW = [WriteSystem] in {
"#SEH_UnwindV2Start", []>;
}
+// Chain instructions:
+let isPseudo = 1, isMeta = 1, SchedRW = [WriteSystem] in {
+ def SEH_SplitChained : I<0, Pseudo, (outs), (ins),
+ "#SEH_SplitChained", []>;
+}
+
//===----------------------------------------------------------------------===//
// Pseudo instructions used by KCFI.
//===----------------------------------------------------------------------===//
diff --git a/llvm/lib/Target/X86/X86MCInstLower.cpp b/llvm/lib/Target/X86/X86MCInstLower.cpp
index 481a9be8374ab..46bd8e0332650 100644
--- a/llvm/lib/Target/X86/X86MCInstLower.cpp
+++ b/llvm/lib/Target/X86/X86MCInstLower.cpp
@@ -1795,6 +1795,10 @@ void X86AsmPrinter::EmitSEHInstruction(const MachineInstr *MI) {
OutStreamer->emitWinCFIUnwindVersion(MI->getOperand(0).getImm());
break;
+ case X86::SEH_SplitChained:
+ OutStreamer->emitWinCFISplitChained();
+ break;
+
default:
llvm_unreachable("expected SEH_ instruction");
}
@@ -2526,6 +2530,7 @@ void X86AsmPrinter::emitInstruction(const MachineInstr *MI) {
case X86::SEH_EndEpilogue:
case X86::SEH_UnwindV2Start:
case X86::SEH_UnwindVersion:
+ case X86::SEH_SplitChained:
EmitSEHInstruction(MI);
return;
diff --git a/llvm/lib/Target/X86/X86WinEHUnwindV2.cpp b/llvm/lib/Target/X86/X86WinEHUnwindV2.cpp
index 9bf0abb018c99..b7c544bfb3114 100644
--- a/llvm/lib/Target/X86/X86WinEHUnwindV2.cpp
+++ b/llvm/lib/Target/X86/X86WinEHUnwindV2.cpp
@@ -32,15 +32,22 @@ STATISTIC(MeetsUnwindV2Criteria,
STATISTIC(FailsUnwindV2Criteria,
"Number of functions that fail Unwind v2 criteria");
-static cl::opt<unsigned> MaximumUnwindCodes(
- "x86-wineh-unwindv2-max-unwind-codes", cl::Hidden,
- cl::desc("Maximum number of unwind codes permitted in each unwind info."),
- cl::init(UINT8_MAX));
+static cl::opt<unsigned>
+ UnwindCodeThreshold("x86-wineh-unwindv2-unwind-codes-threshold", cl::Hidden,
+ cl::desc("Maximum number of unwind codes before "
+ "splitting into a new unwind info."),
+ cl::init(UINT8_MAX));
static cl::opt<unsigned>
ForceMode("x86-wineh-unwindv2-force-mode", cl::Hidden,
cl::desc("Overwrites the Unwind v2 mode for testing purposes."));
+static cl::opt<unsigned> InstructionCountThreshold(
+ "x86-wineh-unwindv2-instruction-count-threshold", cl::Hidden,
+ cl::desc("Maximum number of (approximate) instructions before splitting "
+ "into a new unwind info."),
+ cl::init(1000));
+
namespace {
class X86WinEHUnwindV2 : public MachineFunctionPass {
@@ -69,6 +76,11 @@ enum class FunctionState {
FinishedEpilog,
};
+struct EpilogInfo {
+ MachineInstr *UnwindV2StartLocation;
+ unsigned ApproximateInstructionPosition;
+};
+
} // end anonymous namespace
char X86WinEHUnwindV2::ID = 0;
@@ -109,7 +121,9 @@ bool X86WinEHUnwindV2::runOnMachineFunction(MachineFunction &MF) {
unsigned ApproximatePrologCodeCount = 0;
// Requested changes.
- SmallVector<MachineInstr *> UnwindV2StartLocations;
+ SmallVector<EpilogInfo> EpilogInfos;
+
+ unsigned ApproximateInstructionCount = 0;
for (MachineBasicBlock &MBB : MF) {
// Current epilog information. We assume that epilogs cannot cross basic
@@ -117,8 +131,12 @@ bool X86WinEHUnwindV2::runOnMachineFunction(MachineFunction &MF) {
unsigned PoppedRegCount = 0;
bool HasStackDealloc = false;
MachineInstr *UnwindV2StartLocation = nullptr;
+ MachineInstr *BeginEpilog = nullptr;
for (MachineInstr &MI : MBB) {
+ if (!MI.isPseudo() && !MI.isMetaInstruction())
+ ApproximateInstructionCount++;
+
switch (MI.getOpcode()) {
//
// Prolog handling.
@@ -172,6 +190,7 @@ bool X86WinEHUnwindV2::runOnMachineFunction(MachineFunction &MF) {
if (State != FunctionState::HasProlog)
llvm_unreachable("SEH_BeginEpilogue in prolog or another epilog");
State = FunctionState::InEpilog;
+ BeginEpilog = &MI;
break;
case X86::SEH_EndEpilogue:
@@ -192,7 +211,8 @@ bool X86WinEHUnwindV2::runOnMachineFunction(MachineFunction &MF) {
// epilog.
if (!UnwindV2StartLocation)
UnwindV2StartLocation = &MI;
- UnwindV2StartLocations.push_back(UnwindV2StartLocation);
+ EpilogInfos.push_back(
+ {UnwindV2StartLocation, ApproximateInstructionCount});
State = FunctionState::FinishedEpilog;
break;
@@ -305,37 +325,39 @@ bool X86WinEHUnwindV2::runOnMachineFunction(MachineFunction &MF) {
}
}
- if (UnwindV2StartLocations.empty())
- return false;
-
- MachineBasicBlock &FirstMBB = MF.front();
- // Assume +1 for the "header" UOP_Epilog that contains the epilog size, and
- // that we won't be able to use the "last epilog at the end of function"
- // optimization.
- if (ApproximatePrologCodeCount + UnwindV2StartLocations.size() + 1 >
- static_cast<unsigned>(MaximumUnwindCodes)) {
- if (Mode == WinX64EHUnwindV2Mode::Required)
- MF.getFunction().getContext().diagnose(DiagnosticInfoGenericWithLoc(
- "Windows x64 Unwind v2 is required, but the function '" +
- MF.getName() +
- "' has too many unwind codes. Try splitting the function or "
- "reducing the number of places where it exits early with a tail "
- "call.",
- MF.getFunction(), findDebugLoc(FirstMBB)));
-
- FailsUnwindV2Criteria++;
+ if (EpilogInfos.empty())
return false;
- }
MeetsUnwindV2Criteria++;
- // Emit the pseudo instruction that marks the start of each epilog.
+ // Walk the list of epilogs backwards and add new SEH pseudo instructions:
+ // * SEH_UnwindV2Start at the start of each epilog.
+ // * If the current instruction is too far away from where the last unwind
+ // info ended OR there are too many unwind codes in the info, then add
+ // SEH_SplitChained to finish the current info.
+ unsigned LastUnwindInfoEndPosition = ApproximateInstructionCount;
+ unsigned UnwindCodeCount = ApproximatePrologCodeCount + 1;
const TargetInstrInfo *TII = MF.getSubtarget().getInstrInfo();
- for (MachineInstr *MI : UnwindV2StartLocations) {
- BuildMI(*MI->getParent(), MI, MI->getDebugLoc(),
+ for (auto &Info : llvm::reverse(EpilogInfos)) {
+ MachineBasicBlock &MBB = *Info.UnwindV2StartLocation->getParent();
+ BuildMI(MBB, Info.UnwindV2StartLocation,
+ Info.UnwindV2StartLocation->getDebugLoc(),
TII->get(X86::SEH_UnwindV2Start));
+
+ if ((LastUnwindInfoEndPosition - Info.ApproximateInstructionPosition >=
+ InstructionCountThreshold) ||
+ (UnwindCodeCount >= UnwindCodeThreshold)) {
+ BuildMI(&MBB, Info.UnwindV2StartLocation->getDebugLoc(),
+ TII->get(X86::SEH_SplitChained));
+ LastUnwindInfoEndPosition = Info.ApproximateInstructionPosition;
+ // Doesn't reset to 0, as the prolog unwind codes are now in this info.
+ UnwindCodeCount = ApproximatePrologCodeCount + 1;
+ }
+
+ UnwindCodeCount++;
}
// Note that the function is using Unwind v2.
+ MachineBasicBlock &FirstMBB = MF.front();
BuildMI(FirstMBB, FirstMBB.front(), findDebugLoc(FirstMBB),
TII->get(X86::SEH_UnwindVersion))
.addImm(2);
diff --git a/llvm/test/CodeGen/X86/win64-eh-unwindv2-too-many-epilogs.mir b/llvm/test/CodeGen/X86/win64-eh-unwindv2-too-many-epilogs.mir
index 70c87ad87f792..ac54ec9f0652d 100644
--- a/llvm/test/CodeGen/X86/win64-eh-unwindv2-too-many-epilogs.mir
+++ b/llvm/test/CodeGen/X86/win64-eh-unwindv2-too-many-epilogs.mir
@@ -1,32 +1,22 @@
-# Require V2 and restrict the number of unwind codes to 8
-# RUN: not llc -mtriple=x86_64-pc-windows-msvc -o - %s \
-# RUN: -run-pass=x86-wineh-unwindv2 -x86-wineh-unwindv2-max-unwind-codes=8 \
-# RUN: 2>&1 | FileCheck %s -check-prefix=REQUIREV2
-
-# Force best-effort and restrict the number of unwind codes to 8
+# Restrict the number of unwind codes to 8
# RUN: llc -mtriple=x86_64-pc-windows-msvc -o - %s \
-# RUN: -run-pass=x86-wineh-unwindv2 -x86-wineh-unwindv2-max-unwind-codes=8 \
-# RUN: -x86-wineh-unwindv2-force-mode=1 | \
-# RUN: FileCheck %s -check-prefix=BESTEFFORT
+# RUN: -x86-wineh-unwindv2-unwind-codes-threshold=8 \
+# RUN: -run-pass=x86-wineh-unwindv2 | FileCheck %s \
+# RUN: -check-prefixes=ALLOWLESS,CHECK
-# Require V2, but allow the default number of unwind codes (255)
+# Allow the default number of unwind codes (255)
# RUN: llc -mtriple=x86_64-pc-windows-msvc -o - %s \
-# RUN: -run-pass=x86-wineh-unwindv2 | FileCheck %s -check-prefix=ALLOWMORE
-
-# Usually 255 unwind codes are permitted, but we passed an arg to llc to limit
-# it to 8.
-# REQUIREV2...
[truncated]
|
|
@llvm/pr-subscribers-llvm-mc Author: Daniel Paoliello (dpaoliello) ChangesWindows x64 unwind v2 only permits an offset from the end of the current unwind info frame to the epilog of 4Kb. Since LLVM only uses one frame info per function, the furthest an epilog could be from the end of the function is 4Kb. One way to permit offsets that are further is to use frame info chaining: each of the chained frame infos covers a different portion of the function and all point back to the parent unwind info. LLVM's existing support for chaining frame infos is broken. Windows requires that each frame info (including the parent) is adjacent, but not overlapping, and the whole function is covered (i.e., each instruction maps to exactly one frame info). Therefore, having start/end chaining pseudo instructions doesn't make any sense, as every "end chain" pseudo would need a start pseudo immediately after it or be at the end of function. Instead, I switched to having a "split chain" pseudo instruction that ends the current frame and starts a new chained frame. The x64 unwind v2 lowering will now keep track of the approximate number of instructions between each epilog and, if there are more than 1000 instructions (chosen based on real world testing, this can be overridden with an LLVM flag) between the current epilog and the end of the current frame info, then a split frame info will be inserted. Additionally, if the lowering detects that the current unwind info might have more than 255 bytes of unwind codes (limit per frame info, also overridable via LLVM flag) it will also split the frame info. Patch is 29.03 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/159206.diff 16 Files Affected:
diff --git a/llvm/include/llvm/MC/MCStreamer.h b/llvm/include/llvm/MC/MCStreamer.h
index 79c715e3820a6..2a0095c9b7613 100644
--- a/llvm/include/llvm/MC/MCStreamer.h
+++ b/llvm/include/llvm/MC/MCStreamer.h
@@ -1024,8 +1024,7 @@ class LLVM_ABI MCStreamer {
/// for the frame. We cannot use the End marker, as it is not set at the
/// point of emitting .xdata, in order to indicate that the frame is active.
virtual void emitWinCFIFuncletOrFuncEnd(SMLoc Loc = SMLoc());
- virtual void emitWinCFIStartChained(SMLoc Loc = SMLoc());
- virtual void emitWinCFIEndChained(SMLoc Loc = SMLoc());
+ virtual void emitWinCFISplitChained(SMLoc Loc = SMLoc());
virtual void emitWinCFIPushReg(MCRegister Register, SMLoc Loc = SMLoc());
virtual void emitWinCFISetFrame(MCRegister Register, unsigned Offset,
SMLoc Loc = SMLoc());
diff --git a/llvm/include/llvm/MC/MCWinEH.h b/llvm/include/llvm/MC/MCWinEH.h
index 1bbfb9f59bc4c..be5fdb32a48f9 100644
--- a/llvm/include/llvm/MC/MCWinEH.h
+++ b/llvm/include/llvm/MC/MCWinEH.h
@@ -59,7 +59,7 @@ struct FrameInfo {
uint8_t Version = DefaultVersion;
int LastFrameInst = -1;
- const FrameInfo *ChainedParent = nullptr;
+ FrameInfo *ChainedParent = nullptr;
std::vector<Instruction> Instructions;
struct Epilog {
std::vector<Instruction> Instructions;
@@ -90,9 +90,9 @@ struct FrameInfo {
FrameInfo(const MCSymbol *Function, const MCSymbol *BeginFuncEHLabel)
: Begin(BeginFuncEHLabel), Function(Function) {}
FrameInfo(const MCSymbol *Function, const MCSymbol *BeginFuncEHLabel,
- const FrameInfo *ChainedParent)
+ FrameInfo *ChainedParent)
: Begin(BeginFuncEHLabel), Function(Function),
- ChainedParent(ChainedParent) {}
+ ChainedParent(ChainedParent), Version(ChainedParent->Version) {}
bool empty() const {
if (!Instructions.empty())
diff --git a/llvm/lib/MC/MCAsmStreamer.cpp b/llvm/lib/MC/MCAsmStreamer.cpp
index be8c022f39ad1..bee7badbe0bc9 100644
--- a/llvm/lib/MC/MCAsmStreamer.cpp
+++ b/llvm/lib/MC/MCAsmStreamer.cpp
@@ -375,8 +375,7 @@ class MCAsmStreamer final : public MCStreamer {
void emitWinCFIStartProc(const MCSymbol *Symbol, SMLoc Loc) override;
void emitWinCFIEndProc(SMLoc Loc) override;
void emitWinCFIFuncletOrFuncEnd(SMLoc Loc) override;
- void emitWinCFIStartChained(SMLoc Loc) override;
- void emitWinCFIEndChained(SMLoc Loc) override;
+ void emitWinCFISplitChained(SMLoc Loc) override;
void emitWinCFIPushReg(MCRegister Register, SMLoc Loc) override;
void emitWinCFISetFrame(MCRegister Register, unsigned Offset,
SMLoc Loc) override;
@@ -2175,17 +2174,10 @@ void MCAsmStreamer::emitWinCFIFuncletOrFuncEnd(SMLoc Loc) {
EmitEOL();
}
-void MCAsmStreamer::emitWinCFIStartChained(SMLoc Loc) {
- MCStreamer::emitWinCFIStartChained(Loc);
+void MCAsmStreamer::emitWinCFISplitChained(SMLoc Loc) {
+ MCStreamer::emitWinCFISplitChained(Loc);
- OS << "\t.seh_startchained";
- EmitEOL();
-}
-
-void MCAsmStreamer::emitWinCFIEndChained(SMLoc Loc) {
- MCStreamer::emitWinCFIEndChained(Loc);
-
- OS << "\t.seh_endchained";
+ OS << "\t.seh_splitchained";
EmitEOL();
}
diff --git a/llvm/lib/MC/MCParser/COFFAsmParser.cpp b/llvm/lib/MC/MCParser/COFFAsmParser.cpp
index 5dd79946d8779..b795eca73cce5 100644
--- a/llvm/lib/MC/MCParser/COFFAsmParser.cpp
+++ b/llvm/lib/MC/MCParser/COFFAsmParser.cpp
@@ -80,10 +80,8 @@ class COFFAsmParser : public MCAsmParserExtension {
".seh_endproc");
addDirectiveHandler<&COFFAsmParser::parseSEHDirectiveEndFuncletOrFunc>(
".seh_endfunclet");
- addDirectiveHandler<&COFFAsmParser::parseSEHDirectiveStartChained>(
- ".seh_startchained");
- addDirectiveHandler<&COFFAsmParser::parseSEHDirectiveEndChained>(
- ".seh_endchained");
+ addDirectiveHandler<&COFFAsmParser::parseSEHDirectiveSplitChained>(
+ ".seh_splitchained");
addDirectiveHandler<&COFFAsmParser::parseSEHDirectiveHandler>(
".seh_handler");
addDirectiveHandler<&COFFAsmParser::parseSEHDirectiveHandlerData>(
@@ -143,8 +141,7 @@ class COFFAsmParser : public MCAsmParserExtension {
bool parseSEHDirectiveStartProc(StringRef, SMLoc);
bool parseSEHDirectiveEndProc(StringRef, SMLoc);
bool parseSEHDirectiveEndFuncletOrFunc(StringRef, SMLoc);
- bool parseSEHDirectiveStartChained(StringRef, SMLoc);
- bool parseSEHDirectiveEndChained(StringRef, SMLoc);
+ bool parseSEHDirectiveSplitChained(StringRef, SMLoc);
bool parseSEHDirectiveHandler(StringRef, SMLoc);
bool parseSEHDirectiveHandlerData(StringRef, SMLoc);
bool parseSEHDirectiveAllocStack(StringRef, SMLoc);
@@ -685,15 +682,9 @@ bool COFFAsmParser::parseSEHDirectiveEndFuncletOrFunc(StringRef, SMLoc Loc) {
return false;
}
-bool COFFAsmParser::parseSEHDirectiveStartChained(StringRef, SMLoc Loc) {
+bool COFFAsmParser::parseSEHDirectiveSplitChained(StringRef, SMLoc Loc) {
Lex();
- getStreamer().emitWinCFIStartChained(Loc);
- return false;
-}
-
-bool COFFAsmParser::parseSEHDirectiveEndChained(StringRef, SMLoc Loc) {
- Lex();
- getStreamer().emitWinCFIEndChained(Loc);
+ getStreamer().emitWinCFISplitChained(Loc);
return false;
}
diff --git a/llvm/lib/MC/MCStreamer.cpp b/llvm/lib/MC/MCStreamer.cpp
index bc7398120096e..0773a15b0efc7 100644
--- a/llvm/lib/MC/MCStreamer.cpp
+++ b/llvm/lib/MC/MCStreamer.cpp
@@ -749,13 +749,14 @@ void MCStreamer::emitWinCFIEndProc(SMLoc Loc) {
WinEH::FrameInfo *CurFrame = EnsureValidWinFrameInfo(Loc);
if (!CurFrame)
return;
- if (CurFrame->ChainedParent)
- getContext().reportError(Loc, "Not all chained regions terminated!");
MCSymbol *Label = emitCFILabel();
CurFrame->End = Label;
- if (!CurFrame->FuncletOrFuncEnd)
- CurFrame->FuncletOrFuncEnd = CurFrame->End;
+ const MCSymbol **FuncletOrFuncEndPtr =
+ CurFrame->ChainedParent ? &CurFrame->ChainedParent->FuncletOrFuncEnd
+ : &CurFrame->FuncletOrFuncEnd;
+ if (!*FuncletOrFuncEndPtr)
+ *FuncletOrFuncEndPtr = CurFrame->End;
for (size_t I = CurrentProcWinFrameInfoStartIndex, E = WinFrameInfos.size();
I != E; ++I)
@@ -767,38 +768,38 @@ void MCStreamer::emitWinCFIFuncletOrFuncEnd(SMLoc Loc) {
WinEH::FrameInfo *CurFrame = EnsureValidWinFrameInfo(Loc);
if (!CurFrame)
return;
- if (CurFrame->ChainedParent)
- getContext().reportError(Loc, "Not all chained regions terminated!");
MCSymbol *Label = emitCFILabel();
- CurFrame->FuncletOrFuncEnd = Label;
+ const MCSymbol **FuncletOrFuncEndPtr =
+ CurFrame->ChainedParent ? &CurFrame->ChainedParent->FuncletOrFuncEnd
+ : &CurFrame->FuncletOrFuncEnd;
+ *FuncletOrFuncEndPtr = Label;
}
-void MCStreamer::emitWinCFIStartChained(SMLoc Loc) {
+void MCStreamer::emitWinCFISplitChained(SMLoc Loc) {
WinEH::FrameInfo *CurFrame = EnsureValidWinFrameInfo(Loc);
if (!CurFrame)
return;
- MCSymbol *StartProc = emitCFILabel();
-
- WinFrameInfos.emplace_back(std::make_unique<WinEH::FrameInfo>(
- CurFrame->Function, StartProc, CurFrame));
- CurrentWinFrameInfo = WinFrameInfos.back().get();
- CurrentWinFrameInfo->TextSection = getCurrentSectionOnly();
-}
-
-void MCStreamer::emitWinCFIEndChained(SMLoc Loc) {
- WinEH::FrameInfo *CurFrame = EnsureValidWinFrameInfo(Loc);
- if (!CurFrame)
- return;
- if (!CurFrame->ChainedParent)
+ if (!CurFrame->PrologEnd && !CurFrame->ChainedParent)
return getContext().reportError(
- Loc, "End of a chained region outside a chained region!");
+ Loc, "can't split into a new chained region (.seh_splitchained) in the "
+ "middle of a prolog in " +
+ CurFrame->Function->getName());
MCSymbol *Label = emitCFILabel();
+ // Complete the current frame before starting a new, chained one.
CurFrame->End = Label;
- CurrentWinFrameInfo = const_cast<WinEH::FrameInfo *>(CurFrame->ChainedParent);
+
+ // All chained frames point to the same parent.
+ WinEH::FrameInfo *ChainedParent =
+ CurFrame->ChainedParent ? CurFrame->ChainedParent : CurFrame;
+
+ WinFrameInfos.emplace_back(std::make_unique<WinEH::FrameInfo>(
+ CurFrame->Function, Label, ChainedParent));
+ CurrentWinFrameInfo = WinFrameInfos.back().get();
+ CurrentWinFrameInfo->TextSection = getCurrentSectionOnly();
}
void MCStreamer::emitWinEHHandler(const MCSymbol *Sym, bool Unwind, bool Except,
@@ -994,7 +995,8 @@ void MCStreamer::emitWinCFIBeginEpilogue(SMLoc Loc) {
if (!CurFrame)
return;
- if (!CurFrame->PrologEnd)
+ // Chained unwinds aren't guaranteed to have a prolog.
+ if (!CurFrame->PrologEnd && !CurFrame->ChainedParent)
return getContext().reportError(
Loc, "starting epilogue (.seh_startepilogue) before prologue has ended "
"(.seh_endprologue) in " +
diff --git a/llvm/lib/MC/MCWin64EH.cpp b/llvm/lib/MC/MCWin64EH.cpp
index 8111ccb8bc69c..b5c4d3293e09f 100644
--- a/llvm/lib/MC/MCWin64EH.cpp
+++ b/llvm/lib/MC/MCWin64EH.cpp
@@ -23,7 +23,7 @@ class MCSection;
/// MCExpr that represents the epilog unwind code in an unwind table.
class MCUnwindV2EpilogTargetExpr final : public MCTargetExpr {
const MCSymbol *Function;
- const MCSymbol *FunctionEnd;
+ const MCSymbol *EndLabel;
const MCSymbol *UnwindV2Start;
const MCSymbol *EpilogEnd;
uint8_t EpilogSize;
@@ -32,7 +32,7 @@ class MCUnwindV2EpilogTargetExpr final : public MCTargetExpr {
MCUnwindV2EpilogTargetExpr(const WinEH::FrameInfo &FrameInfo,
const WinEH::FrameInfo::Epilog &Epilog,
uint8_t EpilogSize_)
- : Function(FrameInfo.Function), FunctionEnd(FrameInfo.FuncletOrFuncEnd),
+ : Function(FrameInfo.Function), EndLabel(FrameInfo.End),
UnwindV2Start(Epilog.UnwindV2Start), EpilogEnd(Epilog.End),
EpilogSize(EpilogSize_), Loc(Epilog.Loc) {}
@@ -271,7 +271,7 @@ static void EmitUnwindInfo(MCStreamer &streamer, WinEH::FrameInfo *info) {
// encoding for it. Because of our +1 trick for the size, this will only
// work where that final terminator instruction is 1 byte long.
auto LastEpilogToFuncEnd = GetOptionalAbsDifference(
- OS->getAssembler(), info->FuncletOrFuncEnd, LastEpilog.UnwindV2Start);
+ OS->getAssembler(), info->End, LastEpilog.UnwindV2Start);
LastEpilogIsAtEnd = (LastEpilogToFuncEnd == EpilogSize);
// If we have an odd number of epilog codes, we need to add a padding code.
@@ -384,7 +384,7 @@ bool MCUnwindV2EpilogTargetExpr::evaluateAsRelocatableImpl(
MCValue &Res, const MCAssembler *Asm) const {
// Calculate the offset to this epilog, and validate it's within the allowed
// range.
- auto Offset = GetOptionalAbsDifference(*Asm, FunctionEnd, UnwindV2Start);
+ auto Offset = GetOptionalAbsDifference(*Asm, EndLabel, UnwindV2Start);
if (!Offset) {
Asm->getContext().reportError(
Loc, "Failed to evaluate epilog offset for Unwind v2 in " +
@@ -400,7 +400,7 @@ bool MCUnwindV2EpilogTargetExpr::evaluateAsRelocatableImpl(
return false;
}
- // Sanity check that all epilogs are the same size.
+ // Validate that all epilogs are the same size.
auto Size = GetOptionalAbsDifference(*Asm, EpilogEnd, UnwindV2Start);
if (Size != (EpilogSize - 1)) {
Asm->getContext().reportError(
diff --git a/llvm/lib/Target/X86/X86InstrCompiler.td b/llvm/lib/Target/X86/X86InstrCompiler.td
index af7a33abaf758..941fe8e361561 100644
--- a/llvm/lib/Target/X86/X86InstrCompiler.td
+++ b/llvm/lib/Target/X86/X86InstrCompiler.td
@@ -272,6 +272,12 @@ let isPseudo = 1, isMeta = 1, SchedRW = [WriteSystem] in {
"#SEH_UnwindV2Start", []>;
}
+// Chain instructions:
+let isPseudo = 1, isMeta = 1, SchedRW = [WriteSystem] in {
+ def SEH_SplitChained : I<0, Pseudo, (outs), (ins),
+ "#SEH_SplitChained", []>;
+}
+
//===----------------------------------------------------------------------===//
// Pseudo instructions used by KCFI.
//===----------------------------------------------------------------------===//
diff --git a/llvm/lib/Target/X86/X86MCInstLower.cpp b/llvm/lib/Target/X86/X86MCInstLower.cpp
index 481a9be8374ab..46bd8e0332650 100644
--- a/llvm/lib/Target/X86/X86MCInstLower.cpp
+++ b/llvm/lib/Target/X86/X86MCInstLower.cpp
@@ -1795,6 +1795,10 @@ void X86AsmPrinter::EmitSEHInstruction(const MachineInstr *MI) {
OutStreamer->emitWinCFIUnwindVersion(MI->getOperand(0).getImm());
break;
+ case X86::SEH_SplitChained:
+ OutStreamer->emitWinCFISplitChained();
+ break;
+
default:
llvm_unreachable("expected SEH_ instruction");
}
@@ -2526,6 +2530,7 @@ void X86AsmPrinter::emitInstruction(const MachineInstr *MI) {
case X86::SEH_EndEpilogue:
case X86::SEH_UnwindV2Start:
case X86::SEH_UnwindVersion:
+ case X86::SEH_SplitChained:
EmitSEHInstruction(MI);
return;
diff --git a/llvm/lib/Target/X86/X86WinEHUnwindV2.cpp b/llvm/lib/Target/X86/X86WinEHUnwindV2.cpp
index 9bf0abb018c99..b7c544bfb3114 100644
--- a/llvm/lib/Target/X86/X86WinEHUnwindV2.cpp
+++ b/llvm/lib/Target/X86/X86WinEHUnwindV2.cpp
@@ -32,15 +32,22 @@ STATISTIC(MeetsUnwindV2Criteria,
STATISTIC(FailsUnwindV2Criteria,
"Number of functions that fail Unwind v2 criteria");
-static cl::opt<unsigned> MaximumUnwindCodes(
- "x86-wineh-unwindv2-max-unwind-codes", cl::Hidden,
- cl::desc("Maximum number of unwind codes permitted in each unwind info."),
- cl::init(UINT8_MAX));
+static cl::opt<unsigned>
+ UnwindCodeThreshold("x86-wineh-unwindv2-unwind-codes-threshold", cl::Hidden,
+ cl::desc("Maximum number of unwind codes before "
+ "splitting into a new unwind info."),
+ cl::init(UINT8_MAX));
static cl::opt<unsigned>
ForceMode("x86-wineh-unwindv2-force-mode", cl::Hidden,
cl::desc("Overwrites the Unwind v2 mode for testing purposes."));
+static cl::opt<unsigned> InstructionCountThreshold(
+ "x86-wineh-unwindv2-instruction-count-threshold", cl::Hidden,
+ cl::desc("Maximum number of (approximate) instructions before splitting "
+ "into a new unwind info."),
+ cl::init(1000));
+
namespace {
class X86WinEHUnwindV2 : public MachineFunctionPass {
@@ -69,6 +76,11 @@ enum class FunctionState {
FinishedEpilog,
};
+struct EpilogInfo {
+ MachineInstr *UnwindV2StartLocation;
+ unsigned ApproximateInstructionPosition;
+};
+
} // end anonymous namespace
char X86WinEHUnwindV2::ID = 0;
@@ -109,7 +121,9 @@ bool X86WinEHUnwindV2::runOnMachineFunction(MachineFunction &MF) {
unsigned ApproximatePrologCodeCount = 0;
// Requested changes.
- SmallVector<MachineInstr *> UnwindV2StartLocations;
+ SmallVector<EpilogInfo> EpilogInfos;
+
+ unsigned ApproximateInstructionCount = 0;
for (MachineBasicBlock &MBB : MF) {
// Current epilog information. We assume that epilogs cannot cross basic
@@ -117,8 +131,12 @@ bool X86WinEHUnwindV2::runOnMachineFunction(MachineFunction &MF) {
unsigned PoppedRegCount = 0;
bool HasStackDealloc = false;
MachineInstr *UnwindV2StartLocation = nullptr;
+ MachineInstr *BeginEpilog = nullptr;
for (MachineInstr &MI : MBB) {
+ if (!MI.isPseudo() && !MI.isMetaInstruction())
+ ApproximateInstructionCount++;
+
switch (MI.getOpcode()) {
//
// Prolog handling.
@@ -172,6 +190,7 @@ bool X86WinEHUnwindV2::runOnMachineFunction(MachineFunction &MF) {
if (State != FunctionState::HasProlog)
llvm_unreachable("SEH_BeginEpilogue in prolog or another epilog");
State = FunctionState::InEpilog;
+ BeginEpilog = &MI;
break;
case X86::SEH_EndEpilogue:
@@ -192,7 +211,8 @@ bool X86WinEHUnwindV2::runOnMachineFunction(MachineFunction &MF) {
// epilog.
if (!UnwindV2StartLocation)
UnwindV2StartLocation = &MI;
- UnwindV2StartLocations.push_back(UnwindV2StartLocation);
+ EpilogInfos.push_back(
+ {UnwindV2StartLocation, ApproximateInstructionCount});
State = FunctionState::FinishedEpilog;
break;
@@ -305,37 +325,39 @@ bool X86WinEHUnwindV2::runOnMachineFunction(MachineFunction &MF) {
}
}
- if (UnwindV2StartLocations.empty())
- return false;
-
- MachineBasicBlock &FirstMBB = MF.front();
- // Assume +1 for the "header" UOP_Epilog that contains the epilog size, and
- // that we won't be able to use the "last epilog at the end of function"
- // optimization.
- if (ApproximatePrologCodeCount + UnwindV2StartLocations.size() + 1 >
- static_cast<unsigned>(MaximumUnwindCodes)) {
- if (Mode == WinX64EHUnwindV2Mode::Required)
- MF.getFunction().getContext().diagnose(DiagnosticInfoGenericWithLoc(
- "Windows x64 Unwind v2 is required, but the function '" +
- MF.getName() +
- "' has too many unwind codes. Try splitting the function or "
- "reducing the number of places where it exits early with a tail "
- "call.",
- MF.getFunction(), findDebugLoc(FirstMBB)));
-
- FailsUnwindV2Criteria++;
+ if (EpilogInfos.empty())
return false;
- }
MeetsUnwindV2Criteria++;
- // Emit the pseudo instruction that marks the start of each epilog.
+ // Walk the list of epilogs backwards and add new SEH pseudo instructions:
+ // * SEH_UnwindV2Start at the start of each epilog.
+ // * If the current instruction is too far away from where the last unwind
+ // info ended OR there are too many unwind codes in the info, then add
+ // SEH_SplitChained to finish the current info.
+ unsigned LastUnwindInfoEndPosition = ApproximateInstructionCount;
+ unsigned UnwindCodeCount = ApproximatePrologCodeCount + 1;
const TargetInstrInfo *TII = MF.getSubtarget().getInstrInfo();
- for (MachineInstr *MI : UnwindV2StartLocations) {
- BuildMI(*MI->getParent(), MI, MI->getDebugLoc(),
+ for (auto &Info : llvm::reverse(EpilogInfos)) {
+ MachineBasicBlock &MBB = *Info.UnwindV2StartLocation->getParent();
+ BuildMI(MBB, Info.UnwindV2StartLocation,
+ Info.UnwindV2StartLocation->getDebugLoc(),
TII->get(X86::SEH_UnwindV2Start));
+
+ if ((LastUnwindInfoEndPosition - Info.ApproximateInstructionPosition >=
+ InstructionCountThreshold) ||
+ (UnwindCodeCount >= UnwindCodeThreshold)) {
+ BuildMI(&MBB, Info.UnwindV2StartLocation->getDebugLoc(),
+ TII->get(X86::SEH_SplitChained));
+ LastUnwindInfoEndPosition = Info.ApproximateInstructionPosition;
+ // Doesn't reset to 0, as the prolog unwind codes are now in this info.
+ UnwindCodeCount = ApproximatePrologCodeCount + 1;
+ }
+
+ UnwindCodeCount++;
}
// Note that the function is using Unwind v2.
+ MachineBasicBlock &FirstMBB = MF.front();
BuildMI(FirstMBB, FirstMBB.front(), findDebugLoc(FirstMBB),
TII->get(X86::SEH_UnwindVersion))
.addImm(2);
diff --git a/llvm/test/CodeGen/X86/win64-eh-unwindv2-too-many-epilogs.mir b/llvm/test/CodeGen/X86/win64-eh-unwindv2-too-many-epilogs.mir
index 70c87ad87f792..ac54ec9f0652d 100644
--- a/llvm/test/CodeGen/X86/win64-eh-unwindv2-too-many-epilogs.mir
+++ b/llvm/test/CodeGen/X86/win64-eh-unwindv2-too-many-epilogs.mir
@@ -1,32 +1,22 @@
-# Require V2 and restrict the number of unwind codes to 8
-# RUN: not llc -mtriple=x86_64-pc-windows-msvc -o - %s \
-# RUN: -run-pass=x86-wineh-unwindv2 -x86-wineh-unwindv2-max-unwind-codes=8 \
-# RUN: 2>&1 | FileCheck %s -check-prefix=REQUIREV2
-
-# Force best-effort and restrict the number of unwind codes to 8
+# Restrict the number of unwind codes to 8
# RUN: llc -mtriple=x86_64-pc-windows-msvc -o - %s \
-# RUN: -run-pass=x86-wineh-unwindv2 -x86-wineh-unwindv2-max-unwind-codes=8 \
-# RUN: -x86-wineh-unwindv2-force-mode=1 | \
-# RUN: FileCheck %s -check-prefix=BESTEFFORT
+# RUN: -x86-wineh-unwindv2-unwind-codes-threshold=8 \
+# RUN: -run-pass=x86-wineh-unwindv2 | FileCheck %s \
+# RUN: -check-prefixes=ALLOWLESS,CHECK
-# Require V2, but allow the default number of unwind codes (255)
+# Allow the default number of unwind codes (255)
# RUN: llc -mtriple=x86_64-pc-windows-msvc -o - %s \
-# RUN: -run-pass=x86-wineh-unwindv2 | FileCheck %s -check-prefix=ALLOWMORE
-
-# Usually 255 unwind codes are permitted, but we passed an arg to llc to limit
-# it to 8.
-# REQUIREV2...
[truncated]
|
dpaoliello
force-pushed
the
unwindchain
branch
2 times, most recently
from
d3942a9 to
3d5cf87
Compare
dpaoliello
force-pushed
the
unwindchain
branch
from
3d5cf87 to
cd1e74a
Compare
dpaoliello
force-pushed
the
unwindchain
branch
from
cd1e74a to
7ab29e3
Compare
dpaoliello
force-pushed
the
unwindchain
branch
from
7ab29e3 to
f75db3e
Compare
dpaoliello
changed the title
dpaoliello
force-pushed
the
unwindchain
branch
from
f75db3e to
0e5176c
Compare
dpaoliello
force-pushed
the
unwindchain
branch
2 times, most recently
from
8001c09 to
5115a46
Compare
dpaoliello
force-pushed
the
unwindchain
branch
3 times, most recently
from
67179a3 to
1b53af3
Compare
dpaoliello
changed the title
dpaoliello
force-pushed
the
unwindchain
branch
from
1b53af3 to
dbb9bab
Compare
dpaoliello
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Windows x64 unwind v2 only permits an offset from the end of the current unwind info frame to the epilog of 4Kb. Since LLVM only uses one frame info per function, the furthest an epilog could be from the end of the function is 4Kb.
One way to permit offsets that are further is to use frame info chaining: each of the chained frame infos covers a different portion of the function and all point back to the parent unwind info.
LLVM's existing support for chaining frame infos is broken. Windows requires that each frame info (including the parent) is adjacent, but not overlapping, and the whole function is covered (i.e., each instruction maps to exactly one frame info). Therefore, having start/end chaining pseudo instructions doesn't make any sense, as every "end chain" pseudo would need a start pseudo immediately after it or be at the end of function. Instead, I switched to having a "split chain" pseudo instruction that ends the current frame and starts a new chained frame.
The x64 unwind v2 lowering will now keep track of the approximate number of instructions between each epilog and, if there are more than 600 instructions (chosen based on real world testing, this can be overridden with an LLVM flag) between the current epilog and the end of the current frame info, then a split frame info will be inserted.
Additionally, if the lowering detects that the current unwind info might have more than 255 bytes of unwind codes (limit per frame info, also overridable via LLVM flag) it will also split the frame info.
This change also fixes handling of unwind v2 support in functions with EH Handlers: specifically, the pass can now handler funclets that are bundled into the same
MachineFunctionas their callers and unwind info tables being emitted in the middle of a function.Additionally, allow the explicit stack dealloc to be elided if there is a
movto undo set frame and the function has frame pointers enabled.