Skip to content

Conversation

@zyedidia
Copy link
Contributor

@zyedidia zyedidia commented Nov 8, 2025

This PR is the first step towards introducing LFI into LLVM as a new sub-architecture backend of AArch64. For details, please see the RFC, which has been approved for AArch64.

This patch creates the aarch64_lfi architecture, and marks the appropriate registers as reserved when it is targeted (x25, x26, x27, x28). It also adds a Clang driver toolchain for targeting LFI, and updates the compiler-rt CMake to allow builds for the aarch64_lfi target. The patch also includes documentation for LFI and the rewrites that will be implemented in future patches.

I am planning to split the relevant modifications for LFI into a series of patches, organized as described below (after this one). Please let me know if you'd like me to split the changes in a different way, or provide one big patch.

  1. The next patch will introduce the MCLFIExpander mechanism for applying the MC-level rewrites needed by LFI, along with the .lfi_expand and .lfi_no_expand assembly directives when targeting LFI. A preview can be seen on the lfi-project fork.

  2. The following patch will create an MCLFIExpander for the AArch64 backend that performs LFI expansions. This patch will contain the majority of the LFI-specific logic.

  3. The final patch will add an optimization to the rewriter that can eliminate redundant guard instructions that occur within the same basic block.

We plan to introduce x86-64 support after further discussion and once the MCLFIExpander infrastructure is in place.

Please let me know your feedback, and thank you very much for your help and guidance in the review process.

@llvmbot llvmbot added clang Clang issues not falling into any other category compiler-rt backend:AArch64 clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' clang:frontend Language frontend issues, e.g. anything involving "Sema" compiler-rt:builtins labels Nov 8, 2025
@llvmbot
Copy link
Member

llvmbot commented Nov 8, 2025

@llvm/pr-subscribers-clang-driver
@llvm/pr-subscribers-backend-aarch64

@llvm/pr-subscribers-clang

Author: Zachary Yedidia (zyedidia)

Changes

This PR is the first step towards introducing LFI into LLVM as a new sub-architecture backend of AArch64. For details, please see the RFC.

This patch creates the aarch64_lfi architecture, and marks the appropriate registers as reserved when it is targeted (x25, x26, x27, x28). It also adds a Clang driver toolchain for targeting LFI, and updates the compiler-rt CMake to allow builds for the aarch64_lfi target. The patch also includes documentation for LFI and the rewrites that will be implemented in future patches.

I am planning to split the relevant modifications for LFI into a series of patches, organized as described below (after this one). Please let me know if you'd like me to split the changes in a different way, or provide one big patch.

  1. The next patch will introduce the MCLFIExpander mechanism for applying the MC-level rewrites needed by LFI, along with the .lfi_expand and .lfi_no_expand assembly directives when targeting LFI. A preview can be seen on the lfi-project fork.

  2. The following patch will create an MCLFIExpander for the AArch64 backend that performs LFI expansions. This patch will contain the majority of the LFI-specific logic.

  3. The final patch will add an optimization to the rewriter that can eliminate redundant guard instructions that occur within the same basic block.

We plan to introduce x86-64 support after further discussion and once the MCLFIExpander infrastructure is in place.

Please let me know your feedback, and thank you very much for your help and guidance in the review process.

Tagging @Sharjeel-Khan @talg @pirama-arumuga-nainar @MaskRay @aengelke @efriedma-quic


Patch is 28.12 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/167061.diff

12 Files Affected:

  • (modified) clang/lib/Basic/Targets/AArch64.cpp (+3)
  • (modified) clang/lib/Driver/CMakeLists.txt (+1)
  • (modified) clang/lib/Driver/Driver.cpp (+3)
  • (added) clang/lib/Driver/ToolChains/LFILinux.cpp (+29)
  • (added) clang/lib/Driver/ToolChains/LFILinux.h (+35)
  • (modified) compiler-rt/cmake/builtin-config-ix.cmake (+1-1)
  • (modified) compiler-rt/lib/builtins/CMakeLists.txt (+1)
  • (added) llvm/docs/LFI.rst (+387)
  • (modified) llvm/include/llvm/TargetParser/Triple.h (+7)
  • (modified) llvm/lib/Target/AArch64/AArch64RegisterInfo.cpp (+11)
  • (modified) llvm/lib/Target/AArch64/AArch64Subtarget.h (+1)
  • (modified) llvm/lib/TargetParser/Triple.cpp (+6)
diff --git a/clang/lib/Basic/Targets/AArch64.cpp b/clang/lib/Basic/Targets/AArch64.cpp
index a97e93470987c..0cc4db1e637ce 100644
--- a/clang/lib/Basic/Targets/AArch64.cpp
+++ b/clang/lib/Basic/Targets/AArch64.cpp
@@ -412,6 +412,9 @@ void AArch64TargetInfo::getTargetDefines(const LangOptions &Opts,
     Builder.defineMacro("__aarch64__");
   }
 
+  if (getTriple().isLFI())
+    Builder.defineMacro("__LFI__");
+
   // Inline assembly supports AArch64 flag outputs.
   Builder.defineMacro("__GCC_ASM_FLAG_OUTPUTS__");
 
diff --git a/clang/lib/Driver/CMakeLists.txt b/clang/lib/Driver/CMakeLists.txt
index 7c4f70b966c48..9060b63d75d6e 100644
--- a/clang/lib/Driver/CMakeLists.txt
+++ b/clang/lib/Driver/CMakeLists.txt
@@ -65,6 +65,7 @@ add_clang_library(clangDriver
   ToolChains/Hexagon.cpp
   ToolChains/HLSL.cpp
   ToolChains/Hurd.cpp
+  ToolChains/LFILinux.cpp
   ToolChains/Linux.cpp
   ToolChains/Managarm.cpp
   ToolChains/MipsLinux.cpp
diff --git a/clang/lib/Driver/Driver.cpp b/clang/lib/Driver/Driver.cpp
index a0b82cec9a372..e0323cf631fca 100644
--- a/clang/lib/Driver/Driver.cpp
+++ b/clang/lib/Driver/Driver.cpp
@@ -29,6 +29,7 @@
 #include "ToolChains/Haiku.h"
 #include "ToolChains/Hexagon.h"
 #include "ToolChains/Hurd.h"
+#include "ToolChains/LFILinux.h"
 #include "ToolChains/Lanai.h"
 #include "ToolChains/Linux.h"
 #include "ToolChains/MSP430.h"
@@ -6864,6 +6865,8 @@ const ToolChain &Driver::getToolChain(const ArgList &Args,
         TC = std::make_unique<toolchains::OHOS>(*this, Target, Args);
       else if (Target.isWALI())
         TC = std::make_unique<toolchains::WebAssembly>(*this, Target, Args);
+      else if (Target.isLFI())
+        TC = std::make_unique<toolchains::LFILinuxToolChain>(*this, Target, Args);
       else
         TC = std::make_unique<toolchains::Linux>(*this, Target, Args);
       break;
diff --git a/clang/lib/Driver/ToolChains/LFILinux.cpp b/clang/lib/Driver/ToolChains/LFILinux.cpp
new file mode 100644
index 0000000000000..7ab2372756af9
--- /dev/null
+++ b/clang/lib/Driver/ToolChains/LFILinux.cpp
@@ -0,0 +1,29 @@
+//===-- LFILinux.cpp - LFI ToolChain Implementations ------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "LFILinux.h"
+#include "clang/Driver/Driver.h"
+#include "clang/Driver/Options.h"
+
+using namespace clang::driver::toolchains;
+using namespace llvm::opt;
+
+void LFILinuxToolChain::AddCXXStdlibLibArgs(const ArgList &Args,
+                                            ArgStringList &CmdArgs) const {
+  switch (GetCXXStdlibType(Args)) {
+  case ToolChain::CST_Libcxx:
+    CmdArgs.push_back("-lc++");
+    if (Args.hasArg(options::OPT_fexperimental_library))
+      CmdArgs.push_back("-lc++experimental");
+    CmdArgs.push_back("-lc++abi");
+    break;
+  case ToolChain::CST_Libstdcxx:
+    CmdArgs.push_back("-lstdc++");
+    break;
+  }
+}
diff --git a/clang/lib/Driver/ToolChains/LFILinux.h b/clang/lib/Driver/ToolChains/LFILinux.h
new file mode 100644
index 0000000000000..e85f74803c1ce
--- /dev/null
+++ b/clang/lib/Driver/ToolChains/LFILinux.h
@@ -0,0 +1,35 @@
+//===--- LFILinux.h - LFI ToolChain Implementations -------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_LIB_DRIVER_TOOLCHAINS_LFI_LINUX_H
+#define LLVM_CLANG_LIB_DRIVER_TOOLCHAINS_LFI_LINUX_H
+
+#include "Linux.h"
+
+namespace clang {
+namespace driver {
+namespace toolchains {
+
+class LLVM_LIBRARY_VISIBILITY LFILinuxToolChain : public Linux {
+public:
+  LFILinuxToolChain(const Driver &D, const llvm::Triple &Triple,
+                    const llvm::opt::ArgList &Args)
+      : Linux(D, Triple, Args) {
+    ExtraOpts.push_back("-z");
+    ExtraOpts.push_back("separate-code");
+  }
+
+  void AddCXXStdlibLibArgs(const llvm::opt::ArgList &Args,
+                           llvm::opt::ArgStringList &CmdArgs) const override;
+};
+
+} // end namespace toolchains
+} // end namespace driver
+} // end namespace clang
+
+#endif // LLVM_CLANG_LIB_DRIVER_TOOLCHAINS_LFI_LINUX_H
diff --git a/compiler-rt/cmake/builtin-config-ix.cmake b/compiler-rt/cmake/builtin-config-ix.cmake
index eaff8135cff45..3298fe96cd236 100644
--- a/compiler-rt/cmake/builtin-config-ix.cmake
+++ b/compiler-rt/cmake/builtin-config-ix.cmake
@@ -78,7 +78,7 @@ else()
 endif()
 
 set(AMDGPU amdgcn)
-set(ARM64 aarch64 arm64ec)
+set(ARM64 aarch64 arm64ec aarch64_lfi)
 set(ARM32 arm armhf armv4t armv5te armv6 armv6m armv7m armv7em armv7 armv7s armv7k armv8m.base armv8m.main armv8.1m.main)
 set(AVR avr)
 set(HEXAGON hexagon)
diff --git a/compiler-rt/lib/builtins/CMakeLists.txt b/compiler-rt/lib/builtins/CMakeLists.txt
index 6c226aa7d2d48..29c5a08b828cb 100644
--- a/compiler-rt/lib/builtins/CMakeLists.txt
+++ b/compiler-rt/lib/builtins/CMakeLists.txt
@@ -692,6 +692,7 @@ set(arm64_SOURCES ${aarch64_SOURCES})
 set(arm64e_SOURCES ${aarch64_SOURCES})
 set(arm64_32_SOURCES ${aarch64_SOURCES})
 set(arm64ec_SOURCES ${aarch64_SOURCES})
+set(aarch64_lfi_SOURCES ${aarch64_SOURCES})
 
 # macho_embedded archs
 set(armv6m_SOURCES ${thumb1_SOURCES})
diff --git a/llvm/docs/LFI.rst b/llvm/docs/LFI.rst
new file mode 100644
index 0000000000000..22af7b1174e86
--- /dev/null
+++ b/llvm/docs/LFI.rst
@@ -0,0 +1,387 @@
+=========================================
+Lightweight Fault Isolation (LFI) in LLVM
+=========================================
+
+.. contents::
+   :local:
+
+Introduction
+++++++++++++
+
+Lightweight Fault Isolation (LFI) is a compiler-based sandboxing technology for
+native code. Like WebAssembly and Native Client, LFI isolates sandboxed code in-process
+(i.e., in the same address space as a host application).
+
+LFI is designed from the ground up to sandbox existing code, such as C/C++
+libraries (including assembly code) and device drivers.
+
+LFI aims for the following goals:
+
+* Compatibility: LFI can be used to sandbox nearly all existing C/C++/assembly
+  libraries unmodified (they just need to be recompiled). Sandboxed libraries
+  work with existing system call interfaces, and are compatible with existing
+  development tools such as profilers, debuggers, and sanitizers.
+* Performance: LFI aims for minimal overhead vs. unsandboxed code.
+* Security: The LFI runtime and compiler elements aim to be simple and
+  verifiable when possible.
+* Usability: LFI aims to make it easy as possible to used retrofit sandboxing,
+  i.e., to migrate from unsandboxed to sandboxed libraries with minimal effort.
+
+When building a program for the LFI target the compiler is designed to ensure
+that the program will only be able to access memory within a limited region of
+the virtual address space, starting from where the program is loaded (the
+current design sets this region to a size of 4GiB of virtual memory). Programs
+built for the LFI target are restricted to using a subset of the instruction
+set, designed so that the programs can be soundly confined to their sandbox
+region. LFI programs must run inside of an "emulator" (usually called the LFI
+runtime), responsible for initializing the sandbox region, loading the program,
+and servicing system call requests, or other forms of runtime calls.
+
+LFI uses an architecture-specific sandboxing scheme based on the general
+technique of Software-Based Fault Isolation (SFI). Initial support for LFI in
+LLVM is focused on the AArch64 platform, with x86-64 support planned for the
+future. The initial version of LFI for AArch64 is designed to support the
+Armv8.1 AArch64 architecture.
+
+See `https://github.com/lfi-project <https://github.com/lfi-project/>`__ for
+details about the LFI project and additional software needed to run LFI
+programs.
+
+Compiler Requirements
++++++++++++++++++++++
+
+When building for the ``aarch64_lfi`` target, the compiler must restrict use of
+the instruction set to a subset of instructions, which are known to be safe
+from a sandboxing perspective. To do this, we apply a set of simple rewrites at
+the assembly language level to transform standard native AArch64 assembly into
+LFI-compatible AArch64 assembly.
+
+These rewrites (also called "expansions") are applied at the very end of the
+LLVM compilation pipeline (during the assembler step). This allows the rewrites
+to be applied to hand-written assembly, including inline assembly.
+
+Compiler Options
+================
+
+The LFI target has several configuration options.
+
+* ``+lfi-stores``: create a "stores-only" sandbox, where rewrites are not applied to loads.
+* ``+lfi-jumps``: create a "jumps-only" sandbox, where rewrites are not applied to loads/stores.
+
+Reserved Registers
+==================
+
+The LFI target uses a custom ABI that reserves additional registers for the
+platform. The registers are listed below, along with the security invariant
+that must be maintained.
+
+* ``x27``: always holds the sandbox base address.
+* ``x28``: always holds an address within the sandbox.
+* ``sp``: always holds an address within the sandbox.
+* ``x30``: always holds an address within the sandbox.
+* ``x26``: scratch register.
+* ``x25``: points to a thread-local virtual register file for storing runtime context information.
+
+Linker Support
+==============
+
+In the initial version, LFI only supports static linking, and only supports
+creating ``static-pie`` binaries. There is nothing that fundamentally precludes
+support for dynamic linking on the LFI target, but such support would require
+that the code generated by the linker for PLT entries be slightly modified in
+order to conform to the LFI architecture subset.
+
+Assembly Rewrites
+=================
+
+Terminology
+~~~~~~~~~~~
+
+In the following assembly rewrites, some shorthand is used.
+
+* ``xN`` or ``wN``: refers to any general-purpose non-reserved register.
+* ``{a,b,c}``: matches any of ``a``, ``b``, or ``c``.
+* ``LDSTr``: a load/store instruction that supports register-register addressing modes, with one source/destination register.
+* ``LDSTx``: a load/store instruction not matched by ``LDSTr``.
+
+Control flow
+~~~~~~~~~~~~
+
+Indirect branches get rewritten to branch through register ``x28``, which must
+always contain an address within the sandbox. An ``add`` is used to safely load
+``x28`` with the destination address. Since ``ret`` uses ``x30`` by default,
+which already must contain an address within the sandbox, it does not require
+any rewrite.
+
++--------------------+---------------------------+
+|      Original      |         Rewritten         |
++--------------------+---------------------------+
+| .. code-block::    | .. code-block::           |
+|                    |                           |
+|    {br,blr,ret} xN |    add x28, x27, wN, uxtw |
+|                    |    {br,blr,ret} x28       |
+|                    |                           |
++--------------------+---------------------------+
+| .. code-block::    | .. code-block::           |
+|                    |                           |
+|    ret             |    ret                    |
+|                    |                           |
++--------------------+---------------------------+
+
+Memory accesses
+~~~~~~~~~~~~~~~
+
+Memory accesses are rewritten to use the ``[x27, wM, uxtw]`` addressing mode if
+it is available, which is automatically safe. Otherwise, rewrites fall back to
+using ``x28`` along with an instruction to safely load it with the target
+address.
+
++---------------------------------+-------------------------------+
+|            Original             |           Rewritten           |
++---------------------------------+-------------------------------+
+| .. code-block::                 | .. code-block::               |
+|                                 |                               |
+|    LDSTr xN, [xM]               |    LDSTr xN, [x27, wM, uxtw]  |
+|                                 |                               |
++---------------------------------+-------------------------------+
+| .. code-block::                 | .. code-block::               |
+|                                 |                               |
+|    LDSTr xN, [xM, #I]           |    add x28, x27, wM, uxtw     |
+|                                 |    LDSTr xN, [x28, #I]        |
+|                                 |                               |
++---------------------------------+-------------------------------+
+| .. code-block::                 | .. code-block::               |
+|                                 |                               |
+|    LDSTr xN, [xM, #I]!          |    add xM, xM, #I             |
+|                                 |    LDSTr xN, [x27, wM, uxtw]  |
+|                                 |                               |
++---------------------------------+-------------------------------+
+| .. code-block::                 | .. code-block::               |
+|                                 |                               |
+|    LDSTr xN, [xM], #I           |    LDSTr xN, [x27, wM, uxtw]  |
+|                                 |    add xM, xM, #I             |
+|                                 |                               |
++---------------------------------+-------------------------------+
+| .. code-block::                 | .. code-block::               |
+|                                 |                               |
+|    LDSTr xN, [xM1, xM2]         |    add x26, xM1, xM2          |
+|                                 |    LDSTr xN, [x27, w26, uxtw] |
+|                                 |                               |
++---------------------------------+-------------------------------+
+| .. code-block::                 | .. code-block::               |
+|                                 |                               |
+|    LDSTr xN, [xM1, xM2, MOD #I] |    add x26, xM1, xM2, MOD #I  |
+|                                 |    LDSTr xN, [x27, w26, uxtw] |
+|                                 |                               |
++---------------------------------+-------------------------------+
+| .. code-block::                 | .. code-block::               |
+|                                 |                               |
+|    LDSTx ..., [xM]              |    add x28, x27, wM, uxtw     |
+|                                 |    LDSTx ..., [x28]           |
+|                                 |                               |
++---------------------------------+-------------------------------+
+| .. code-block::                 | .. code-block::               |
+|                                 |                               |
+|    LDSTx ..., [xM, #I]          |    add x28, x27, wM, uxtw     |
+|                                 |    LDSTx ..., [x28, #I]       |
+|                                 |                               |
++---------------------------------+-------------------------------+
+| .. code-block::                 | .. code-block::               |
+|                                 |                               |
+|    LDSTx ..., [xM, #I]!         |    add x28, x27, wM, uxtw     |
+|                                 |    LDSTx ..., [x28, #I]       |
+|                                 |    add xM, xM, #I             |
+|                                 |                               |
++---------------------------------+-------------------------------+
+| .. code-block::                 | .. code-block::               |
+|                                 |                               |
+|    LDSTx ..., [xM], #I          |    add x28, x27, wM, uxtw     |
+|                                 |    LDSTx ..., [x28]           |
+|                                 |    add xM, xM, #I             |
+|                                 |                               |
++---------------------------------+-------------------------------+
+| .. code-block::                 | .. code-block::               |
+|                                 |                               |
+|    LDSTx ..., [xM1], xM2        |    add x28, x27, wM1, uxtw    |
+|                                 |    LDSTx ..., [x28]           |
+|                                 |    add xM1, xM1, xM2          |
+|                                 |                               |
++---------------------------------+-------------------------------+
+
+Stack pointer modification
+~~~~~~~~~~~~~~~~~~
+
+When the stack pointer is modified, we write the modified value to a temporary,
+before loading it back into ``sp`` with a safe ``add``.
+
++------------------------------+-------------------------------+
+|           Original           |           Rewritten           |
++------------------------------+-------------------------------+
+| .. code-block::              | .. code-block::               |
+|                              |                               |
+|    mov sp, xN                |    add sp, x27, wN, uxtw      |
+|                              |                               |
++------------------------------+-------------------------------+
+| .. code-block::              | .. code-block::               |
+|                              |                               |
+|    {add,sub} sp, sp, {#I,xN} |    {add,sub} x26, sp, {#I,xN} |
+|                              |    add sp, x27, w26, uxtw     |
+|                              |                               |
++------------------------------+-------------------------------+
+
+Link register modification
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When the link register is modified, we write the modified value to a
+temporary, before loading it back into ``x30`` with a safe ``add``.
+
++-----------------------+----------------------------+
+|       Original        |         Rewritten          |
++-----------------------+----------------------------+
+| .. code-block::       | .. code-block::            |
+|                       |                            |
+|    ldr x30, [...]     |    ldr x26, [...]          |
+|                       |    add x30, x27, w26, uxtw |
+|                       |                            |
++-----------------------+----------------------------+
+| .. code-block::       | .. code-block::            |
+|                       |                            |
+|    ldp xN, x30, [...] |    ldp xN, x26, [...]      |
+|                       |    add x30, x27, w26, uxtw |
+|                       |                            |
++-----------------------+----------------------------+
+| .. code-block::       | .. code-block::            |
+|                       |                            |
+|    ldp x30, xN, [...] |    ldp x26, xN, [...]      |
+|                       |    add x30, x27, w26, uxtw |
+|                       |                            |
++-----------------------+----------------------------+
+
+System instructions
+~~~~~~~~~~~~~~~~~~~
+
+System calls are rewritten into a sequence that loads the address of the first
+runtime call entrypoint and jumps to it. The runtime call entrypoint table is
+stored at the start of the sandbox, so it can be referenced by ``x27``. The
+rewrite also saves and restores the link register, since it is used for
+branching into the runtime.
+
++-----------------+----------------------------+
+|    Original     |         Rewritten          |
++-----------------+----------------------------+
+| .. code-block:: | .. code-block::            |
+|                 |                            |
+|    svc #0       |    mov w26, w30            |
+|                 |    ldr x30, [x27]          |
+|                 |    blr x30                 |
+|                 |    add x30, x27, w26, uxtw |
+|                 |                            |
++-----------------+----------------------------+
+
+Thread-local storage
+~~~~~~~~~~~~~~~~~~~~
+
+TLS accesses are rewritten into accesses offset from ``x25``, which is a
+reserved register that points to a virtual register file, with a location for
+storing the sandbox's thread pointer. ``TP`` is the offset into that virtual
+register...
[truncated]

@github-actions
Copy link

github-actions bot commented Nov 8, 2025

✅ With the latest revision this PR passed the C/C++ code formatter.

@davemgreen davemgreen requested a review from smithp35 November 18, 2025 09:04
@zyedidia
Copy link
Contributor Author

Pinging for reviews @MaskRay @AaronBallman @davemgreen @smithp35 @efriedma-quic. Thanks!

Copy link
Collaborator

@smithp35 smithp35 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My apologies for the delay, managed to catch me at a busy time in the day job. Will be out of office till next week so may be a bit slow to reply.

I appreciate that you've included the documentation in the patch, I've put some questions there that probably don't affect this patch, but may affect later ones. I can't spot anything obviously wrong in this patch.

The one important decision in this patch is the choice of a subtarget for LFI. I guess the most obvious alternatives would be a LFI envionment (like pauthtest), or an ABI (like aapcs-soft). LFI is a bit strange in that it is kind of a mixture of all three. The instructions are restricted to what the verifier can accept, the syscalls are limited to what the sandbox accepts, and there is a different procedure call standard. Given that objects compiled for LFI won't be compatible with non-LFI I think a sub-target is probably the best choice, or at least I can't think of a better one.

Your suggestions of how to split the patch sound reasonable to me. Without spending a lot of effort looking at the full patch, it is diffcult to know if there is a significantly better way. I guess we'll let you know if it needs splitting again.

I expect to have more comments on later patches. Particularly if there's anything we need to document in the Arm ABI. For example identifying relocatable objects as LFI (to diagnose clashes with non-LFI at link-time) and identifying executables as requiring LFI.

using namespace clang::driver::toolchains;
using namespace llvm::opt;

void LFILinuxToolChain::AddCXXStdlibLibArgs(const ArgList &Args,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't spot a significant difference between this and ToolChain::AddCXXStdlibLibArgs are you able to call the Base class implementation here?

If there's going to be changes in a later patch, would be good to leave a comment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The base implementation doesn't add -lc++abi, but I have updated the code to call the base implementation and then add -lc++abi for some additional code reuse.

technique of Software-Based Fault Isolation (SFI). Initial support for LFI in
LLVM is focused on the AArch64 platform, with x86-64 support planned for the
future. The initial version of LFI for AArch64 is designed to support the
Armv8.1 AArch64 architecture.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be useful to set out what the minimum and maximum are here. If I've understood LFI correctly then the maximum is determined by what the verifier can understand, the minimum could in theory be v8-A but the platform may insist on a minium of v8.1-m, presumably to get inline LSE atomics.

One aspect I'm particularly interested in is the NOP/HINT space instructions used by -mbranch-protection=standard, such as BTI and PACRET. I would expect BTI to be useful within the sandbox without any extra work. For PACRET there could be platform/sandbox interactions with the PAC keys.

What is expected to happen if someone does --target=aarch64_lfi-linux-musl with an out of range target or set of features? Does it give an error message or does it silently remove incompatible features?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes the minimum is v8-A, and the maximum is determined by the verifier. Currently the verifier is designed for v8.1 but currently does allow BTI and PACRET instructions as well, so LFI can be used with -mbranch-protection=standard.

I think a reasonable approach is to have a documented minimal subset (v8.1 initially for example, but probably with selected extensions like BTI/PACRET down the line) supported by the verifier and the rewriter, where we would want to produce compiler errors on a best-effort basis for code that would not pass verification.

The rewriter may be able to safely rewrite many instructions outside that subset as well, just by virtue of many of them being safe without modification, or only requiring a standard rewrite of the addressing mode. If the the user provides an out-of-range set of features the rewriter will just handle it as best it can, but a lack of rewrite or use of a fancy instruction may cause an error from the verifier.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if just allowing armv8.1 is OK. There is a lot of code that people will want to put into a sandbox (like codecs) that has runtime dispatch for making use of new instructions and architecture features, and not allowing that can be a big performance drop on its own.


These rewrites (also called "expansions") are applied at the very end of the
LLVM compilation pipeline (during the assembler step). This allows the rewrites
to be applied to hand-written assembly, including inline assembly.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although likely to be rare. I expect that there are some assembly examples that cannot be rewritten. I assume in that case, the user gets a "incompatible with LFI, please rewite your program" error message.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, these are mostly cases of instructions that use reserved registers, or that we don't have safe rewrites for (scatter-gather or privileged instructions for example). I think down the line we can introduce mechanisms to rewrite uses of reserved registers to spill to a thread-local memory location though.

Sets libc++ as the default libcxx for LFI, and uses superclass for
setting libcxx arguments.
Copy link
Collaborator

@smithp35 smithp35 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates. I've resolved most of the conversations.

One other thing that could be worth considering for the future is how the rewrites interact with linker optimization (relaxation) https://github.com/ARM-software/abi-aa/blob/main/aaelf64/aaelf64.rst#relocation-optimization

To some extent this is just delaying some rewriting until link time, it would only make sense if the extra information available at link time, such as the destination address of targets would permit better code-sequences.

Adds info about the adrp optimization and updates plan for
lfi-stores/lfi-loads features.
@zyedidia
Copy link
Contributor Author

I've updated the docs and the toolchain file based on your feedback, thanks! Regarding linker relaxation: yes some of the rewrites can impede some linker relaxations which is unfortunate but I don't think it's a big problem at the moment. Possibly in the future we could make the linker more aware of the LFI target (e.g., knowledge that x28 is reserved and therefore useless guards that exist post-relaxation can be eliminated).

directory. This code generator is capable of targeting a variety of
AMD GPU processors. Refer to :doc:`AMDGPUUsage` for more information.

The Lightweight Fault Isolation (LFI) backend
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is not expected to be a LFI backend, right? In the llvm sense of backend==target, a new llvm/Target/LFI. It is a modification to the existing backends that support it.

* Performance: LFI aims for minimal overhead vs. unsandboxed code.
* Security: The LFI runtime and compiler elements aim to be simple and
verifiable when possible.
* Usability: LFI aims to make it easy as possible to used retrofit sandboxing,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe to use? or to be used for?

~~~~~~~~~~~~

Indirect branches get rewritten to branch through register ``x28``, which must
always contain an address within the sandbox. An ``add`` is used to safely load
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

load here isnt a load so it might be better to use update.

~~~~~~~~~~~~~~~~~~~~~~~~~~

When the stack pointer is modified, we write the modified value to a temporary,
before loading it back into ``sp`` with a safe ``add``.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

loading -> moving

technique of Software-Based Fault Isolation (SFI). Initial support for LFI in
LLVM is focused on the AArch64 platform, with x86-64 support planned for the
future. The initial version of LFI for AArch64 is designed to support the
Armv8.1 AArch64 architecture.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if just allowing armv8.1 is OK. There is a lot of code that people will want to put into a sandbox (like codecs) that has runtime dispatch for making use of new instructions and architecture features, and not allowing that can be a big performance drop on its own.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend:AArch64 clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' clang:frontend Language frontend issues, e.g. anything involving "Sema" clang Clang issues not falling into any other category compiler-rt:builtins compiler-rt

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants