Discussion: Darwin Sanitizers Stable ABI

We wish to make it possible to include the AddressSanitizer (ASan) runtime implementation in OSes and for this we need a stable ASan ABI. Based on previous discussions about this topic, our understanding is that freezing the present ABI would impose an excessive burden on other sanitizer developers and for unrelated platforms. Therefore, we propose adding a secondary stable ABI for our use and anyone else in the community seeking the same. We believe that we can define a stable ABI with minimal burden on the community, expecting only to keep existing tests running and implementing stubs when new features are added. We are okay with trading performance for stability with no impact for existing users of ASan while minimizing the maintenance burden for ASan maintainers. We wish to commit this functionality to the LLVM project to maintain it there. This new and stable ABI will abstract away the implementation details allowing new and novel approaches to ASan for developers, researchers and others.

Rather than adding a lot of conditional code to the LLVM instrumentation phase, which would incur excessive complexity and maintenance cost of adding conditional code into all places that emit a runtime call, we propose a “shim” layer which will map the unstable ABI to the stable ABI:

* A static library (.a library) shim that maps the existing ASan ABI to a generalized, smaller and stable ABI. The library would implement the __asan functions and call into the new ABI. For example:
    * `void __asan_load1(uptr p) { __asan_abi_loadn(p, 1, true); }`
    * `void __asan_load2(uptr p) { __asan_abi_loadn(p, 2, true); }`
    * `void __asan_noabort_load16(uptr p) { __asan_abi_loadn(p, 16, false); }`
    * `void __asan_poison_cxx_array_cookie(uptr p) { __asan_abi_pac(p); }`
* This “shim” library would only be used by people who opt in: A compilation flag in the Clang driver will be used to gate the use of the stable ABI workflow.
* Utilize the existing ability for the ASan instrumentation to prefer runtime calls instead of inlined direct shadow memory accesses.
* Pursue (under the new driver flag) a better separation of abstraction and implementation with:
    * LLVM instrumentation: Calling out for all poisoning, checking and unpoisoning.
    * Runtime: Implementing the stable ABI and being responsible of implementation details of the shadow memory.

Our aim is that the maintenance burden on the sanitizer developer community be negligible. Stable ABI tests will always pass for non-Darwin platforms. Changes to the existing ABI which would require a change to the shim have been infrequent as the ASan ABI is already relatively stable. Rarely, a change that impacts the contract between LLVM and the shim will occur. Among such foreseeable changes are: 1) changes to a function signature, 2) additions of new functions, or 3) deprecation of an existing function. Following are some examples of reasonable responses to those changes:

* Example: An existing ABI function is changed to return the input parameter on success or NULL on failure. In this scenario, a reasonable change to the shim would be to modify the function signature appropriately and to simply guess at a common-sense implementation.
    * `uptr __asan_load1(uptr p) { __asan_abi_loadn(p, 1, true); return p; }`
* Example: An additional function is added for performance reasons. It has a very similar function signature to other similarly named functions and logically is an extension of that same pattern. In this case it would make sense to apply the same logic as the existing entry points:
    * `void __asan_load128(uptr p) { __asan_abi_loadn(p, 128, true); }`
* Example: An entry point is added to the existing ABI for which there is no obvious stable ABI implementation: In this case, doing nothing in a no-op stub would be acceptable, assuming existing features of ASan can still work without an actual implementation of this new function.
    * `void __asan_prefetch(uptr p) { }`
* Example: An entrypoint in the existing ABI is deprecated and/or deleted:
    * (Delete the entrypoint from the shim.)

We’re looking for buy-in for this level of support.

(Note: Upon acceptance of the general concepts herein, we will add a controlling clang flag, cmake integration, contract for the stable ABI, and the appropriate test infrastructure.)

Reviewed By: eugenis, vitalybuka, MaskRay

Differential Revision: https://reviews.llvm.org/D143675

(cherry picked from commit 6f026ff)

Author: Roy Sundahl

clang/include/clang/Driver/Options.td

@@ -1756,6 +1756,10 @@ def fsanitize_address_outline_instrumentation : Flag<["-"], "fsanitize-address-o
 def fno_sanitize_address_outline_instrumentation : Flag<["-"], "fno-sanitize-address-outline-instrumentation">,
                                                    Group<f_clang_Group>,
                                                    HelpText<"Use default code inlining logic for the address sanitizer">;
+defm sanitize_stable_abi
+  : OptInCC1FFlag<"sanitize-stable-abi", "Stable  ", "Conventional ",
+    "ABI instrumentation for sanitizer runtime. Default: Conventional">;
+
 def fsanitize_memtag_mode_EQ : Joined<["-"], "fsanitize-memtag-mode=">,
                                         Group<f_clang_Group>,
                                         HelpText<"Set default MTE mode to 'sync' (default) or 'async'">;

clang/include/clang/Driver/SanitizerArgs.h

@@ -40,6 +40,7 @@ class SanitizerArgs {
   bool CfiCanonicalJumpTables = false;
   int AsanFieldPadding = 0;
   bool SharedRuntime = false;
+  bool StableABI = false;
   bool AsanUseAfterScope = true;
   bool AsanPoisonCustomArrayCookie = false;
   bool AsanGlobalsDeadStripping = false;

clang/lib/Driver/SanitizerArgs.cpp

@@ -900,6 +900,9 @@ SanitizerArgs::SanitizerArgs(const ToolChain &TC,
       }
     }
 
+    StableABI = Args.hasFlag(options::OPT_fsanitize_stable_abi,
+                             options::OPT_fno_sanitize_stable_abi, false);
+
     AsanUseAfterScope = Args.hasFlag(
         options::OPT_fsanitize_address_use_after_scope,
         options::OPT_fno_sanitize_address_use_after_scope, AsanUseAfterScope);
@@ -1254,6 +1257,16 @@ void SanitizerArgs::addArgs(const ToolChain &TC, const llvm::opt::ArgList &Args,
     CmdArgs.push_back("-asan-instrumentation-with-call-threshold=0");
   }
 
+  // When emitting Stable ABI instrumentation, force outlining calls and avoid
+  // inlining shadow memory poisoning. While this is a big performance burden
+  // for now it allows full abstraction from implementation details.
+  if (StableABI) {
+    CmdArgs.push_back("-mllvm");
+    CmdArgs.push_back("-asan-instrumentation-with-call-threshold=0");
+    CmdArgs.push_back("-mllvm");
+    CmdArgs.push_back("-asan-max-inline-poisoning-size=0");
+  }
+
   // Only pass the option to the frontend if the user requested,
   // otherwise the frontend will just use the codegen default.
   if (AsanDtorKind != llvm::AsanDtorKind::Invalid) {

clang/test/Driver/fsanitize-stable-abi.c

@@ -0,0 +1,15 @@
+// RUN: %clang --target=arm64-apple-darwin -fsanitize-stable-abi %s -### 2>&1 | \
+// RUN:     FileCheck %s --check-prefix=CHECK-ASAN-STABLE-WARN
+// CHECK-ASAN-STABLE-WARN: warning: argument unused during compilation: '-fsanitize-stable-abi'
+// RUN: %clang --target=arm64-apple-darwin -fsanitize=address -fsanitize-stable-abi %s -### 2>&1 | \
+// RUN:     FileCheck %s --check-prefix=CHECK-ASAN-STABLE-OK
+// RUN: %clang --target=arm64-apple-darwin -fsanitize=address -fno-sanitize-stable-abi -fsanitize-stable-abi %s -### 2>&1 | \
+// RUN:     FileCheck %s --check-prefix=CHECK-ASAN-STABLE-OK
+// CHECK-ASAN-STABLE-OK: "-mllvm" "-asan-instrumentation-with-call-threshold=0"
+// CHECK-ASAN-STABLE-OK: "-mllvm" "-asan-max-inline-poisoning-size=0"
+// RUN: %clang --target=arm64-apple-darwin -fsanitize=address -fno-sanitize-stable-abi %s -### 2>&1 | \
+// RUN:     FileCheck %s --check-prefix=CHECK-NO-ASAN-STABLE-OK
+// RUN: %clang --target=arm64-apple-darwin -fsanitize=address -fsanitize-stable-abi -fno-sanitize-stable-abi %s -### 2>&1 | \
+// RUN:     FileCheck %s --check-prefix=CHECK-NO-ASAN-STABLE-OK
+// CHECK-NO-ASAN-STABLE-OK-NOT: "-mllvm" "-asan-instrumentation-with-call-threshold=0"
+// CHECK-NO-ASAN-STABLE-OK-NOT: "-mllvm" "-asan-max-inline-poisoning-size=0"

compiler-rt/cmake/config-ix.cmake

@@ -701,7 +701,7 @@ if(COMPILER_RT_SUPPORTED_ARCH)
 endif()
 message(STATUS "Compiler-RT supported architectures: ${COMPILER_RT_SUPPORTED_ARCH}")
 
-set(ALL_SANITIZERS asan;dfsan;msan;hwasan;tsan;safestack;cfi;scudo;ubsan_minimal;gwp_asan)
+set(ALL_SANITIZERS asan;dfsan;msan;hwasan;tsan;safestack;cfi;scudo;ubsan_minimal;gwp_asan;asan_abi)
 set(COMPILER_RT_SANITIZERS_TO_BUILD all CACHE STRING
     "sanitizers to build if supported on the target (all;${ALL_SANITIZERS})")
 list_replace(COMPILER_RT_SANITIZERS_TO_BUILD all "${ALL_SANITIZERS}")
@@ -721,6 +721,11 @@ else()
   set(COMPILER_RT_HAS_INTERCEPTION FALSE)
 endif()
 
+if (COMPILER_RT_HAS_SANITIZER_COMMON AND ASAN_SUPPORTED_ARCH AND APPLE)
+  set(COMPILER_RT_HAS_ASAN_ABI TRUE)
+else()
+  set(COMPILER_RT_HAS_ASAN_ABI FALSE)
+endif()
 if (COMPILER_RT_HAS_SANITIZER_COMMON AND ASAN_SUPPORTED_ARCH)
   set(COMPILER_RT_HAS_ASAN TRUE)
 else()

compiler-rt/docs/ASanABI.rst

@@ -0,0 +1,51 @@
+.. _BuildingCompilerRT:
+
+============================
+Darwin Sanitizers Stable ABI
+============================
+
+Some OSes like Darwin want to include the AddressSanitizer runtime by establishing a stable ASan ABI. lib/asan_abi contains a secondary stable ABI for Darwin use and potentially others. The Stable ABI has minimal impact on the community, prioritizing stability over performance.
+
+The Stable ABI is isolated by a “shim” layer which maps the unstable ABI to the stable ABI. It consists of a static library (libclang_rt.asan_abi_osx.a) that contains simple mappings of the existing ASan ABI to the smaller Stable ABI. After linking with the static shim library, only calls to the Stable ABI remain. 
+
+  Sample content of the shim:
+
+  .. code-block:: c
+
+    void __asan_load1(uptr p) { __asan_abi_loadn(p, 1, true); }
+    void __asan_load2(uptr p) { __asan_abi_loadn(p, 2, true); }
+    void __asan_noabort_load16(uptr p) { __asan_abi_loadn(p, 16, false); }
+    void __asan_poison_cxx_array_cookie(uptr p) { __asan_abi_pac(p); }
+
+The shim library is only used when ``-fsanitize-stable-abi`` is specified in the Clang driver and the emitted instrumentation favors runtime calls over inline expansion.
+
+Maintenance
+-----------
+
+The maintenance burden on the sanitizer developer community should be negligible. Stable ABI tests should always pass for non-Darwin platforms. Changes to the existing ABI requiring changes to the shim should been infrequent as the existing ASan ABI has long been relatively stable anyway. Rarely, when a change that impacts the contract between LLVM and the shim occurs, some simple responses should suffice. Among such foreseeable changes are: 1) changes to a function signature, 2) additions of new functions, or 3) deprecation of an existing function.
+
+  Following are some examples of reasonable responses to such changes:
+
+  * An existing ABI function is changed to return the input parameter on success or NULL on failure. In this scenario, a reasonable change to the shim would be to modify the function signature appropriately and to simply guess at a common-sense implementation.
+
+    .. code-block:: c
+
+      uptr __asan_load1(uptr p) { __asan_abi_loadn(p, 1, true); return p; }
+
+  * An additional function is added for performance reasons. It has a very similar function signature to other similarly named functions and logically is an extension of that same pattern. In this case it would make sense to apply the same logic as the existing entry points:
+
+    .. code-block:: c
+
+      void __asan_load128(uptr p) { __asan_abi_loadn(p, 128, true); }
+
+  * An entry point is added to the existing ABI for which there is no obvious stable ABI implementation: In this case, doing nothing in a no-op stub would be acceptable, assuming existing features of ASan can still work without an actual implementation of this new function.
+
+    .. code-block:: c
+
+      void __asan_prefetch(uptr p) { }
+
+  * An entrypoint in the existing ABI is deprecated and/or deleted:
+
+    .. code-block:: c
+
+      (Delete the entrypoint from the shim.)

compiler-rt/lib/asan_abi/CMakeLists.txt

@@ -0,0 +1,35 @@
+# Build for the ASAN Stable ABI runtime support library.
+set(ASAN_ABI_SOURCES
+  asan_abi_shim.cpp
+  )
+
+set(ASAN_ABI_HEADERS
+  ../asan/asan_interface_internal.h
+  asan_abi.h
+  )
+
+include_directories(..)
+
+add_compiler_rt_component(asan_abi)
+
+if (APPLE)
+  # TODO: set in config-ix.cmake
+  set(ASAN_ABI_SUPPORTED_OS osx)
+  set(ASAN_ABI_SUPPORTED_ARCHS ${X86_64} ${ARM64})
+  # Compile Stable API sources into an object library.
+  add_compiler_rt_object_libraries(RTASAN_ABI
+    OS ${ASAN_ABI_SUPPORTED_OS}
+    ARCHS ${ASAN_ABI_SUPPORTED_ARCHS}
+    SOURCES ${ASAN_ABI_SOURCES}
+    ADDITIONAL_HEADERS ${ASAN_ABI_HEADERS}
+    CFLAGS ${SANITIZER_COMMON_CFLAGS})
+
+  add_compiler_rt_runtime(clang_rt.asan_abi
+    STATIC
+    OS ${ASAN_ABI_SUPPORTED_OS}
+    ARCHS ${ASAN_ABI_SUPPORTED_ARCHS}
+    OBJECT_LIBS RTASAN_ABI
+    CFLAGS ${SANITIZER_COMMON_CFLAGS}
+    LINK_FLAGS ${WEAK_SYMBOL_LINK_FLAGS}
+    PARENT_TARGET asan_abi)
+endif()

compiler-rt/lib/asan_abi/asan_abi.cpp

@@ -0,0 +1,85 @@
+//===-asan_abi.cpp - ASan Stable ABI---------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "asan_abi.h"
+
+extern "C" {
+// Functions concerning instrumented global variables:
+void __asan_abi_register_image_globals(void) {}
+void __asan_abi_unregister_image_globals(void) {}
+
+// Functions concerning dynamic library initialization
+void __asan_abi_before_dynamic_init(const char *module_name) {}
+void __asan_abi_after_dynamic_init(void) {}
+
+// Functions concerning block memory destinations
+void *__asan_abi_memcpy(void *d, const void *s, size_t n) { return NULL; }
+void *__asan_abi_memmove(void *d, const void *s, size_t n) { return NULL; }
+void *__asan_abi_memset(void *p, int c, size_t n) { return NULL; }
+
+// Functions concerning RTL startup and initialization
+void __asan_abi_init(void) {}
+void __asan_abi_handle_no_return(void) {}
+
+// Functions concerning memory load and store reporting
+void __asan_abi_report_load_n(void *p, size_t n, bool abort) {}
+void __asan_abi_report_exp_load_n(void *p, size_t n, int exp, bool abort) {}
+void __asan_abi_report_store_n(void *p, size_t n, bool abort) {}
+void __asan_abi_report_exp_store_n(void *p, size_t n, int exp, bool abort) {}
+
+// Functions concerning memory load and store
+void __asan_abi_load_n(void *p, size_t n, bool abort) {}
+void __asan_abi_exp_load_n(void *p, size_t n, int exp, bool abort) {}
+void __asan_abi_store_n(void *p, size_t n, bool abort) {}
+void __asan_abi_exp_store_n(void *p, size_t n, int exp, bool abort) {}
+
+// Functions concerning query about whether memory is poisoned
+int __asan_abi_address_is_poisoned(void const volatile *p) { return 0; }
+void *__asan_abi_region_is_poisoned(void const volatile *p, size_t size) {
+  return NULL;
+}
+
+// Functions concerning the poisoning of memory
+void __asan_abi_poison_memory_region(void const volatile *p, size_t n) {}
+void __asan_abi_unpoison_memory_region(void const volatile *p, size_t n) {}
+
+// Functions concerning the partial poisoning of memory
+void __asan_abi_set_shadow_xx_n(void *p, unsigned char xx, size_t n) {}
+
+// Functions concerning stack poisoning
+void __asan_abi_poison_stack_memory(void *p, size_t n) {}
+void __asan_abi_unpoison_stack_memory(void *p, size_t n) {}
+
+// Functions concerning redzone poisoning
+void __asan_abi_poison_intra_object_redzone(void *p, size_t size) {}
+void __asan_abi_unpoison_intra_object_redzone(void *p, size_t size) {}
+
+// Functions concerning array cookie poisoning
+void __asan_abi_poison_cxx_array_cookie(void *p) {}
+void *__asan_abi_load_cxx_array_cookie(void **p) { return NULL; }
+
+// Functions concerning fake stacks
+void *__asan_abi_get_current_fake_stack(void) { return NULL; }
+void *__asan_abi_addr_is_in_fake_stack(void *fake_stack, void *addr, void **beg,
+                                       void **end) {
+  return NULL;
+}
+
+// Functions concerning poisoning and unpoisoning fake stack alloca
+void __asan_abi_alloca_poison(void *addr, size_t size) {}
+void __asan_abi_allocas_unpoison(void *top, void *bottom) {}
+
+// Functions concerning fake stack malloc
+void *__asan_abi_stack_malloc_n(size_t scale, size_t size) { return NULL; }
+void *__asan_abi_stack_malloc_always_n(size_t scale, size_t size) {
+  return NULL;
+}
+
+// Functions concerning fake stack free
+void __asan_abi_stack_free_n(int scale, void *p, size_t n) {}
+}

compiler-rt/lib/asan_abi/asan_abi.h

@@ -0,0 +1,84 @@
+//===-asan_abi.h - ASan Stable ABI Interface-------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef ASAN_ABI_H
+#define ASAN_ABI_H
+
+#include <stdbool.h>
+#include <stddef.h>
+#include <sys/types.h>
+
+extern "C" {
+// Functions concerning instrumented global variables:
+void __asan_abi_register_image_globals();
+void __asan_abi_unregister_image_globals();
+
+// Functions concerning dynamic library initialization
+void __asan_abi_before_dynamic_init(const char *module_name);
+void __asan_abi_after_dynamic_init();
+
+// Functions concerning block memory destinations
+void *__asan_abi_memcpy(void *d, const void *s, size_t n);
+void *__asan_abi_memmove(void *d, const void *s, size_t n);
+void *__asan_abi_memset(void *p, int c, size_t n);
+
+// Functions concerning RTL startup and initialization
+void __asan_abi_init();
+void __asan_abi_handle_no_return();
+
+// Functions concerning memory load and store reporting
+void __asan_abi_report_load_n(void *p, size_t n, bool abort);
+void __asan_abi_report_exp_load_n(void *p, size_t n, int exp, bool abort);
+void __asan_abi_report_store_n(void *p, size_t n, bool abort);
+void __asan_abi_report_exp_store_n(void *p, size_t n, int exp, bool abort);
+
+// Functions concerning memory load and store
+void __asan_abi_load_n(void *p, size_t n, bool abort);
+void __asan_abi_exp_load_n(void *p, size_t n, int exp, bool abort);
+void __asan_abi_store_n(void *p, size_t n, bool abort);
+void __asan_abi_exp_store_n(void *p, size_t n, int exp, bool abort);
+
+// Functions concerning query about whether memory is poisoned
+int __asan_abi_address_is_poisoned(void const volatile *p);
+void *__asan_abi_region_is_poisoned(void const volatile *p, size_t size);
+
+// Functions concerning the poisoning of memory
+void __asan_abi_unpoison_memory_region(void const volatile *p, size_t n);
+void __asan_abi_poison_memory_region(void const volatile *p, size_t n);
+
+// Functions concerning the partial poisoning of memory
+void __asan_abi_set_shadow_xx_n(void *p, unsigned char xx, size_t n);
+
+// Functions concerning stack poisoning
+void __asan_abi_poison_stack_memory(void *p, size_t n);
+void __asan_abi_unpoison_stack_memory(void *p, size_t n);
+
+// Functions concerning redzone poisoning
+void __asan_abi_poison_intra_object_redzone(void *p, size_t size);
+void __asan_abi_unpoison_intra_object_redzone(void *p, size_t size);
+
+// Functions concerning array cookie poisoning
+void __asan_abi_poison_cxx_array_cookie(void *p);
+void *__asan_abi_load_cxx_array_cookie(void **p);
+
+// Functions concerning fake stacks
+void *__asan_abi_get_current_fake_stack();
+void *__asan_abi_addr_is_in_fake_stack(void *fake_stack, void *addr, void **beg,
+                                       void **end);
+// Functions concerning poisoning and unpoisoning fake stack alloca
+void __asan_abi_alloca_poison(void *addr, size_t size);
+void __asan_abi_allocas_unpoison(void *top, void *bottom);
+
+// Functions concerning fake stack malloc
+void *__asan_abi_stack_malloc_n(size_t scale, size_t size);
+void *__asan_abi_stack_malloc_always_n(size_t scale, size_t size);
+
+// Functions concerning fake stack free
+void __asan_abi_stack_free_n(int scale, void *p, size_t n);
+}
+#endif // ASAN_ABI_H