[TySan][Clang] Add clang flag to use tysan outlined instrumentation and update docs

Author: gbMattN

clang/docs/TypeSanitizer.rst

@@ -20,9 +20,13 @@ code is build with ``-fno-strict-aliasing``, sacrificing performance.
 TypeSanitizer is built to catch when these strict aliasing rules have been violated, helping 
 users find where such bugs originate in their code despite the code looking valid at first glance.
 
-As TypeSanitizer is still experimental, it can currently have a large impact on runtime speed, 
-memory use, and code size. It also has a large compile-time overhead. Work is being done to 
-reduce these impacts.
+Typical memory overhead introduced by TypeSanitizer is about **8x**. Runtime slowdown varies greatly
+depending on how often the instrumented code relies on type aliasing. In the best case slowdown is
+**2x-3x**.
+
+The compiler instrumentation also has an impact on code size and compilation overhead. There is an
+experimental :ref:`instrumentation outlining option<outlining_flag>` which can greatly reduce this
+but this may decrease runtime performance.
 
 The TypeSanitizer Algorithm
 ===========================
@@ -128,6 +132,23 @@ references to LLVM IR specific terms.
 Sanitizer features
 ==================
 
+.. _outlining_flag:
+
+Instrumentation code outlining
+------------------------------
+
+By default TypeSanitizer inlines the instrumentation code. This leads to increased
+binary size and compilation time. Using the clang flag
+``-fsanitize-type-outline-instrumentation`` (default: ``false``)
+forces all code instrumentation to be outlined. This reduces the size of the
+generated code and reduces compile-time overhead, but it also reduces runtime
+performance.
+
+This outlined instrumentation is new. If you wish to verify that the outlined instrumentation
+is behaving in the same way as the inline instrumentation, you can force TypeSanitizer
+to use both types of instrumentation. You can use the clang flag
+``-fsanitize-type-verify-outlined-instrumentation`` (default: ``false``) to do this.
+
 ``__has_feature(type_sanitizer)``
 ------------------------------------
 

clang/docs/UsersManual.rst

@@ -2277,6 +2277,15 @@ are listed below.
 
    See :doc: `AddressSanitizer` for more details.
 
+.. option:: -f[no-]sanitize-type-outline-instrumentation
+
+   Controls how type sanitizer code is generated. If enabled will always use
+   a function call instead of inlining the code. Turning this option on may
+   reduce the binary size and compilation overhead, but might result in a worse
+   run-time performance.
+
+   See :doc: `TypeSanitizer` for more details.
+
 .. option:: -f[no-]sanitize-stats
 
    Enable simple statistics gathering for the enabled sanitizers.

clang/include/clang/Driver/Options.td

@@ -2454,6 +2454,18 @@ def fsanitize_address_outline_instrumentation : Flag<["-"], "fsanitize-address-o
 def fno_sanitize_address_outline_instrumentation : Flag<["-"], "fno-sanitize-address-outline-instrumentation">,
                                                    Group<f_clang_Group>,
                                                    HelpText<"Use default code inlining logic for the address sanitizer">;
+def fsanitize_type_outline_instrumentation : Flag<["-"], "fsanitize-type-outline-instrumentation">,
+                                                Group<f_clang_Group>,
+                                                HelpText<"Always generate function calls for type sanitizer instrumentation">;
+def fno_sanitize_type_outline_instrumentation : Flag<["-"], "fno-sanitize-type-outline-instrumentation">,
+                                                   Group<f_clang_Group>,
+                                                   HelpText<"Use default code inlining logic for the type sanitizer">;
+def fsanitize_type_verify_outlined_instrumentation : Flag<["-"], "fsanitize_type_verify_outlined_instrumentation">,
+                                                   Group<f_clang_Group>,
+                                                   HelpText<"Use both inlined and outlined instrumentation for type sanitizer to verify equivilence">;
+def fno_sanitize_type_verify_outlined_instrumentation : Flag<["-"], "fno_sanitize_type_verify_outlined_instrumentation">,
+                                                   Group<f_clang_Group>,
+                                                   HelpText<"Don't use both inlined and outlined instrumentation for type sanitizer to verify equivilence">;
 defm sanitize_stable_abi
   : OptInCC1FFlag<"sanitize-stable-abi", "Stable  ", "Conventional ",
     "ABI instrumentation for sanitizer runtime. Default: Conventional">;

clang/include/clang/Driver/SanitizerArgs.h

@@ -67,6 +67,8 @@ class SanitizerArgs {
   bool TsanFuncEntryExit = true;
   bool TsanAtomics = true;
   bool MinimalRuntime = false;
+  bool TysanOutlineInstrumentation = false;
+  bool TysanVerifyOutlinedInstrumentation = false;
   // True if cross-dso CFI support if provided by the system (i.e. Android).
   bool ImplicitCfiRuntime = false;
   bool NeedsMemProfRt = false;

clang/lib/Driver/SanitizerArgs.cpp

@@ -1176,6 +1176,17 @@ SanitizerArgs::SanitizerArgs(const ToolChain &TC,
         options::OPT_fno_sanitize_alloc_token_extended, AllocTokenExtended);
   }
 
+  if (AllAddedKinds & SanitizerKind::Type) {
+    TysanOutlineInstrumentation =
+        Args.hasFlag(options::OPT_fsanitize_type_outline_instrumentation,
+                     options::OPT_fno_sanitize_type_outline_instrumentation,
+                     TysanOutlineInstrumentation);
+    TysanVerifyOutlinedInstrumentation = Args.hasFlag(
+        options::OPT_fsanitize_type_verify_outlined_instrumentation,
+        options::OPT_fno_sanitize_type_verify_outlined_instrumentation,
+        TysanVerifyOutlinedInstrumentation);
+  }
+
   LinkRuntimes = Args.hasFlag(options::OPT_fsanitize_link_runtime,
                               options::OPT_fno_sanitize_link_runtime,
                               !Args.hasArg(options::OPT_r));
@@ -1500,6 +1511,15 @@ void SanitizerArgs::addArgs(const ToolChain &TC, const llvm::opt::ArgList &Args,
     CmdArgs.push_back("-asan-instrumentation-with-call-threshold=0");
   }
 
+  if (TysanOutlineInstrumentation || TysanVerifyOutlinedInstrumentation) {
+    CmdArgs.push_back("-mllvm");
+    CmdArgs.push_back("-tysan-outline-instrumentation");
+  }
+  if (TysanVerifyOutlinedInstrumentation) {
+    CmdArgs.push_back("-mllvm");
+    CmdArgs.push_back("-tysan-verify-outlined-instrumentation");
+  }
+
   // When emitting Stable ABI instrumentation, force outlining calls and avoid
   // inlining shadow memory poisoning. While this is a big performance burden
   // for now it allows full abstraction from implementation details.

clang/test/CodeGen/sanitize-type-outlined.cpp

@@ -0,0 +1,23 @@
+// RUN: %clang -S -fsanitize=type -emit-llvm -o - -fsanitize=type %s \
+// RUN:     | FileCheck %s --check-prefixes=CHECK-NO-OUTLINE
+// RUN: %clang -S -fsanitize=type -emit-llvm -o - -fsanitize=type %s \
+// RUN:     -fsanitize-type-outline-instrumentation \
+// RUN:     | FileCheck %s --check-prefixes=CHECK-OUTLINE
+
+// RUN: %clang -S -fsanitize=type -emit-llvm -o - -fsanitize=type %s \
+// RUN:     -fsanitize-type-outline-instrumentation \
+// RUN:     -fsanitize-type-verify-outlined-instrumentation \
+// RUN:     | FileCheck %s --check-prefixes=CHECK-OUTLINE-VERIFY
+// RUN: %clang -S -fsanitize=type -emit-llvm -o - -fsanitize=type %s \
+// RUN:     -fsanitize-type-verify-outlined-instrumentation \
+// RUN:     | FileCheck %s --check-prefixes=CHECK-VERIFY
+
+// CHECK-NO-OUTLINE-NOT: call{{.*}}@__tysan_instrument_mem_inst
+// CHECK-OUTLINE: call{{.*}}@__tysan_instrument_mem_inst
+// CHECK-OUTLINE-VERIFY: call{{.*}}@__tysan_instrument_mem_inst
+// CHECK-VERIFY: call{{.*}}@__tysan_instrument_mem_inst
+
+float alias(int *ptr){
+    float *aliasedPtr = (float *)ptr;
+    return *aliasedPtr;
+}