[TySan][Clang] Add clang flag to use tysan outlined instrumentation and update docs

Author: gbMattN

clang/docs/TypeSanitizer.rst

@@ -20,9 +20,13 @@ code is build with ``-fno-strict-aliasing``, sacrificing performance.
 TypeSanitizer is built to catch when these strict aliasing rules have been violated, helping 
 users find where such bugs originate in their code despite the code looking valid at first glance.
 
-As TypeSanitizer is still experimental, it can currently have a large impact on runtime speed, 
-memory use, and code size. It also has a large compile-time overhead. Work is being done to 
-reduce these impacts.
+Typical memory overhead introduced by TypeSanitizer is about **8x**. Runtime slowdown varies greatly 
+depending on how often the instrumented code relies on type aliasing. In the best case slowdown is 
+**2x-3x**.
+
+The compiler instrumentation also has an impact on code size and compilation overhead. There is an 
+experimental instrumentation outlining option which can greatly reduce this (see sanitizer options) 
+but this may decrease runtime performance.
 
 The TypeSanitizer Algorithm
 ===========================
@@ -128,6 +132,18 @@ references to LLVM IR specific terms.
 Sanitizer features
 ==================
 
+Instrumentation code outlining
+------------------------------
+
+By default TypeSanitizer inlines the instrumentation code. This leads to increased
+binary size and compilation time. 
+
+By default AddressSanitizer inlines the instrumentation code to improve the
+run-time performance, which leads to increased binary size. Using the
+(clang flag ``-fsanitize-address-outline-instrumentation`` default: ``false``)
+flag forces all code instrumentation to be outlined, which reduces the size
+of the generated code, but also reduces the run-time performance.
+
 ``__has_feature(type_sanitizer)``
 ------------------------------------
 

clang/docs/UsersManual.rst

@@ -2276,6 +2276,15 @@ are listed below.
    reduce the binary size, but might result in a worse run-time performance.
 
    See :doc: `AddressSanitizer` for more details.
+  
+.. option:: -f[no-]sanitize-type-outline-instrumentation
+
+   Controls how type sanitizer code is generated. If enabled will always use
+   a function call instead of inlining the code. Turning this option on may
+   reduce the binary size and compilation overhead, but might result in a worse 
+   run-time performance.
+
+   See :doc: `TypeSanitizer` for more details.
 
 .. option:: -f[no-]sanitize-stats
 

clang/include/clang/Driver/Options.td

@@ -2454,6 +2454,18 @@ def fsanitize_address_outline_instrumentation : Flag<["-"], "fsanitize-address-o
 def fno_sanitize_address_outline_instrumentation : Flag<["-"], "fno-sanitize-address-outline-instrumentation">,
                                                    Group<f_clang_Group>,
                                                    HelpText<"Use default code inlining logic for the address sanitizer">;
+def fsanitize_type_outline_instrumentation : Flag<["-"], "fsanitize-type-outline-instrumentation">,
+                                                Group<f_clang_Group>,
+                                                HelpText<"Always generate function calls for type sanitizer instrumentation">;
+def fno_sanitize_type_outline_instrumentation : Flag<["-"], "fno-sanitize-type-outline-instrumentation">,
+                                                   Group<f_clang_Group>,
+                                                   HelpText<"Use default code inlining logic for the type sanitizer">;
+def fsanitize_type_verify_outlined_instrumentation : Flag<["-"], "fsanitize_type_verify_outlined_instrumentation">,
+                                                   Group<f_clang_Group>,
+                                                   HelpText<"Use both inlined and outlined instrumentation for type sanitizer to verify equivilence">;
+def fno_sanitize_type_verify_outlined_instrumentation : Flag<["-"], "fno_sanitize_type_verify_outlined_instrumentation">,
+                                                   Group<f_clang_Group>,
+                                                   HelpText<"Don't use both inlined and outlined instrumentation for type sanitizer to verify equivilence">;
 defm sanitize_stable_abi
   : OptInCC1FFlag<"sanitize-stable-abi", "Stable  ", "Conventional ",
     "ABI instrumentation for sanitizer runtime. Default: Conventional">;

clang/include/clang/Driver/SanitizerArgs.h

@@ -67,6 +67,8 @@ class SanitizerArgs {
   bool TsanFuncEntryExit = true;
   bool TsanAtomics = true;
   bool MinimalRuntime = false;
+  bool TysanOutlineInstrumentation = false;
+  bool TysanVerifyOutlinedInstrumentation = false;
   // True if cross-dso CFI support if provided by the system (i.e. Android).
   bool ImplicitCfiRuntime = false;
   bool NeedsMemProfRt = false;

clang/lib/Driver/SanitizerArgs.cpp

@@ -1176,6 +1176,17 @@ SanitizerArgs::SanitizerArgs(const ToolChain &TC,
         options::OPT_fno_sanitize_alloc_token_extended, AllocTokenExtended);
   }
 
+  if(AllAddedKinds & SanitizerKind::Type) {
+    TysanOutlineInstrumentation =
+        Args.hasFlag(options::OPT_fsanitize_type_outline_instrumentation,
+                     options::OPT_fno_sanitize_type_outline_instrumentation,
+                     TysanOutlineInstrumentation);
+    TysanVerifyOutlinedInstrumentation =
+        Args.hasFlag(options::OPT_fsanitize_type_verify_outlined_instrumentation,
+                     options::OPT_fno_sanitize_type_verify_outlined_instrumentation,
+                     TysanVerifyOutlinedInstrumentation);
+  }
+
   LinkRuntimes = Args.hasFlag(options::OPT_fsanitize_link_runtime,
                               options::OPT_fno_sanitize_link_runtime,
                               !Args.hasArg(options::OPT_r));
@@ -1500,6 +1511,15 @@ void SanitizerArgs::addArgs(const ToolChain &TC, const llvm::opt::ArgList &Args,
     CmdArgs.push_back("-asan-instrumentation-with-call-threshold=0");
   }
 
+  if (TysanOutlineInstrumentation || TysanVerifyOutlinedInstrumentation) {
+    CmdArgs.push_back("-mllvm");
+    CmdArgs.push_back("-tysan-outline-instrumentation");
+  }
+  if (TysanVerifyOutlinedInstrumentation) {
+    CmdArgs.push_back("-mllvm");
+    CmdArgs.push_back("-tysan-verify-outlined-instrumentation");
+  }
+
   // When emitting Stable ABI instrumentation, force outlining calls and avoid
   // inlining shadow memory poisoning. While this is a big performance burden
   // for now it allows full abstraction from implementation details.

clang/test/CodeGen/sanitize-type-outlined.cpp

@@ -0,0 +1,13 @@
+// RUN: %clang -S -fsanitize=type -emit-llvm -o - -fsanitize=type %s \
+// RUN:     | FileCheck %s --check-prefixes=CHECK-NO-OUTLINE
+// RUN: %clang -S -fsanitize=type -emit-llvm -o - -fsanitize=type %s \
+// RUN:     -fsanitize-type-outline-instrumentation \
+// RUN:     | FileCheck %s --check-prefixes=CHECK-OUTLINE
+
+// CHECK-NO-OUTLINE-NOT: call{{.*}}@__tysan_instrument_mem_inst
+// CHECK-OUTLINE: call{{.*}}@__tysan_instrument_mem_inst
+
+float alias(int *ptr){
+    float *aliasedPtr = (float *)ptr;
+    return *aliasedPtr;
+}