Automerge: [UBSan][BoundsSafety] Implement support for more expressive "trap reasons" (#154618)

In 29992cf (#145967) support was added
for "trap reasons" on traps emitted in UBSan in trapping mode (e.g.
`-fsanitize-trap=undefined`). This improved the debugging experience by
attaching the reason for trapping as a string on the debug info on trap
instructions. Consumers such as LLDB can display this trap reason string
when the trap is reached.

A limitation of that patch is that the trap reason string is hard-coded
for each `SanitizerKind` even though the compiler actually has much more
information about the trap available at compile time that could be shown
to the user.

This patch is an incremental step in fixing that. It consists of two
main steps.

**1. Introduce infrastructure for building trap reason strings**

To make it convenient to construct trap reason strings this patch
re-uses Clang's powerful diagnostic infrastructure to provide a
convenient API for constructing trap reason strings. This is achieved
by:

* Introducing a new `Trap` diagnostic kind to represent trap diagnostics
in TableGen files.
* Adding a new `Trap` diagnostic component. While this part probably
isn't technically necessary it seemed like I should follow the existing
convention used by the diagnostic system.
* Adding `DiagnosticTrapKinds.td` to describe the different trap
reasons.
* Add the `TrapReasonBuilder` and `TrapReason` classes to provide an
interface for constructing trap reason strings and the trap category.
Note this API while similar to `DiagnosticBuilder` has different
semantics which are described in the code comments. In particular the
behavior when the destructor is called is very different.
* Adding `CodeGenModule::BuildTrapReason()` as a convenient constructor
for the `TrapReasonBuilder`.

This use of the diagnostic system is a little unusual in that the
emitted trap diagnostics aren't actually consumed by normal diagnostic
consumers (e.g. the console). Instead the `TrapReasonBuilder` is just
used to format a string, so in effect the builder is somewhat analagous
to "printf". However, re-using the diagnostics system in this way brings
a several benefits:

* The powerful diagnostic templating languge (e.g. `%select`) can be
used.
* Formatting Clang data types (e.g. `Type`, `Expr`, etc.) just work
out-of-the-box.
* Describing trap reasons in tablegen files opens the door for
translation to different languages in the future.
* The `TrapReasonBuilder` API is very similar to `DiagnosticBuilder`
which makes it easy to use by anyone already familiar with Clang's
diagnostic system.

While UBSan is the first consumer of this new infrastructure the intent
is to use this to overhaul how trap reasons are implemented in the
`-fbounds-safety` implementation (currently exists downstream).

**2. Apply the new infrastructure to UBSan checks for arithmetic
overflow**

To demonstrate using `TrapReasonBuilder` this patch applies it to UBSan
traps for arithmetic overflow. The intention is that we would
iteratively switch to using the `TrapReasonBuilder` for all UBSan traps
where it makes sense in future patches.

Previously for code like

```
int test(int a, int b) { return a + b; }
```

The trap reason string looked like

```
Undefined Behavior Sanitizer: Integer addition overflowed
```

now the trap message looks like:

```
Undefined Behavior Sanitizer: signed integer addition overflow in 'a + b'
```

This string is much more specific because

* It explains if signed or unsigned overflow occurred
* It actually shows the expression that overflowed

One possible downside of this approach is it may blow up Debug info size
because now there can be many more distinct trap reason strings. To
allow users to avoid this a new driver/cc1 flag
`-fsanitize-debug-trap-reasons=` has been added which can either be
`none` (disable trap reasons entirely), `basic` (use the per
`SanitizerKind` hard coded strings), and `detailed` (use the new
expressive trap reasons implemented in this patch). The default is
`detailed` to give the best out-of-the-box debugging experience. The
existing `-fsanitize-debug-trap-reasons` and
`-fno-sanitize-debug-trap-reasons` have been kept for compatibility and
are aliases of the new flag with `detailed` and `none` arguments passed
respectively.

rdar://158612755

Author: delcypher

clang/docs/ReleaseNotes.rst

@@ -141,14 +141,27 @@ Non-comprehensive list of changes in this release
 
 - Added ``__builtin_elementwise_minnumnum`` and ``__builtin_elementwise_maxnumnum``.
 
-- Trapping UBSan (e.g. ``-fsanitize-trap=undefined``) now emits a string describing the reason for
-  trapping into the generated debug info. This feature allows debuggers (e.g. LLDB) to display
-  the reason for trapping if the trap is reached. The string is currently encoded in the debug
-  info as an artificial frame that claims to be inlined at the trap location. The function used
-  for the artificial frame is an artificial function whose name encodes the reason for trapping.
-  The encoding used is currently the same as ``__builtin_verbose_trap`` but might change in the future.
-  This feature is enabled by default but can be disabled by compiling with
-  ``-fno-sanitize-annotate-debug-info-traps``.
+- Trapping UBSan (e.g. ``-fsanitize=undefined -fsanitize-trap=undefined``) now
+  emits a string describing the reason for trapping into the generated debug
+  info. This feature allows debuggers (e.g. LLDB) to display the reason for
+  trapping if the trap is reached. The string is currently encoded in the debug
+  info as an artificial frame that claims to be inlined at the trap location.
+  The function used for the artificial frame is an artificial function whose
+  name encodes the reason for trapping. The encoding used is currently the same
+  as ``__builtin_verbose_trap`` but might change in the future. This feature is
+  enabled by default but can be disabled by compiling with
+  ``-fno-sanitize-debug-trap-reasons``. The feature has a ``basic`` and
+  ``detailed`` mode (the default). The ``basic`` mode emits a hard-coded string
+  per trap kind (e.g. ``Integer addition overflowed``) and the ``detailed`` mode
+  emits a more descriptive string describing each individual trap (e.g. ``signed
+  integer addition overflow in 'a + b'``). The ``detailed`` mode produces larger
+  debug info than ``basic`` but is more helpful for debugging. The
+  ``-fsanitize-debug-trap-reasons=`` flag can be used to switch between the
+  different modes or disable the feature entirely. Note due to trap merging in
+  optimized builds (i.e. in each function all traps of the same kind get merged
+  into the same trap instruction) the trap reasons might be removed. To prevent
+  this build without optimizations (i.e. use `-O0` or use the `optnone` function
+  attribute) or use the `fno-sanitize-merge=` flag in optimized builds.
 
 - ``__builtin_elementwise_max`` and ``__builtin_elementwise_min`` functions for integer types can
   now be used in constant expressions.
@@ -185,7 +198,9 @@ Non-comprehensive list of changes in this release
 
 New Compiler Flags
 ------------------
-- New option ``-fno-sanitize-annotate-debug-info-traps`` added to disable emitting trap reasons into the debug info when compiling with trapping UBSan (e.g. ``-fsanitize-trap=undefined``).
+- New option ``-fno-sanitize-debug-trap-reasons`` added to disable emitting trap reasons into the debug info when compiling with trapping UBSan (e.g. ``-fsanitize-trap=undefined``).
+- New option ``-fsanitize-debug-trap-reasons=`` added to control emitting trap reasons into the debug info when compiling with trapping UBSan (e.g. ``-fsanitize-trap=undefined``).
+
 
 Lanai Support
 ^^^^^^^^^^^^^^

clang/include/clang/Basic/AllDiagnosticKinds.inc

@@ -30,4 +30,5 @@
 #include "clang/Basic/DiagnosticAnalysisKinds.inc"
 #include "clang/Basic/DiagnosticRefactoringKinds.inc"
 #include "clang/Basic/DiagnosticInstallAPIKinds.inc"
+#include "clang/Basic/DiagnosticTrapKinds.inc"
 // clang-format on

clang/include/clang/Basic/AllDiagnostics.h

@@ -26,6 +26,7 @@
 #include "clang/Basic/DiagnosticRefactoring.h"
 #include "clang/Basic/DiagnosticSema.h"
 #include "clang/Basic/DiagnosticSerialization.h"
+#include "clang/Basic/DiagnosticTrap.h"
 
 namespace clang {
 template <size_t SizeOfStr, typename FieldType> class StringSizerHelper {

clang/include/clang/Basic/CMakeLists.txt

@@ -33,6 +33,7 @@ clang_diag_gen(Parse)
 clang_diag_gen(Refactoring)
 clang_diag_gen(Sema)
 clang_diag_gen(Serialization)
+clang_diag_gen(Trap)
 clang_tablegen(DiagnosticGroups.inc -gen-clang-diag-groups
   SOURCE Diagnostic.td
   TARGET ClangDiagnosticGroups)

clang/include/clang/Basic/CodeGenOptions.def

@@ -307,7 +307,7 @@ CODEGENOPT(SanitizeBinaryMetadataAtomics, 1, 0, Benign) ///< Emit PCs for atomic
 CODEGENOPT(SanitizeBinaryMetadataUAR, 1, 0, Benign) ///< Emit PCs for start of functions
                                                     ///< that are subject for use-after-return checking.
 CODEGENOPT(SanitizeStats     , 1, 0, Benign) ///< Collect statistics for sanitizers.
-CODEGENOPT(SanitizeDebugTrapReasons, 1, 1 , Benign) ///< Enable UBSan trapping messages
+ENUM_CODEGENOPT(SanitizeDebugTrapReasons, SanitizeDebugTrapReasonKind, 2, SanitizeDebugTrapReasonKind::Detailed, Benign) ///< Control how "trap reasons" are emitted in debug info
 CODEGENOPT(SimplifyLibCalls  , 1, 1, Benign) ///< Set when -fbuiltin is enabled.
 CODEGENOPT(SoftFloat         , 1, 0, Benign) ///< -soft-float.
 CODEGENOPT(SpeculativeLoadHardening, 1, 0, Benign) ///< Enable speculative load hardening.

clang/include/clang/Basic/CodeGenOptions.h

@@ -198,6 +198,16 @@ class CodeGenOptions : public CodeGenOptionsBase {
     Forced,
   };
 
+  enum SanitizeDebugTrapReasonKind {
+    None,  ///< Trap Messages are omitted. This offers the smallest debug info
+           ///< size but at the cost of making traps hard to debug.
+    Basic, ///< Trap Message is fixed per SanitizerKind. Produces smaller debug
+           ///< info than `Detailed` but is not as helpful for debugging.
+    Detailed, ///< Trap Message includes more context (e.g. the expression being
+              ///< overflowed). This is more helpful for debugging but produces
+              ///< larger debug info than `Basic`.
+  };
+
   /// The code model to use (-mcmodel).
   std::string CodeModel;
 

clang/include/clang/Basic/Diagnostic.h

@@ -23,6 +23,7 @@
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/FunctionExtras.h"
 #include "llvm/ADT/IntrusiveRefCntPtr.h"
+#include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/iterator_range.h"
 #include "llvm/Support/Compiler.h"
@@ -1259,10 +1260,13 @@ class DiagnosticBuilder : public StreamingDiagnostic {
 
   DiagnosticBuilder() = default;
 
+protected:
   DiagnosticBuilder(DiagnosticsEngine *DiagObj, SourceLocation DiagLoc,
                     unsigned DiagID);
 
-protected:
+  DiagnosticsEngine *getDiagnosticsEngine() const { return DiagObj; }
+  unsigned getDiagID() const { return DiagID; }
+
   /// Clear out the current diagnostic.
   void Clear() const {
     DiagObj = nullptr;

clang/include/clang/Basic/Diagnostic.td

@@ -30,6 +30,7 @@ def CLASS_REMARK    : DiagClass;
 def CLASS_WARNING   : DiagClass;
 def CLASS_EXTENSION : DiagClass;
 def CLASS_ERROR     : DiagClass;
+def CLASS_TRAP      : DiagClass;
 
 // Responses to a diagnostic in a SFINAE context.
 class SFINAEResponse;
@@ -144,7 +145,8 @@ class Extension<string str> : Diagnostic<str, CLASS_EXTENSION, SEV_Ignored>;
 class ExtWarn<string str>   : Diagnostic<str, CLASS_EXTENSION, SEV_Warning>;
 // Notes can provide supplementary information on errors, warnings, and remarks.
 class Note<string str>      : Diagnostic<str, CLASS_NOTE, SEV_Fatal/*ignored*/>;
-
+// Trap messages attached to traps in debug info.
+class Trap<string str>      : Diagnostic<str, CLASS_TRAP, SEV_Fatal/*ignored*/>;
 
 class DefaultIgnore { Severity DefaultSeverity = SEV_Ignored; }
 class DefaultWarn   { Severity DefaultSeverity = SEV_Warning; }
@@ -235,3 +237,4 @@ include "DiagnosticParseKinds.td"
 include "DiagnosticRefactoringKinds.td"
 include "DiagnosticSemaKinds.td"
 include "DiagnosticSerializationKinds.td"
+include "DiagnosticTrapKinds.td"

clang/include/clang/Basic/DiagnosticIDs.h

@@ -47,6 +47,7 @@ enum {
   DIAG_SIZE_ANALYSIS = 100,
   DIAG_SIZE_REFACTORING = 1000,
   DIAG_SIZE_INSTALLAPI = 100,
+  DIAG_SIZE_TRAP = 100,
 };
 // Start position for diagnostics.
 // clang-format off
@@ -64,7 +65,8 @@ enum {
   DIAG_START_ANALYSIS      = DIAG_START_SEMA          + static_cast<int>(DIAG_SIZE_SEMA),
   DIAG_START_REFACTORING   = DIAG_START_ANALYSIS      + static_cast<int>(DIAG_SIZE_ANALYSIS),
   DIAG_START_INSTALLAPI    = DIAG_START_REFACTORING   + static_cast<int>(DIAG_SIZE_REFACTORING),
-  DIAG_UPPER_LIMIT         = DIAG_START_INSTALLAPI    + static_cast<int>(DIAG_SIZE_INSTALLAPI)
+  DIAG_START_TRAP          = DIAG_START_INSTALLAPI    + static_cast<int>(DIAG_SIZE_INSTALLAPI),
+  DIAG_UPPER_LIMIT         = DIAG_START_TRAP          + static_cast<int>(DIAG_SIZE_TRAP)
 };
 // clang-format on
 
@@ -189,7 +191,8 @@ class DiagnosticIDs : public RefCountedBase<DiagnosticIDs> {
     CLASS_REMARK = 0x02,
     CLASS_WARNING = 0x03,
     CLASS_EXTENSION = 0x04,
-    CLASS_ERROR = 0x05
+    CLASS_ERROR = 0x05,
+    CLASS_TRAP = 0x06
   };
 
   static bool IsCustomDiag(diag::kind Diag) {
@@ -363,6 +366,10 @@ class DiagnosticIDs : public RefCountedBase<DiagnosticIDs> {
   ///
   bool isExtensionDiag(unsigned DiagID, bool &EnabledByDefault) const;
 
+  bool isTrapDiag(unsigned DiagID) const {
+    return getDiagClass(DiagID) == CLASS_TRAP;
+  }
+
   /// Given a group ID, returns the flag that toggles the group.
   /// For example, for Group::DeprecatedDeclarations, returns
   /// "deprecated-declarations".

clang/include/clang/Basic/DiagnosticTrap.h

@@ -0,0 +1,14 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+#ifndef LLVM_CLANG_BASIC_DIAGNOSTICTRAP_H
+#define LLVM_CLANG_BASIC_DIAGNOSTICTRAP_H
+
+#include "clang/Basic/Diagnostic.h"
+#include "clang/Basic/DiagnosticTrapInterface.inc"
+
+#endif