[Intrinsics] Add @llvm.dereferenceable intrinsic.

This patch adds a @llvm.dereferenceable intrinsic that can be used to
mark pointers as dereferenceable at the point the intrinsic is called.
The semantics match the meaning of the dereferenceable function
argument attribute.

The goal of the intrinsic is the preserve dereferenceability information
after moving memory instructions. This allows us to vectorize cases
such as https://clang.godbolt.org/z/Y1bedbhs3, where we currently fail
due to instcombine sinking the load into then/else blocks.

Alternatively we could use an assume bundle with a dereferenceable
attribute. But as a follow-up I would like to allow non-immediate size
arguments, and this may not mesh well with the attribute.

Author: fhahn

llvm/docs/LangRef.rst

@@ -29153,6 +29153,45 @@ attach various forms of information to operands that dominate specific
 uses.  It is not meant for general use, only for building temporary
 renaming forms that require value splits at certain points.
 
+.. _int_dereferenceable:
+
+'``llvm.dereferenceable``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare void @llvm.dereferenceable(ptr %p, i64 <size>)
+
+Overview:
+"""""""""
+
+The ``llvm.dereferenceable`` allows the optimizer to assume that the returned
+pointer is dereferenceable at the point the intrinsic is called.
+
+Arguments:
+""""""""""
+
+The arguments of the call are the pointer which will be marked as
+dereferenceable and the number of bytes known to be dereferenceable. ``<size>``
+must be a constant.
+
+Semantics:
+""""""""""
+
+The intrinsic returns the input pointer. The returned pointer is dereferenceable
+at the point the intrinsic is called. A pointer that is dereferenceable can be
+loaded from speculatively without a risk of trapping. This implies that the
+input pointer is not null and neither undef or poison. The number of bytes known
+to be dereferenceable is provided as second argument. It is legal for the number
+of bytes to be less than the size of the pointee type.
+
+The semantics above match the semantics of the ``dereferenceable(<n>)``
+parameter attribute.
+
+
 .. _type.test:
 
 '``llvm.type.test``' Intrinsic

llvm/include/llvm/IR/Intrinsics.td

@@ -954,6 +954,10 @@ def int_call_preallocated_teardown : DefaultAttrsIntrinsic<[], [llvm_token_ty]>;
 def int_callbr_landingpad : Intrinsic<[llvm_any_ty], [LLVMMatchType<0>],
                                       [IntrNoMerge]>;
 
+// Attach dereferenceability information to a pointer.
+def int_dereferenceable: DefaultAttrsIntrinsic<[llvm_ptr_ty], [llvm_ptr_ty, llvm_i64_ty],
+                             [IntrInaccessibleMemOnly, ImmArg<ArgIndex<1>>]>;
+
 //===------------------- Standard C Library Intrinsics --------------------===//
 //
 

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

@@ -8293,6 +8293,9 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
     visitVectorExtractLastActive(I, Intrinsic);
     return;
   }
+  case Intrinsic::dereferenceable:
+    setValue(&I, getValue(I.getArgOperand(0)));
+    return;
   }
 }
 

llvm/test/CodeGen/AArch64/dereferenceable-intrinsics.ll

@@ -0,0 +1,14 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc -mtriple=arm64-apple-macosx -o - %s | FileCheck %s
+
+declare ptr @llvm.dereferenceable(ptr, i64 immarg)
+
+define i64 @foo(ptr %a) {
+; CHECK-LABEL: foo:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    ldr x0, [x0]
+; CHECK-NEXT:    ret
+  %d = call ptr @llvm.dereferenceable(ptr %a, i64 4)
+  %l = load i64, ptr %d
+  ret i64 %l
+}

llvm/test/Verifier/dereferenceable-intrinsics.ll

@@ -0,0 +1,11 @@
+; RUN: not llvm-as < %s -o /dev/null 2>&1 | FileCheck %s
+
+declare ptr @llvm.dereferenceable(ptr, i64 immarg)
+
+define void @transpose(ptr %p, i64 %x) {
+; CHECK: immarg operand has non-immediate parameter
+  %d.0 = call ptr @llvm.dereferenceable(ptr %p, i64 4)
+  %d.1 = call ptr @llvm.dereferenceable(ptr %p, i64 %x)
+  ret void
+}
+