RuntimeLibcalls: Add call entries for sincos sleef and armpl libcalls

[arsenm]

These are the tested set of libcalls used for codegen of llvm.sincos
and are needed to get the legalization to follow standard procedure.

[arsenm]

• PowerPC: Add vec_malloc functions to AIX in RuntimeLibcalls #167089
• XCore: Add iprintf to RuntimeLibcalls system library #167088
• RuntimeLibcalls: Add small_printf functions to emscripten #167087
• RuntimeLibcalls: Add macos unlocked IO functions to systems #167084
• RuntimeLibcalls: Add memset_pattern* calls to darwin systems #167083
• RuntimeLibcalls: Add more function entries from TargetLibraryInfo #167082
• RuntimeLibcalls: Add malloc and free entries #167081
• RuntimeLibcalls: Add mustprogress to common function attributes #167080
• RuntimeLibcalls: Add __memcpy_chk, __memmove_chk, __memset_chk #167053
• RuntimeLibcalls: Add a few libm entries from TargetLibraryInfo #167049
• RuntimeLibcalls: Add definitions for vector math functions #167026
• DAG: Move expandMultipleResultFPLibCall to TargetLowering (NFC) #166988
• DAG: Stop using TargetLibraryInfo for multi-result FP intrinsic codegen #166987
• DAG: Use modf vector libcalls through RuntimeLibcalls #166986
• RuntimeLibcalls: Add libcall entries for sleef and armpl modf functions #166985
• DAG: Use sincos vector libcalls through RuntimeLibcalls #166984
• RuntimeLibcalls: Add call entries for sincos sleef and armpl libcalls #166983 👈 (View in Graphite)
• RuntimeLibcalls: Remove incorrect sincospi from most targets #166982
• RuntimeLibcalls: Add entries for vector sincospi functions #166981
• TargetLibraryInfo: Split off VectorLibrary enum and flag #166980
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[llvmbot]

@llvm/pr-subscribers-llvm-ir

@llvm/pr-subscribers-llvm-transforms

Author: Matt Arsenault (arsenm)

Changes

These are the tested set of libcalls used for codegen of llvm.sincos
and are needed to get the legalization to follow standard procedure.

---

Full diff: https://github.com/llvm/llvm-project/pull/166983.diff

4 Files Affected:

• (modified) llvm/include/llvm/IR/RuntimeLibcalls.td (+13)
• (modified) llvm/lib/IR/RuntimeLibcalls.cpp (+33-4)
• (modified) llvm/test/Transforms/Util/DeclareRuntimeLibcalls/armpl.ll (+11-2)
• (modified) llvm/test/Transforms/Util/DeclareRuntimeLibcalls/sleef.ll (+11-3)

diff --git a/llvm/include/llvm/IR/RuntimeLibcalls.td b/llvm/include/llvm/IR/RuntimeLibcalls.td
index a0b52395498c5..ba096414d1802 100644
--- a/llvm/include/llvm/IR/RuntimeLibcalls.td
+++ b/llvm/include/llvm/IR/RuntimeLibcalls.td
@@ -183,6 +183,7 @@ foreach FPTy = ["F32", "F64", "F80", "F128", "PPCF128"] in {
 }
 
 foreach VecTy = ["V4F32", "V2F64", "NXV4F32", "NXV2F64"] in {
+  def SINCOS_#VecTy : RuntimeLibcall;
   def SINCOSPI_#VecTy : RuntimeLibcall;
 }
 
@@ -1092,6 +1093,11 @@ def __security_check_cookie_arm64ec : RuntimeLibcallImpl<SECURITY_CHECK_COOKIE,
 //===----------------------------------------------------------------------===//
 
 defset list<RuntimeLibcallImpl> SleefLibcalls = {
+  def _ZGVnN2vl8l8_sincos : RuntimeLibcallImpl<SINCOS_V2F64>;
+  def _ZGVnN4vl4l4_sincosf : RuntimeLibcallImpl<SINCOS_V4F32>;
+  def _ZGVsNxvl8l8_sincos : RuntimeLibcallImpl<SINCOS_NXV2F64>;
+  def _ZGVsNxvl4l4_sincosf : RuntimeLibcallImpl<SINCOS_NXV4F32>;
+
   def _ZGVnN4vl4l4_sincospif : RuntimeLibcallImpl<SINCOSPI_V4F32>;
   def _ZGVnN2vl8l8_sincospi : RuntimeLibcallImpl<SINCOSPI_V2F64>;
   def _ZGVsNxvl4l4_sincospif : RuntimeLibcallImpl<SINCOSPI_NXV4F32>;
@@ -1103,6 +1109,13 @@ defset list<RuntimeLibcallImpl> SleefLibcalls = {
 //===----------------------------------------------------------------------===//
 
 defset list<RuntimeLibcallImpl> ARMPLLibcalls = {
+  def armpl_vsincosq_f64
+      : RuntimeLibcallImpl<SINCOS_V2F64>; // CallingConv::AArch64_VectorCall
+  def armpl_vsincosq_f32
+      : RuntimeLibcallImpl<SINCOS_V4F32>; // CallingConv::AArch64_VectorCall
+  def armpl_svsincos_f64_x : RuntimeLibcallImpl<SINCOS_NXV2F64>;
+  def armpl_svsincos_f32_x : RuntimeLibcallImpl<SINCOS_NXV4F32>;
+
   def armpl_vsincospiq_f32 : RuntimeLibcallImpl<SINCOSPI_V4F32>;
   def armpl_vsincospiq_f64 : RuntimeLibcallImpl<SINCOSPI_V2F64>;
   def armpl_svsincospi_f32_x : RuntimeLibcallImpl<SINCOSPI_NXV4F32>;
diff --git a/llvm/lib/IR/RuntimeLibcalls.cpp b/llvm/lib/IR/RuntimeLibcalls.cpp
index 3d4462ed7fcb2..b79c26efa16d9 100644
--- a/llvm/lib/IR/RuntimeLibcalls.cpp
+++ b/llvm/lib/IR/RuntimeLibcalls.cpp
@@ -43,17 +43,26 @@ RuntimeLibcallsInfo::RuntimeLibcallsInfo(const Triple &TT,
   switch (ClVectorLibrary) {
   case VectorLibrary::SLEEFGNUABI:
     for (RTLIB::LibcallImpl Impl :
-         {RTLIB::impl__ZGVnN4vl4l4_sincospif, RTLIB::impl__ZGVnN2vl8l8_sincospi,
+         {RTLIB::impl__ZGVnN2vl8l8_sincos, RTLIB::impl__ZGVnN4vl4l4_sincosf,
+          RTLIB::impl__ZGVsNxvl8l8_sincos, RTLIB::impl__ZGVsNxvl4l4_sincosf,
+          RTLIB::impl__ZGVnN4vl4l4_sincospif, RTLIB::impl__ZGVnN2vl8l8_sincospi,
           RTLIB::impl__ZGVsNxvl4l4_sincospif,
           RTLIB::impl__ZGVsNxvl8l8_sincospi})
       setAvailable(Impl);
     break;
   case VectorLibrary::ArmPL:
     for (RTLIB::LibcallImpl Impl :
-         {RTLIB::impl_armpl_vsincospiq_f32, RTLIB::impl_armpl_vsincospiq_f64,
+         {RTLIB::impl_armpl_vsincosq_f64, RTLIB::impl_armpl_vsincosq_f32,
+          RTLIB::impl_armpl_svsincos_f64_x, RTLIB::impl_armpl_svsincos_f32_x,
+          RTLIB::impl_armpl_vsincospiq_f32, RTLIB::impl_armpl_vsincospiq_f64,
           RTLIB::impl_armpl_svsincospi_f32_x,
           RTLIB::impl_armpl_svsincospi_f64_x})
       setAvailable(Impl);
+
+    for (RTLIB::LibcallImpl Impl :
+         {RTLIB::impl_armpl_vsincosq_f64, RTLIB::impl_armpl_vsincosq_f32})
+      setLibcallImplCallingConv(Impl, CallingConv::AArch64_VectorCall);
+
     break;
   default:
     break;
@@ -188,6 +197,14 @@ RuntimeLibcallsInfo::getFunctionTy(LLVMContext &Ctx, const Triple &TT,
                                                   fcNegNormal));
     return {FuncTy, Attrs};
   }
+  case RTLIB::impl__ZGVnN2vl8l8_sincos:
+  case RTLIB::impl__ZGVnN4vl4l4_sincosf:
+  case RTLIB::impl__ZGVsNxvl8l8_sincos:
+  case RTLIB::impl__ZGVsNxvl4l4_sincosf:
+  case RTLIB::impl_armpl_vsincosq_f64:
+  case RTLIB::impl_armpl_vsincosq_f32:
+  case RTLIB::impl_armpl_svsincos_f64_x:
+  case RTLIB::impl_armpl_svsincos_f32_x:
   case RTLIB::impl__ZGVnN4vl4l4_sincospif:
   case RTLIB::impl__ZGVnN2vl8l8_sincospi:
   case RTLIB::impl__ZGVsNxvl4l4_sincospif:
@@ -201,14 +218,24 @@ RuntimeLibcallsInfo::getFunctionTy(LLVMContext &Ctx, const Triple &TT,
     bool IsF32 = LibcallImpl == RTLIB::impl__ZGVnN4vl4l4_sincospif ||
                  LibcallImpl == RTLIB::impl__ZGVsNxvl4l4_sincospif ||
                  LibcallImpl == RTLIB::impl_armpl_vsincospiq_f32 ||
-                 LibcallImpl == RTLIB::impl_armpl_svsincospi_f32_x;
+                 LibcallImpl == RTLIB::impl_armpl_svsincospi_f32_x ||
+                 LibcallImpl == RTLIB::impl__ZGVnN4vl4l4_sincosf ||
+                 LibcallImpl == RTLIB::impl__ZGVsNxvl4l4_sincosf ||
+                 LibcallImpl == RTLIB::impl_armpl_vsincosq_f32 ||
+                 LibcallImpl == RTLIB::impl_armpl_svsincos_f32_x;
+
     Type *ScalarTy = IsF32 ? Type::getFloatTy(Ctx) : Type::getDoubleTy(Ctx);
     unsigned EC = IsF32 ? 4 : 2;
 
-    bool IsScalable = LibcallImpl == RTLIB::impl__ZGVsNxvl4l4_sincospif ||
+    bool IsScalable = LibcallImpl == RTLIB::impl__ZGVsNxvl8l8_sincos ||
+                      LibcallImpl == RTLIB::impl__ZGVsNxvl4l4_sincosf ||
+                      LibcallImpl == RTLIB::impl_armpl_svsincos_f32_x ||
+                      LibcallImpl == RTLIB::impl_armpl_svsincos_f64_x ||
+                      LibcallImpl == RTLIB::impl__ZGVsNxvl4l4_sincospif ||
                       LibcallImpl == RTLIB::impl__ZGVsNxvl8l8_sincospi ||
                       LibcallImpl == RTLIB::impl_armpl_svsincospi_f32_x ||
                       LibcallImpl == RTLIB::impl_armpl_svsincospi_f64_x;
+
     Type *VecTy =
         IsScalable ? static_cast<Type *>(ScalableVectorType::get(ScalarTy, EC))
                    : static_cast<Type *>(FixedVectorType::get(ScalarTy, EC));
@@ -247,6 +274,8 @@ bool RuntimeLibcallsInfo::hasVectorMaskArgument(RTLIB::LibcallImpl Impl) {
   /// FIXME: This should be generated by tablegen and support the argument at an
   /// arbitrary position
   switch (Impl) {
+  case RTLIB::impl_armpl_svsincos_f32_x:
+  case RTLIB::impl_armpl_svsincos_f64_x:
   case RTLIB::impl_armpl_svsincospi_f32_x:
   case RTLIB::impl_armpl_svsincospi_f64_x:
     return true;
diff --git a/llvm/test/Transforms/Util/DeclareRuntimeLibcalls/armpl.ll b/llvm/test/Transforms/Util/DeclareRuntimeLibcalls/armpl.ll
index c45f319f80122..1d9cf6a5d77fe 100644
--- a/llvm/test/Transforms/Util/DeclareRuntimeLibcalls/armpl.ll
+++ b/llvm/test/Transforms/Util/DeclareRuntimeLibcalls/armpl.ll
@@ -1,12 +1,21 @@
 ; REQUIRES: aarch64-registered-target
 ; RUN: opt -S -passes=declare-runtime-libcalls -mtriple=aarch64-unknown-linux -mattr=+neon,+sve -vector-library=ArmPL < %s | FileCheck %s
 
-; CHECK: declare void @armpl_svsincospi_f32_x(<vscale x 4 x float>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16, <vscale x 4 x i1>) [[ATTRS:#[0-9]+]]
+; CHECK: declare void @armpl_svsincos_f32_x(<vscale x 4 x float>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16, <vscale x 4 x i1>) [[ATTRS:#[0-9]+]]
 
-; CHECK: declare void @armpl_svsincospi_f64_x(<vscale x 2 x double>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16, <vscale x 2 x i1>) [[ATTRS:#[0-9]+]]
+; CHECK: declare void @armpl_svsincos_f64_x(<vscale x 2 x double>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16, <vscale x 2 x i1>) [[ATTRS]]
+
+; CHECK: declare void @armpl_svsincospi_f32_x(<vscale x 4 x float>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16, <vscale x 4 x i1>) [[ATTRS]]
+
+; CHECK: declare void @armpl_svsincospi_f64_x(<vscale x 2 x double>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16, <vscale x 2 x i1>) [[ATTRS]]
 
 ; CHECK: declare void @armpl_vsincospiq_f32(<4 x float>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16) [[ATTRS]]
 
 ; CHECK: declare void @armpl_vsincospiq_f64(<2 x double>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16) [[ATTRS]]
 
+; CHECK: declare aarch64_vector_pcs void @armpl_vsincosq_f32(<4 x float>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16) [[ATTRS]]
+
+; CHECK: declare aarch64_vector_pcs void @armpl_vsincosq_f64(<2 x double>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16) [[ATTRS]]
+
+
 ; CHECK: attributes [[ATTRS]] = { nocallback nofree nosync nounwind willreturn memory(argmem: write) }
diff --git a/llvm/test/Transforms/Util/DeclareRuntimeLibcalls/sleef.ll b/llvm/test/Transforms/Util/DeclareRuntimeLibcalls/sleef.ll
index 7972e0ca1c487..2c6900761b1c0 100644
--- a/llvm/test/Transforms/Util/DeclareRuntimeLibcalls/sleef.ll
+++ b/llvm/test/Transforms/Util/DeclareRuntimeLibcalls/sleef.ll
@@ -1,12 +1,20 @@
 ; REQUIRES: aarch64-registered-target
 ; RUN: opt -S -passes=declare-runtime-libcalls -mtriple=aarch64-unknown-linux -mattr=+neon,+sve -vector-library=sleefgnuabi < %s | FileCheck %s
 
-; CHECK: declare void @_ZGVnN2vl8l8_sincospi(<2 x double>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16) [[ATTRS:#[0-9]+]]
+; CHECK: declare void @_ZGVnN2vl8l8_sincos(<2 x double>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16) [[ATTRS:#[0-9]+]]
+
+; CHECK: declare void @_ZGVnN2vl8l8_sincospi(<2 x double>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16) [[ATTRS]]
+
+; CHECK: declare void @_ZGVnN4vl4l4_sincosf(<4 x float>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16) [[ATTRS]]
 
 ; CHECK: declare void @_ZGVnN4vl4l4_sincospif(<4 x float>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16) [[ATTRS]]
 
-; CHECK: declare void @_ZGVsNxvl4l4_sincospif(<vscale x 4 x float>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16) [[ATTRS:#[0-9]+]]
+; CHECK: declare void @_ZGVsNxvl4l4_sincosf(<vscale x 4 x float>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16) [[ATTRS]]
+
+; CHECK: declare void @_ZGVsNxvl4l4_sincospif(<vscale x 4 x float>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16) [[ATTRS]]
+
+; CHECK: declare void @_ZGVsNxvl8l8_sincos(<vscale x 2 x double>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16) [[ATTRS]]
 
-; CHECK: declare void @_ZGVsNxvl8l8_sincospi(<vscale x 2 x double>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16) [[ATTRS:#[0-9]+]]
+; CHECK: declare void @_ZGVsNxvl8l8_sincospi(<vscale x 2 x double>, ptr noalias nonnull writeonly align 16, ptr noalias nonnull writeonly align 16) [[ATTRS]]
 
 ; CHECK: attributes [[ATTRS]] = { nocallback nofree nosync nounwind willreturn memory(argmem: write) }