[Hexagon] Bugfix in VarArg lowering: Special case for musl (#157564)

While debugging #145206 I found that a possible cause for the problem is
the call to printf, which is variadic.

In a musl environment VarArgs are treated like *non* VarArgs.
The handling of this special case was bypassed by the commit 
a4f8551.

The reason is that the arguement `TreatAsVarArg` is only set to `true`
in
an *non* musl env. `TreatAsVarArg` determines the result of
`isVarArg()`.
The `CCIfArgVarArg` class only checks each individual
variable, but not whether `isVarArg()` is true.

Without the special case, the unnamed arguments are always passed
on the stack, as is standard calling convention. But with musl
also unnamed arguments can be passed in registers.

Possibly, this fixes #145206.

Author: benedikt-l-huber

llvm/lib/Target/Hexagon/HexagonCallingConv.td

@@ -6,6 +6,15 @@
 //
 //===----------------------------------------------------------------------===//
 
+// We cannot use the standard CCIfArgVarArg class here since in Hexagon there
+// exists a special case for a musl environment.  In a musl environment VarArgs
+// are treated like non VarArgs.  I.e., in a musl enviroment unnamed arguments
+// can also be passed in registers.  The CCIfArgVarArg class only checks each
+// individual argument, but not whether State.isVarArg() is true.  We also have
+// to check State.isVarArg() which is determined by the TreatAsVarArg argument.
+class CCIfStateVarArgAndArgVarArg<CCAction A>
+  : CCIf<"State.isVarArg() && ArgFlags.isVarArg()", A>;
+
 def CC_HexagonStack: CallingConv<[
   CCIfType<[i32,v2i16,v4i8],
     CCAssignToStack<4,4>>,
@@ -23,7 +32,7 @@ def CC_Hexagon_Legacy: CallingConv<[
 
   CCIfByVal<
     CCPassByVal<8,8>>,
-  CCIfArgVarArg<
+  CCIfStateVarArgAndArgVarArg<
     CCDelegateTo<CC_HexagonStack>>,
 
   // Pass split values in pairs, allocate odd register if necessary.
@@ -53,7 +62,7 @@ def CC_Hexagon: CallingConv<[
 
   CCIfByVal<
     CCPassByVal<8,1>>,
-  CCIfArgVarArg<
+  CCIfStateVarArgAndArgVarArg<
     CCDelegateTo<CC_HexagonStack>>,
 
   // Pass split values in pairs, allocate odd register if necessary.

llvm/test/CodeGen/Hexagon/vararg-musl.ll

@@ -0,0 +1,15 @@
+; RUN: llc -mtriple=hexagon-unknown-linux-musl < %s | FileCheck %s -check-prefix=MUSL
+; RUN: llc -mtriple=hexagon-unknown-none-elf   < %s | FileCheck %s -check-prefix=NONMUSL
+
+; MUSL-NOT: memw
+; NONMUSL: memw
+
+declare i32 @f0(i32 %a0, ...)
+
+define i32 @f1(i32 %a0, i32 %a1) #0 {
+b1:
+  %v7 = call i32 (i32, ...) @f0(i32 %a0, i32 %a1)
+  ret i32 %v7
+}
+
+attributes #0 = { nounwind }