[clang] Pass fp128 indirectly and return in xmm0 on Windows

Clang currently passes and returns `__float128` in vector registers on
MinGW targets, which is LLVM's default ABI for `fp128`. However, the
Windows x86-64 calling convention [1] states the following:

    __m128 types, arrays, and strings are never passed by immediate
    value. Instead, a pointer is passed to memory allocated by the
    caller. Structs and unions of size 8, 16, 32, or 64 bits, and __m64
    types, are passed as if they were integers of the same size. Structs
    or unions of other sizes are passed as a pointer to memory allocated
    by the caller. For these aggregate types passed as a pointer,
    including __m128, the caller-allocated temporary memory must be
    16-byte aligned.

Based on the above it sounds like `__float128` should be passed
indirectly; this is what MinGW GCC already does, so change Clang to
match. Passing by value also causes problems with varargs. E.g. the
below completes successfully when built with GCC but has a runtime crash
when built with Clang:

    void va_f128(int count, ...) {
        va_list args;
        va_start(args, count);
        __float128 val = va_arg(args, __float128);
        va_end(args);
    }

    int main() {
        va_f128(0, 0.0);
    }

Change `f128` passing to use the stack and make the return in xmm0
explicit. This is the identical to `i128`, and passing is the same as
GCC.

Regarding return values, the documentation states:

    A scalar return value that can fit into 64 bits, including the __m64
    type, is returned through RAX. Non-scalar types including floats,
    doubles, and vector types such as __m128, __m128i, __m128d are
    returned in XMM0.

This makes it sound like it should be acceptable to return `__float128`
in xmm0; however, GCC returns `__float128` on the stack. That above ABI
statement as well as consistency with `i128` (which is returned in xmm0)
mean that it would likely be better for GCC to change its return ABI to
match Clang rather than the other way around, so that portion is left
as-is.

Clang's MSVC targets do not support `__float128` or `_Float128`, but
these changes would also apply there if it is eventually enabled.

[1]: https://learn.microsoft.com/en-us/cpp/build/x64-calling-convention?view=msvc-170

Author: tgross35

clang/lib/CodeGen/Targets/X86.cpp

@@ -3390,6 +3390,9 @@ ABIArgInfo WinX86_64ABIInfo::classify(QualType Ty, unsigned &FreeSSERegs,
 
     case BuiltinType::Int128:
     case BuiltinType::UInt128:
+    case BuiltinType::Float128:
+      // 128-bit float and integer types share the same ABI.
+      
       // If it's a parameter type, the normal ABI rule is that arguments larger
       // than 8 bytes are passed indirectly. GCC follows it. We follow it too,
       // even though it isn't particularly efficient.
@@ -3400,6 +3403,8 @@ ABIArgInfo WinX86_64ABIInfo::classify(QualType Ty, unsigned &FreeSSERegs,
 
       // Mingw64 GCC returns i128 in XMM0. Coerce to v2i64 to handle that.
       // Clang matches them for compatibility.
+      // NOTE: GCC actually returns f128 indirectly but will hopefully change.
+      // See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115054#c8.
       return ABIArgInfo::getDirect(llvm::FixedVectorType::get(
           llvm::Type::getInt64Ty(getVMContext()), 2));
 

clang/test/CodeGen/win64-fp128.c

@@ -3,17 +3,16 @@
 // __float128 is unsupported on MSVC
 
 __float128 fp128_ret(void) { return 0; }
-// GNU64: define dso_local fp128 @fp128_ret()
+// GNU64: define dso_local <2 x i64>  @fp128_ret()
 
 __float128 fp128_args(__float128 a, __float128 b) { return a * b; }
-// GNU64: define dso_local fp128 @fp128_args(fp128 noundef %a, fp128 noundef %b)
+// GNU64: define dso_local <2 x i64> @fp128_args(ptr noundef %0, ptr noundef %1)
 
 void fp128_vararg(int a, ...) {
   // GNU64-LABEL: define dso_local void @fp128_vararg
   __builtin_va_list ap;
   __builtin_va_start(ap, a);
   __float128 i = __builtin_va_arg(ap, __float128);
-  // movaps  xmm0, xmmword ptr [rax]
   // GNU64: load ptr, ptr
   // GNU64: load fp128, ptr
   __builtin_va_end(ap);