Skip to content

[clang][CodeGen][MSVC] Match how MSVC returns vector types from member functions #157365

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

[clang][CodeGen][MSVC] Match how MSVC returns vector types from member functions #157365

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
8f42b86
[clang][CodeGen][MSVC] Return vector types from methods indirectly
henrybw Sep 7, 2025
fd8dd19
amend! [clang][CodeGen][MSVC] Return vector types from methods indire…
henrybw Sep 8, 2025
5970180
fixup! [clang][CodeGen][MSVC] Return vector types from methods indire…
henrybw Sep 12, 2025
017bfea
fixup! [clang][CodeGen][MSVC] Return vector types from methods indire…
henrybw Sep 12, 2025
c0f4b22
amend! [clang][CodeGen][MSVC] Return vector types from methods indire…
henrybw Oct 3, 2025
c6b0cd3
fixup! [clang][CodeGen][MSVC] Return vector types from methods indire…
henrybw Oct 1, 2025
71cd7fd
fixup! [clang][CodeGen][MSVC] Return vector types from methods indire…
henrybw Oct 3, 2025
2964b82
fixup! [clang][CodeGen][MSVC] Return vector types from methods indire…
henrybw Oct 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions clang/docs/ReleaseNotes.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,9 @@ C++ Specific Potentially Breaking Changes
ABI Changes in This Version
---------------------------

- Fixed Microsoft calling convention for returning vector types from C++ member
functions. Such vector types should be returned indirectly. (GH#104)

AST Dumping Potentially Breaking Changes
----------------------------------------
- How nested name specifiers are dumped and printed changes, keeping track of clang AST changes.
Expand Down
23 changes: 14 additions & 9 deletions clang/lib/CodeGen/MicrosoftCXXABI.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1168,15 +1168,20 @@ static bool isTrivialForMSVC(const CXXRecordDecl *RD, QualType Ty,
}

bool MicrosoftCXXABI::classifyReturnType(CGFunctionInfo &FI) const {
const CXXRecordDecl *RD = FI.getReturnType()->getAsCXXRecordDecl();
if (!RD)
return false;

bool isTrivialForABI = RD->canPassInRegisters() &&
isTrivialForMSVC(RD, FI.getReturnType(), CGM);

// MSVC always returns structs indirectly from C++ instance methods.
bool isIndirectReturn = !isTrivialForABI || FI.isInstanceMethod();
bool isIndirectReturn = false;
if (const CXXRecordDecl *RD = FI.getReturnType()->getAsCXXRecordDecl()) {
bool isTrivialForABI = RD->canPassInRegisters() &&
isTrivialForMSVC(RD, FI.getReturnType(), CGM);

// MSVC always returns structs indirectly from C++ instance methods.
isIndirectReturn = !isTrivialForABI || FI.isInstanceMethod();
} else if (isa<VectorType>(FI.getReturnType())) {
// On x86, MSVC seems to only return vector types indirectly from non-
// vectorcall C++ instance methods.
isIndirectReturn =
CGM.getTarget().getTriple().isX86() && FI.isInstanceMethod() &&
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is worth expanding this out into if blocks. I think you can put the ABI ver check into the else if condition above, and combine the vector call check with a size check for > 64 bits to be ABI compatible with __vectorcall __m64. Please add a test for that as well.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

FI.getCallingConvention() != llvm::CallingConv::X86_VectorCall;
}

if (isIndirectReturn) {
CharUnits Align = CGM.getContext().getTypeAlignInChars(FI.getReturnType());
Expand Down
36 changes: 36 additions & 0 deletions clang/test/CodeGenCXX/microsoft-abi-vector-types.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
// RUN: %clang_cc1 -ffreestanding -emit-llvm %s -o - -triple=i686-pc-windows-msvc | FileCheck %s
// RUN: %clang_cc1 -ffreestanding -emit-llvm %s -o - -triple=x86_64-pc-windows-msvc | FileCheck %s

// To match the MSVC ABI, vector types must be returned indirectly from member
// functions (as long as they do not use the vectorcall calling convention),
// but must be returned directly everywhere else.

#include <xmmintrin.h>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please extend this test to cover arm64. I fed it to an AI tool and it did the straightforward rewrite, which I think you can adapt with some ifdefs to make this a single test case:
https://godbolt.org/z/xWon6163j

This demonstrates that ARM64 passes vectors directly. Honestly, I found that surprising, because if you look into arm(64)_neon.h, you see that MSVC defines vectors as "intrinsic_type" structs, so it makes sense that "records" go down the path of "instance methods return structs indirectly".

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. (I was also surprised by MSVC returning vectors directly on ARM64, but that's also why I constrained this behavior to only apply to x86/x86-64 targets.)


struct Foo {
__m128 method_m128();
__m128 __vectorcall vectorcall_method_m128();
};

__m128 Foo::method_m128() {
return __m128{};
// GH104
// CHECK: store <4 x float>
// CHECK: ret void
}

__m128 __vectorcall Foo::vectorcall_method_m128() {
return __m128{};
// CHECK: ret <4 x float>
}

__m128 func_m128() {
return __m128{};
// CHECK: ret <4 x float>
}

__m128 __vectorcall vectorcall_func_m128() {
return __m128{};
// CHECK: ret <4 x float>
}