Skip to content

[AArch64] Generalize integer FPR lane stores for all types #134117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Apr 17, 2025
Merged

[AArch64] Generalize integer FPR lane stores for all types #134117

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
24cbbb3
[AArch64] Make use of byte FPR stores bytes extracted from vectors
MacDue Mar 4, 2025
bc61bd5
WIP: Attempt vector truncstore
MacDue Apr 2, 2025
7842bc4
Prefer st1.b in some cases
MacDue Apr 2, 2025
f6466ce
Avoid NOP movs
MacDue Apr 3, 2025
7abb342
Add note
MacDue Apr 3, 2025
36d54fe
Fixups
MacDue Apr 4, 2025
6857792
Generalize fold
MacDue Apr 8, 2025
9ce811e
Fixups
MacDue Apr 11, 2025
87e277f
Prefer scalar mov form
MacDue Apr 11, 2025
2f39546
Remove SVE FIXME
MacDue Apr 11, 2025
33346f8
Restore zero store behaviour
MacDue Apr 11, 2025
ab5439a
Move check
MacDue Apr 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24025,6 +24025,13 @@ static SDValue performSTORECombine(SDNode *N,
EVT VectorVT = Vector.getValueType();
EVT ElemVT = VectorVT.getVectorElementType();

// Propagate zero constants (applying this fold may miss optimizations).
if (ISD::isConstantSplatVectorAllZeros(Vector.getNode())) {
SDValue ZeroElt = DAG.getConstant(0, DL, ValueVT);
DAG.ReplaceAllUsesWith(Value, ZeroElt);
return SDValue();
}
Copy link
Collaborator

@paulwalker-arm paulwalker-arm Apr 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say we're missing an obvious combine and looking at DAGCombiner::visitBITCAST I see

TODO: Support FP bitcasts after legalize types

It's not worth looking for trouble so I'll take the current fix and we can circle back later if necessary. That said, DAG.getConstant is specific to integers so the code needs moving after the !ValueVT.isInteger() bailout.

Copy link
Member Author

@MacDue MacDue Apr 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 I added this fold:

+  // extract_vector_elt of undef index -> UNDEF
+  if (Index.isUndef())
+    return DAG.getUNDEF(ScalarVT);
+
+  // extract_vector_elt of zero splat -> zero
+  if (ISD::isConstantSplatVectorAllZeros(VecOp.getNode()))
+    return ScalarVT.isFloatingPoint() ? DAG.getConstantFP(0.0, DL, ScalarVT)
+                                      : DAG.getConstant(0, DL, ScalarVT);
+

to visitEXTRACT_VECTOR_ELT, which also seemed to resolve this (and a few more cases). But I can maybe post that in a later patch (since it results in changes across a few targets).

Copy link
Collaborator

@paulwalker-arm paulwalker-arm Apr 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, definitely future work. It's worth looking at visitBITCAST first though because I think it only affects floating point build_vectors that are bit casted to integer. When the "redundant" bit casting is removed I believe existing combines will then take over.


if (!ValueVT.isInteger())
return SDValue();
if (ValueVT != MemVT && !ST->isTruncatingStore())
Expand Down
9 changes: 4 additions & 5 deletions llvm/test/CodeGen/AArch64/aarch64-neon-vector-insert-uaddlv.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,10 @@ define void @insert_vec_v23i32_uaddlv_from_v8i16(ptr %0) {
; CHECK-NEXT: movi.2d v0, #0000000000000000
; CHECK-NEXT: movi.2d v2, #0000000000000000
; CHECK-NEXT: str wzr, [x0, #88]
; CHECK-NEXT: str xzr, [x0, #80]
; CHECK-NEXT: uaddlv.8h s1, v0
; CHECK-NEXT: stp q0, q0, [x0, #16]
; CHECK-NEXT: stp q0, q0, [x0, #48]
; CHECK-NEXT: str d0, [x0, #80]
; CHECK-NEXT: mov.s v2[0], v1[0]
; CHECK-NEXT: ucvtf.4s v1, v2
; CHECK-NEXT: str q1, [x0]
Expand Down Expand Up @@ -146,13 +146,12 @@ define void @insert_vec_v6i64_uaddlv_from_v4i32(ptr %0) {
; CHECK-LABEL: insert_vec_v6i64_uaddlv_from_v4i32:
; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: movi.2d v0, #0000000000000000
; CHECK-NEXT: movi.2d v2, #0000000000000000
; CHECK-NEXT: str xzr, [x0, #16]
; CHECK-NEXT: uaddlv.4s d1, v0
; CHECK-NEXT: str d0, [x0, #16]
; CHECK-NEXT: fmov x8, d1
; CHECK-NEXT: ucvtf s1, x8
; CHECK-NEXT: mov.s v2[0], v1[0]
; CHECK-NEXT: str q2, [x0]
; CHECK-NEXT: mov.s v0[0], v1[0]
; CHECK-NEXT: str q0, [x0]
; CHECK-NEXT: ret

entry:
Expand Down