Skip to content

[AArch64] Enable using SVE2 bit-sel instructions with Neon types. #146906

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 8, 2025
Merged

[AArch64] Enable using SVE2 bit-sel instructions with Neon types. #146906

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6f069e1
Add tests.
rj-jesus Jun 5, 2025
2e7bb05
[AArch64] Enable SVE2 bit-sel instructions with Neon types.
rj-jesus May 6, 2025
385ab19
Add tests for v4i8.
rj-jesus Jul 3, 2025
0eeb5a3
Add RUN lines for BCAX and EOR3 lowering with +sha3,+sve2.
rj-jesus Jul 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions llvm/lib/Target/AArch64/SVEInstrFormats.td
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5368,6 +5368,19 @@ multiclass sve2_int_bitwise_ternary_op<bits<3> opc, string asm,
def : SVE_3_Op_Pat<nxv8i16, op, nxv8i16, nxv8i16, nxv8i16, !cast<Instruction>(NAME)>;
def : SVE_3_Op_Pat<nxv4i32, op, nxv4i32, nxv4i32, nxv4i32, !cast<Instruction>(NAME)>;
def : SVE_3_Op_Pat<nxv2i64, op, nxv2i64, nxv2i64, nxv2i64, !cast<Instruction>(NAME)>;

// Allow selecting SVE2 ternary ops with Neon types.
foreach VT = [nxv16i8, nxv8i16, nxv4i32, nxv2i64] in {
Copy link
Collaborator

@paulwalker-arm paulwalker-arm Jul 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you specifically care about the case when FEAT_SHA3 is not available? I ask because when available the NEON variants look to have less restrictive register requirements? Specially for BCAX and EOR3 which have a dedicated result register.

If you just want to use the other instructions (bsl1n etc) then a quick fix would be to pass the VT array into the class and then only set the parameter for the instructions that are in addition to those available under FEAT_SHA3.

Copy link
Contributor Author

@rj-jesus rj-jesus Jul 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks - passing VT into the class sounds good to me. I did check previously that the Neon BCAX/EOR3 instructions were selected over the SVE ones when FEAT_SHA3 is available (due to the less restrictive register requirements, as you point out). Since they were, I didn't see a reason not to enable the SVE2 patterns for them too.

Would you rather I do as you suggested and only enable the patterns for BSL1N/BSL1N/NBSL, or perhaps add a runline such as

RUN: llc -mtriple=aarch64 -mattr=+sha3,+sve2 < %s | FileCheck --check-prefix=SHA3 %s

in llvm/test/CodeGen/AArch64/bcax.ll and llvm/test/CodeGen/AArch64/eor3.ll to ensure we select the SHA3 patterns even when SVE2 is available?

Copy link
Collaborator

@paulwalker-arm paulwalker-arm Jul 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this works as is then just adding the RUN lines for verification works for me.

Copy link
Contributor Author

@rj-jesus rj-jesus Jul 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks very much - done. :)

def : Pat<(SVEType<VT>.DSub (op V64:$op1, V64:$op2, V64:$op3)),
(EXTRACT_SUBREG (!cast<Instruction>(NAME) (INSERT_SUBREG (IMPLICIT_DEF), $op1, dsub),
(INSERT_SUBREG (IMPLICIT_DEF), $op2, dsub),
(INSERT_SUBREG (IMPLICIT_DEF), $op3, dsub)), dsub)>;

def : Pat<(SVEType<VT>.ZSub (op V128:$op1, V128:$op2, V128:$op3)),
(EXTRACT_SUBREG (!cast<Instruction>(NAME) (INSERT_SUBREG (IMPLICIT_DEF), $op1, zsub),
(INSERT_SUBREG (IMPLICIT_DEF), $op2, zsub),
(INSERT_SUBREG (IMPLICIT_DEF), $op3, zsub)), zsub)>;
}
}

class sve2_int_rotate_right_imm<bits<4> tsz8_64, string asm,
Expand Down
38 changes: 38 additions & 0 deletions llvm/test/CodeGen/AArch64/bcax.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --extra_scrub
; RUN: llc -mtriple=aarch64 -mattr=+sha3 < %s | FileCheck --check-prefix=SHA3 %s
; RUN: llc -mtriple=aarch64 -mattr=-sha3 < %s | FileCheck --check-prefix=NOSHA3 %s
; RUN: llc -mtriple=aarch64 -mattr=+sve2 < %s | FileCheck --check-prefix=SVE2 %s
; RUN: llc -mtriple=aarch64 -mattr=+sha3,+sve2 < %s | FileCheck --check-prefix=SHA3 %s

define <2 x i64> @bcax_64x2(<2 x i64> %0, <2 x i64> %1, <2 x i64> %2) {
; SHA3-LABEL: bcax_64x2:
Expand All @@ -13,6 +15,15 @@ define <2 x i64> @bcax_64x2(<2 x i64> %0, <2 x i64> %1, <2 x i64> %2) {
; NOSHA3-NEXT: bic v0.16b, v0.16b, v1.16b
; NOSHA3-NEXT: eor v0.16b, v0.16b, v2.16b
; NOSHA3-NEXT: ret
;
; SVE2-LABEL: bcax_64x2:
; SVE2: // %bb.0:
; SVE2-NEXT: // kill: def $q2 killed $q2 def $z2
; SVE2-NEXT: // kill: def $q1 killed $q1 def $z1
; SVE2-NEXT: // kill: def $q0 killed $q0 def $z0
; SVE2-NEXT: bcax z2.d, z2.d, z0.d, z1.d
; SVE2-NEXT: mov v0.16b, v2.16b
; SVE2-NEXT: ret
%4 = xor <2 x i64> %1, <i64 -1, i64 -1>
%5 = and <2 x i64> %4, %0
%6 = xor <2 x i64> %5, %2
Expand All @@ -30,6 +41,15 @@ define <4 x i32> @bcax_32x4(<4 x i32> %0, <4 x i32> %1, <4 x i32> %2) {
; NOSHA3-NEXT: bic v0.16b, v0.16b, v1.16b
; NOSHA3-NEXT: eor v0.16b, v0.16b, v2.16b
; NOSHA3-NEXT: ret
;
; SVE2-LABEL: bcax_32x4:
; SVE2: // %bb.0:
; SVE2-NEXT: // kill: def $q2 killed $q2 def $z2
; SVE2-NEXT: // kill: def $q1 killed $q1 def $z1
; SVE2-NEXT: // kill: def $q0 killed $q0 def $z0
; SVE2-NEXT: bcax z2.d, z2.d, z0.d, z1.d
; SVE2-NEXT: mov v0.16b, v2.16b
; SVE2-NEXT: ret
%4 = xor <4 x i32> %1, <i32 -1, i32 -1, i32 -1, i32 -1>
%5 = and <4 x i32> %4, %0
%6 = xor <4 x i32> %5, %2
Expand All @@ -47,6 +67,15 @@ define <8 x i16> @bcax_16x8(<8 x i16> %0, <8 x i16> %1, <8 x i16> %2) {
; NOSHA3-NEXT: bic v0.16b, v0.16b, v1.16b
; NOSHA3-NEXT: eor v0.16b, v0.16b, v2.16b
; NOSHA3-NEXT: ret
;
; SVE2-LABEL: bcax_16x8:
; SVE2: // %bb.0:
; SVE2-NEXT: // kill: def $q2 killed $q2 def $z2
; SVE2-NEXT: // kill: def $q1 killed $q1 def $z1
; SVE2-NEXT: // kill: def $q0 killed $q0 def $z0
; SVE2-NEXT: bcax z2.d, z2.d, z0.d, z1.d
; SVE2-NEXT: mov v0.16b, v2.16b
; SVE2-NEXT: ret
%4 = xor <8 x i16> %1, <i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1>
%5 = and <8 x i16> %4, %0
%6 = xor <8 x i16> %5, %2
Expand All @@ -64,6 +93,15 @@ define <16 x i8> @bcax_8x16(<16 x i8> %0, <16 x i8> %1, <16 x i8> %2) {
; NOSHA3-NEXT: bic v0.16b, v0.16b, v1.16b
; NOSHA3-NEXT: eor v0.16b, v0.16b, v2.16b
; NOSHA3-NEXT: ret
;
; SVE2-LABEL: bcax_8x16:
; SVE2: // %bb.0:
; SVE2-NEXT: // kill: def $q2 killed $q2 def $z2
; SVE2-NEXT: // kill: def $q1 killed $q1 def $z1
; SVE2-NEXT: // kill: def $q0 killed $q0 def $z0
; SVE2-NEXT: bcax z2.d, z2.d, z0.d, z1.d
; SVE2-NEXT: mov v0.16b, v2.16b
; SVE2-NEXT: ret
%4 = xor <16 x i8> %1, <i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1>
%5 = and <16 x i8> %4, %0
%6 = xor <16 x i8> %5, %2
Expand Down
Loading