Skip to content

[PowerPC] Exploit xxeval instruction for operations of the form ternary(A,X,B) and ternary(A,X,C). #152956

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 1, 2025
Merged

[PowerPC] Exploit xxeval instruction for operations of the form ternary(A,X,B) and ternary(A,X,C). #152956

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9aaebd9
[PowerPC] Exploit xxeval instruction for operations of the form terna…
Aug 11, 2025
f1c8063
Added comments why certain ternary patterns are omitted, fixed the op…
Aug 20, 2025
ce384e6
Merge branch 'main' into tvarghese/xxevalxbc
tonykuttai Aug 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions llvm/lib/Target/PowerPC/PPCInstrP10.td
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2231,6 +2231,13 @@ def VEqv
(v4i32(bitconvert node:$a)),
(v4i32(bitconvert node:$b)))))]>;

// Vector NAND operation (not(and))
def VNand
: PatFrags<(ops node:$a, node:$b), [(vnot(and node:$a, node:$b)),
(bitconvert(vnot(and
(v4i32(bitconvert node:$a)),
(v4i32(bitconvert node:$b)))))]>;

// =============================================================================
// XXEVAL Ternary Pattern Multiclass: XXEvalTernarySelectAnd
// This class matches the equivalent Ternary Operation: A ? f(B,C) : AND(B,C)
Expand Down Expand Up @@ -2266,6 +2273,56 @@ multiclass XXEvalTernarySelectAnd<ValueType Vt> {
Vt, (vselect Vt:$vA, (VNot Vt:$vB), (VAnd Vt:$vB, Vt:$vC)), 28>;
}

// =============================================================================
// XXEVAL Ternary Pattern Multiclass: XXEvalTernarySelectB
// This class matches the equivalent Ternary Operation: A ? f(B,C) : B
// and emit the corresponding xxeval instruction with the imm value.
//
// The patterns implement xxeval vector select operations where:
// - A is the selector vector
// - f(B,C) is the "true" case op on vectors B and C (AND, NOR, EQV, NAND)
Copy link
Contributor

@Himadhith Himadhith Aug 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Group Code Review - Change op to operand to have a more clear description.

Copy link
Contributor Author

@tonykuttai tonykuttai Aug 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here op refers to f which is correct.

Copy link
Contributor

@amy-kwan amy-kwan Aug 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is operation more accurate?

Copy link
Contributor Author

@tonykuttai tonykuttai Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed it.

// - B is the "false" case operand (vector B)
//
// Note: Patterns (A? C : B) and (A? not(C) : B) are not considered
// for XXEVAL instruction (4 Cycle) as XXSEL (3 cycle) instruction performs
// better.
// =============================================================================
multiclass XXEvalTernarySelectB<ValueType Vt>{
// Pattern: (A ? AND(B,C) : B) XXEVAL immediate value: 49
def : XXEvalPattern<Vt, (vselect Vt:$vA, (VAnd Vt:$vB, Vt:$vC), Vt:$vB), 49>;
// Pattern: (A ? NOR(B,C) : B) XXEVAL immediate value: 56
def : XXEvalPattern<Vt, (vselect Vt:$vA, (VNor Vt:$vB, Vt:$vC), Vt:$vB), 56>;
// Pattern: (A ? EQV(B,C) : B) XXEVAL immediate value: 57
def : XXEvalPattern<Vt, (vselect Vt:$vA, (VEqv Vt:$vB, Vt:$vC), Vt:$vB), 57>;
// Pattern: (A ? NAND(B,C) : B) XXEVAL immediate value: 62
def : XXEvalPattern<Vt, (vselect Vt:$vA, (VNand Vt:$vB, Vt:$vC), Vt:$vB), 62>;
}

// =============================================================================
// XXEVAL Ternary Pattern Multiclass: XXEvalTernarySelectC
// This class matches the equivalent Ternary Operation: A ? f(B,C) : C
// and emit the corresponding xxeval instruction with the imm value.
//
// The patterns implement xxeval vector select operations where:
// - A is the selector vector
// - f(B,C) is the "true" case op on vectors B and C (AND, NOR, EQV, NAND)
// - C is the "false" case operand (vector C)
//
// Note: Patterns (A? B : C) and (A? not(B) : C) are not considered
// for XXEVAL instruction (4 Cycle) as XXSEL (3 cycle) instruction performs
// better.
// =============================================================================
multiclass XXEvalTernarySelectC<ValueType Vt>{
// Pattern: (A ? AND(B,C) : C) XXEVAL immediate value: 81
def : XXEvalPattern<Vt, (vselect Vt:$vA, (VAnd Vt:$vB, Vt:$vC), Vt:$vC), 81>;
// Pattern: (A ? NOR(B,C) : C) XXEVAL immediate value: 88
def : XXEvalPattern<Vt, (vselect Vt:$vA, (VNor Vt:$vB, Vt:$vC), Vt:$vC), 88>;
// Pattern: (A ? EQV(B,C) : C) XXEVAL immediate value: 89
def : XXEvalPattern<Vt, (vselect Vt:$vA, (VEqv Vt:$vB, Vt:$vC), Vt:$vC), 89>;
// Pattern: (A ? NAND(B,C) : C) XXEVAL immediate value: 94
def : XXEvalPattern<Vt, (vselect Vt:$vA, (VNand Vt:$vB, Vt:$vC), Vt:$vC), 94>;
}

let Predicates = [PrefixInstrs, HasP10Vector] in {
let AddedComplexity = 400 in {
def : Pat<(v4i32 (build_vector i32immNonAllOneNonZero:$A,
Expand Down Expand Up @@ -2377,6 +2434,8 @@ let Predicates = [PrefixInstrs, HasP10Vector] in {
// XXEval Patterns for ternary Operations.
foreach Ty = [v4i32, v2i64, v8i16, v16i8] in {
defm : XXEvalTernarySelectAnd<Ty>;
defm : XXEvalTernarySelectB<Ty>;
defm : XXEvalTernarySelectC<Ty>;
}

// Anonymous patterns to select prefixed VSX loads and stores.
Expand Down
50 changes: 17 additions & 33 deletions llvm/test/CodeGen/PowerPC/xxeval-vselect-x-b.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
; Test file to verify the emission of Vector Selection instructions when ternary operators are used.
; Test file to verify the emission of Vector Evaluation instructions when ternary operators are used.

; RUN: llc -verify-machineinstrs -mcpu=pwr10 -mtriple=powerpc64le-unknown-unknown \
; RUN: -ppc-asm-full-reg-names --ppc-vsr-nums-as-vr < %s | FileCheck %s
Expand All @@ -15,10 +15,9 @@ define <4 x i32> @ternary_A_and_BC_B_4x32(<4 x i1> %A, <4 x i32> %B, <4 x i32> %
; CHECK-LABEL: ternary_A_and_BC_B_4x32:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: xxleqv v5, v5, v5
; CHECK-NEXT: xxland vs0, v3, v4
; CHECK-NEXT: vslw v2, v2, v5
; CHECK-NEXT: vsraw v2, v2, v5
; CHECK-NEXT: xxsel v2, v3, vs0, v2
; CHECK-NEXT: xxeval v2, v2, v3, v4, 49
; CHECK-NEXT: blr
entry:
%and = and <4 x i32> %B, %C
Expand All @@ -31,11 +30,10 @@ define <2 x i64> @ternary_A_and_BC_B_2x64(<2 x i1> %A, <2 x i64> %B, <2 x i64> %
; CHECK-LABEL: ternary_A_and_BC_B_2x64:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: xxlxor v5, v5, v5
; CHECK-NEXT: xxland vs0, v3, v4
; CHECK-NEXT: xxsplti32dx v5, 1, 63
; CHECK-NEXT: vsld v2, v2, v5
; CHECK-NEXT: vsrad v2, v2, v5
; CHECK-NEXT: xxsel v2, v3, vs0, v2
; CHECK-NEXT: xxeval v2, v2, v3, v4, 49
; CHECK-NEXT: blr
entry:
%and = and <2 x i64> %B, %C
Expand All @@ -48,10 +46,9 @@ define <16 x i8> @ternary_A_and_BC_B_16x8(<16 x i1> %A, <16 x i8> %B, <16 x i8>
; CHECK-LABEL: ternary_A_and_BC_B_16x8:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: xxspltib v5, 7
; CHECK-NEXT: xxland vs0, v3, v4
; CHECK-NEXT: vslb v2, v2, v5
; CHECK-NEXT: vsrab v2, v2, v5
; CHECK-NEXT: xxsel v2, v3, vs0, v2
; CHECK-NEXT: xxeval v2, v2, v3, v4, 49
; CHECK-NEXT: blr
entry:
%and = and <16 x i8> %B, %C
Expand All @@ -64,10 +61,9 @@ define <8 x i16> @ternary_A_and_BC_B_8x16(<8 x i1> %A, <8 x i16> %B, <8 x i16> %
; CHECK-LABEL: ternary_A_and_BC_B_8x16:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: xxspltiw v5, 983055
; CHECK-NEXT: xxland vs0, v3, v4
; CHECK-NEXT: vslh v2, v2, v5
; CHECK-NEXT: vsrah v2, v2, v5
; CHECK-NEXT: xxsel v2, v3, vs0, v2
; CHECK-NEXT: xxeval v2, v2, v3, v4, 49
; CHECK-NEXT: blr
entry:
%and = and <8 x i16> %B, %C
Expand All @@ -80,10 +76,9 @@ define <4 x i32> @ternary_A_nor_BC_B_4x32(<4 x i1> %A, <4 x i32> %B, <4 x i32> %
; CHECK-LABEL: ternary_A_nor_BC_B_4x32:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: xxleqv v5, v5, v5
; CHECK-NEXT: xxlnor vs0, v3, v4
; CHECK-NEXT: vslw v2, v2, v5
; CHECK-NEXT: vsraw v2, v2, v5
; CHECK-NEXT: xxsel v2, v3, vs0, v2
; CHECK-NEXT: xxeval v2, v2, v3, v4, 56
; CHECK-NEXT: blr
entry:
%or = or <4 x i32> %B, %C
Expand All @@ -97,11 +92,10 @@ define <2 x i64> @ternary_A_nor_BC_B_2x64(<2 x i1> %A, <2 x i64> %B, <2 x i64> %
; CHECK-LABEL: ternary_A_nor_BC_B_2x64:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: xxlxor v5, v5, v5
; CHECK-NEXT: xxlnor vs0, v3, v4
; CHECK-NEXT: xxsplti32dx v5, 1, 63
; CHECK-NEXT: vsld v2, v2, v5
; CHECK-NEXT: vsrad v2, v2, v5
; CHECK-NEXT: xxsel v2, v3, vs0, v2
; CHECK-NEXT: xxeval v2, v2, v3, v4, 56
; CHECK-NEXT: blr
entry:
%or = or <2 x i64> %B, %C
Expand All @@ -115,10 +109,9 @@ define <16 x i8> @ternary_A_nor_BC_B_16x8(<16 x i1> %A, <16 x i8> %B, <16 x i8>
; CHECK-LABEL: ternary_A_nor_BC_B_16x8:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: xxspltib v5, 7
; CHECK-NEXT: xxlnor vs0, v3, v4
; CHECK-NEXT: vslb v2, v2, v5
; CHECK-NEXT: vsrab v2, v2, v5
; CHECK-NEXT: xxsel v2, v3, vs0, v2
; CHECK-NEXT: xxeval v2, v2, v3, v4, 56
; CHECK-NEXT: blr
entry:
%or = or <16 x i8> %B, %C
Expand All @@ -132,10 +125,9 @@ define <8 x i16> @ternary_A_nor_BC_B_8x16(<8 x i1> %A, <8 x i16> %B, <8 x i16> %
; CHECK-LABEL: ternary_A_nor_BC_B_8x16:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: xxspltiw v5, 983055
; CHECK-NEXT: xxlnor vs0, v3, v4
; CHECK-NEXT: vslh v2, v2, v5
; CHECK-NEXT: vsrah v2, v2, v5
; CHECK-NEXT: xxsel v2, v3, vs0, v2
; CHECK-NEXT: xxeval v2, v2, v3, v4, 56
; CHECK-NEXT: blr
entry:
%or = or <8 x i16> %B, %C
Expand All @@ -149,10 +141,9 @@ define <4 x i32> @ternary_A_eqv_BC_B_4x32(<4 x i1> %A, <4 x i32> %B, <4 x i32> %
; CHECK-LABEL: ternary_A_eqv_BC_B_4x32:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: xxleqv v5, v5, v5
; CHECK-NEXT: xxleqv vs0, v3, v4
; CHECK-NEXT: vslw v2, v2, v5
; CHECK-NEXT: vsraw v2, v2, v5
; CHECK-NEXT: xxsel v2, v3, vs0, v2
; CHECK-NEXT: xxeval v2, v2, v3, v4, 57
; CHECK-NEXT: blr
entry:
%xor = xor <4 x i32> %B, %C
Expand All @@ -166,11 +157,10 @@ define <2 x i64> @ternary_A_eqv_BC_B_2x64(<2 x i1> %A, <2 x i64> %B, <2 x i64> %
; CHECK-LABEL: ternary_A_eqv_BC_B_2x64:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: xxlxor v5, v5, v5
; CHECK-NEXT: xxleqv vs0, v3, v4
; CHECK-NEXT: xxsplti32dx v5, 1, 63
; CHECK-NEXT: vsld v2, v2, v5
; CHECK-NEXT: vsrad v2, v2, v5
; CHECK-NEXT: xxsel v2, v3, vs0, v2
; CHECK-NEXT: xxeval v2, v2, v3, v4, 57
; CHECK-NEXT: blr
entry:
%xor = xor <2 x i64> %B, %C
Expand All @@ -184,10 +174,9 @@ define <16 x i8> @ternary_A_eqv_BC_B_16x8(<16 x i1> %A, <16 x i8> %B, <16 x i8>
; CHECK-LABEL: ternary_A_eqv_BC_B_16x8:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: xxspltib v5, 7
; CHECK-NEXT: xxleqv vs0, v3, v4
; CHECK-NEXT: vslb v2, v2, v5
; CHECK-NEXT: vsrab v2, v2, v5
; CHECK-NEXT: xxsel v2, v3, vs0, v2
; CHECK-NEXT: xxeval v2, v2, v3, v4, 57
; CHECK-NEXT: blr
entry:
%xor = xor <16 x i8> %B, %C
Expand All @@ -201,10 +190,9 @@ define <8 x i16> @ternary_A_eqv_BC_B_8x16(<8 x i1> %A, <8 x i16> %B, <8 x i16> %
; CHECK-LABEL: ternary_A_eqv_BC_B_8x16:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: xxspltiw v5, 983055
; CHECK-NEXT: xxleqv vs0, v3, v4
; CHECK-NEXT: vslh v2, v2, v5
; CHECK-NEXT: vsrah v2, v2, v5
; CHECK-NEXT: xxsel v2, v3, vs0, v2
; CHECK-NEXT: xxeval v2, v2, v3, v4, 57
; CHECK-NEXT: blr
entry:
%xor = xor <8 x i16> %B, %C
Expand All @@ -218,10 +206,9 @@ define <4 x i32> @ternary_A_nand_BC_B_4x32(<4 x i1> %A, <4 x i32> %B, <4 x i32>
; CHECK-LABEL: ternary_A_nand_BC_B_4x32:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: xxleqv v5, v5, v5
; CHECK-NEXT: xxlnand vs0, v3, v4
; CHECK-NEXT: vslw v2, v2, v5
; CHECK-NEXT: vsraw v2, v2, v5
; CHECK-NEXT: xxsel v2, v3, vs0, v2
; CHECK-NEXT: xxeval v2, v2, v3, v4, 62
; CHECK-NEXT: blr
entry:
%and = and <4 x i32> %B, %C
Expand All @@ -235,11 +222,10 @@ define <2 x i64> @ternary_A_nand_BC_B_2x64(<2 x i1> %A, <2 x i64> %B, <2 x i64>
; CHECK-LABEL: ternary_A_nand_BC_B_2x64:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: xxlxor v5, v5, v5
; CHECK-NEXT: xxlnand vs0, v3, v4
; CHECK-NEXT: xxsplti32dx v5, 1, 63
; CHECK-NEXT: vsld v2, v2, v5
; CHECK-NEXT: vsrad v2, v2, v5
; CHECK-NEXT: xxsel v2, v3, vs0, v2
; CHECK-NEXT: xxeval v2, v2, v3, v4, 62
; CHECK-NEXT: blr
entry:
%and = and <2 x i64> %B, %C
Expand All @@ -253,10 +239,9 @@ define <16 x i8> @ternary_A_nand_BC_B_16x8(<16 x i1> %A, <16 x i8> %B, <16 x i8>
; CHECK-LABEL: ternary_A_nand_BC_B_16x8:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: xxspltib v5, 7
; CHECK-NEXT: xxlnand vs0, v3, v4
; CHECK-NEXT: vslb v2, v2, v5
; CHECK-NEXT: vsrab v2, v2, v5
; CHECK-NEXT: xxsel v2, v3, vs0, v2
; CHECK-NEXT: xxeval v2, v2, v3, v4, 62
; CHECK-NEXT: blr
entry:
%and = and <16 x i8> %B, %C
Expand All @@ -270,10 +255,9 @@ define <8 x i16> @ternary_A_nand_BC_B_8x16(<8 x i1> %A, <8 x i16> %B, <8 x i16>
; CHECK-LABEL: ternary_A_nand_BC_B_8x16:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: xxspltiw v5, 983055
; CHECK-NEXT: xxlnand vs0, v3, v4
; CHECK-NEXT: vslh v2, v2, v5
; CHECK-NEXT: vsrah v2, v2, v5
; CHECK-NEXT: xxsel v2, v3, vs0, v2
; CHECK-NEXT: xxeval v2, v2, v3, v4, 62
; CHECK-NEXT: blr
entry:
%and = and <8 x i16> %B, %C
Expand Down
Loading