[AArch64] Define constructive EXT_ZZI pseudo instruction #152552

gbossu · 2025-08-07T17:17:11Z

It will get expanded into MOVPRFX_ZZ and EXT_ZZI by the AArch64ExpandPseudo pass. This instruction takes a single Z register as input, as opposed to the existing destructive EXT_ZZI instruction.

Note this patch only defines the pseudo, it isn't used in any ISel pattern yet. It will later be used for vector.extract.

This is a chained PR: #152552 - #152553 - #152554

llvmbot · 2025-08-07T17:17:43Z

@llvm/pr-subscribers-backend-aarch64

Author: Gaëtan Bossu (gbossu)

Changes

It will get expanded into MOVPRFX_ZZ and EXT_ZZI by the AArch64ExpandPseudo pass.

Note this patch only defines the pseudo, it isn't used in any ISel pattern yet.

This is a chained PR

Full diff: https://github.com/llvm/llvm-project/pull/152552.diff

13 Files Affected:

(modified) llvm/lib/Target/AArch64/AArch64ExpandPseudoInsts.cpp (+24-4)
(modified) llvm/lib/Target/AArch64/AArch64InstrFormats.td (+2-1)
(modified) llvm/lib/Target/AArch64/AArch64InstrInfo.h (+2-1)
(modified) llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td (+3-1)
(modified) llvm/lib/Target/AArch64/AArch64SchedA320.td (+1-1)
(modified) llvm/lib/Target/AArch64/AArch64SchedA510.td (+1-1)
(modified) llvm/lib/Target/AArch64/AArch64SchedNeoverseN2.td (+1-1)
(modified) llvm/lib/Target/AArch64/AArch64SchedNeoverseN3.td (+1-1)
(modified) llvm/lib/Target/AArch64/AArch64SchedNeoverseV1.td (+1-1)
(modified) llvm/lib/Target/AArch64/AArch64SchedNeoverseV2.td (+1-1)
(modified) llvm/lib/Target/AArch64/SVEInstrFormats.td (+9-3)
(added) llvm/test/CodeGen/AArch64/expand-constructive-zpzz.mir (+95)
(added) llvm/test/CodeGen/AArch64/expand-constructive-zzzi.mir (+91)

diff --git a/llvm/lib/Target/AArch64/AArch64ExpandPseudoInsts.cpp b/llvm/lib/Target/AArch64/AArch64ExpandPseudoInsts.cpp
index 201bfe0a443d6..45f9c8fdec567 100644
--- a/llvm/lib/Target/AArch64/AArch64ExpandPseudoInsts.cpp
+++ b/llvm/lib/Target/AArch64/AArch64ExpandPseudoInsts.cpp
@@ -528,6 +528,10 @@ bool AArch64ExpandPseudo::expand_DestructiveOp(
       UseRev = true;
     }
     break;
+  case AArch64::DestructiveRegRegImmUnpred:
+    // EXT_ZZZI Zd, Zs1, Zs2, Imm ==> EXT_ZZI Zds1, Zs2, Imm
+    std::tie(DOPIdx, SrcIdx, Src2Idx) = std::make_tuple(1, 2, 3);
+    break;
   default:
     llvm_unreachable("Unsupported Destructive Operand type");
   }
@@ -538,6 +542,7 @@ bool AArch64ExpandPseudo::expand_DestructiveOp(
   bool DOPRegIsUnique = false;
   switch (DType) {
   case AArch64::DestructiveBinary:
+  case AArch64::DestructiveRegRegImmUnpred:
     DOPRegIsUnique = DstReg != MI.getOperand(SrcIdx).getReg();
     break;
   case AArch64::DestructiveBinaryComm:
@@ -639,10 +644,20 @@ bool AArch64ExpandPseudo::expand_DestructiveOp(
           .addImm(0);
     }
   } else if (DstReg != MI.getOperand(DOPIdx).getReg()) {
-    assert(DOPRegIsUnique && "The destructive operand should be unique");
-    PRFX = BuildMI(MBB, MBBI, MI.getDebugLoc(), TII->get(MovPrfx))
-               .addReg(DstReg, RegState::Define)
-               .addReg(MI.getOperand(DOPIdx).getReg(), DOPRegState);
+    if (DOPRegIsUnique) {
+      PRFX = BuildMI(MBB, MBBI, MI.getDebugLoc(), TII->get(MovPrfx))
+                 .addReg(DstReg, RegState::Define)
+                 .addReg(MI.getOperand(DOPIdx).getReg(), DOPRegState);
+    } else {
+      // MOVPRFX requires unique operands: Just build a COPY (Using ORR directly
+      // as we are past PostRAPseudo expansion).
+      assert(DType == AArch64::DestructiveRegRegImmUnpred &&
+             "Unexpected destructive operation type");
+      PRFX = BuildMI(MBB, MBBI, MI.getDebugLoc(), TII->get(AArch64::ORR_ZZZ))
+                 .addReg(DstReg, RegState::Define)
+                 .addReg(MI.getOperand(DOPIdx).getReg(), DOPRegState)
+                 .addReg(MI.getOperand(DOPIdx).getReg(), DOPRegState);
+    }
     DOPIdx = 0;
     DOPRegState = 0;
   }
@@ -674,6 +689,11 @@ bool AArch64ExpandPseudo::expand_DestructiveOp(
         .add(MI.getOperand(SrcIdx))
         .add(MI.getOperand(Src2Idx));
     break;
+  case AArch64::DestructiveRegRegImmUnpred:
+    DOP.addReg(MI.getOperand(DOPIdx).getReg(), DOPRegState)
+        .add(MI.getOperand(SrcIdx))
+        .add(MI.getOperand(Src2Idx));
+    break;
   }
 
   if (PRFX) {
diff --git a/llvm/lib/Target/AArch64/AArch64InstrFormats.td b/llvm/lib/Target/AArch64/AArch64InstrFormats.td
index d068a12c7f7d5..35f1151f90068 100644
--- a/llvm/lib/Target/AArch64/AArch64InstrFormats.td
+++ b/llvm/lib/Target/AArch64/AArch64InstrFormats.td
@@ -36,7 +36,8 @@ def DestructiveBinary             : DestructiveInstTypeEnum<5>;
 def DestructiveBinaryComm         : DestructiveInstTypeEnum<6>;
 def DestructiveBinaryCommWithRev  : DestructiveInstTypeEnum<7>;
 def DestructiveTernaryCommWithRev : DestructiveInstTypeEnum<8>;
-def DestructiveUnaryPassthru      : DestructiveInstTypeEnum<9>;
+def DestructiveRegRegImmUnpred    : DestructiveInstTypeEnum<9>;
+def DestructiveUnaryPassthru      : DestructiveInstTypeEnum<10>;
 
 class FalseLanesEnum<bits<2> val> {
   bits<2> Value = val;
diff --git a/llvm/lib/Target/AArch64/AArch64InstrInfo.h b/llvm/lib/Target/AArch64/AArch64InstrInfo.h
index 7c255da333e4b..9df1d80101dfe 100644
--- a/llvm/lib/Target/AArch64/AArch64InstrInfo.h
+++ b/llvm/lib/Target/AArch64/AArch64InstrInfo.h
@@ -820,7 +820,8 @@ enum DestructiveInstType {
   DestructiveBinaryComm         = TSFLAG_DESTRUCTIVE_INST_TYPE(0x6),
   DestructiveBinaryCommWithRev  = TSFLAG_DESTRUCTIVE_INST_TYPE(0x7),
   DestructiveTernaryCommWithRev = TSFLAG_DESTRUCTIVE_INST_TYPE(0x8),
-  DestructiveUnaryPassthru      = TSFLAG_DESTRUCTIVE_INST_TYPE(0x9),
+  DestructiveRegRegImmUnpred    = TSFLAG_DESTRUCTIVE_INST_TYPE(0x9),
+  DestructiveUnaryPassthru      = TSFLAG_DESTRUCTIVE_INST_TYPE(0xa),
 };
 
 enum FalseLaneType {
diff --git a/llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td b/llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td
index 0c4b4f4c3ed88..85e647af6684c 100644
--- a/llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td
+++ b/llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td
@@ -1021,7 +1021,9 @@ let Predicates = [HasNonStreamingSVE_or_SME2p2] in {
 let Predicates = [HasSVE_or_SME] in {
   defm INSR_ZR : sve_int_perm_insrs<"insr", AArch64insr>;
   defm INSR_ZV : sve_int_perm_insrv<"insr", AArch64insr>;
-  defm EXT_ZZI : sve_int_perm_extract_i<"ext", AArch64ext>;
+  defm EXT_ZZI : sve_int_perm_extract_i<"ext", AArch64ext, "EXT_ZZZI">;
+
+  def EXT_ZZZI : UnpredRegRegImmPseudo<"EXT_ZZZI", ZPR8, imm0_255>;
 
   defm RBIT_ZPmZ : sve_int_perm_rev_rbit<"rbit", AArch64rbit_mt>;
   defm REVB_ZPmZ : sve_int_perm_rev_revb<"revb", AArch64revb_mt>;
diff --git a/llvm/lib/Target/AArch64/AArch64SchedA320.td b/llvm/lib/Target/AArch64/AArch64SchedA320.td
index 89ed13389daf0..7f44a35ec37fc 100644
--- a/llvm/lib/Target/AArch64/AArch64SchedA320.td
+++ b/llvm/lib/Target/AArch64/AArch64SchedA320.td
@@ -847,7 +847,7 @@ def : InstRW<[CortexA320Write<3, CortexA320UnitVALU>], (instregex "^[SU]XTB_ZPmZ
                                             "^[SU]XTW_ZPmZ_[D]")>;
 
 // Extract
-def : InstRW<[CortexA320Write<3, CortexA320UnitVALU>], (instrs EXT_ZZI, EXT_ZZI_B)>;
+def : InstRW<[CortexA320Write<3, CortexA320UnitVALU>], (instrs EXT_ZZI, EXT_ZZZI, EXT_ZZI_B)>;
 
 // Extract narrow saturating
 def : InstRW<[CortexA320Write<4, CortexA320UnitVALU>], (instregex "^[SU]QXTN[BT]_ZZ_[BHS]",
diff --git a/llvm/lib/Target/AArch64/AArch64SchedA510.td b/llvm/lib/Target/AArch64/AArch64SchedA510.td
index 9456878946151..4126d876ca45a 100644
--- a/llvm/lib/Target/AArch64/AArch64SchedA510.td
+++ b/llvm/lib/Target/AArch64/AArch64SchedA510.td
@@ -825,7 +825,7 @@ def : InstRW<[CortexA510Write<3, CortexA510UnitVALU>], (instregex "^[SU]XTB_ZPmZ
                                             "^[SU]XTW_ZPmZ_[D]")>;
 
 // Extract
-def : InstRW<[CortexA510Write<3, CortexA510UnitVALU>], (instrs EXT_ZZI, EXT_ZZI_B)>;
+def : InstRW<[CortexA510Write<3, CortexA510UnitVALU>], (instrs EXT_ZZI, EXT_ZZZI, EXT_ZZI_B)>;
 
 // Extract narrow saturating
 def : InstRW<[CortexA510Write<4, CortexA510UnitVALU>], (instregex "^[SU]QXTN[BT]_ZZ_[BHS]",
diff --git a/llvm/lib/Target/AArch64/AArch64SchedNeoverseN2.td b/llvm/lib/Target/AArch64/AArch64SchedNeoverseN2.td
index 91a707910a7f3..82a50daae7e19 100644
--- a/llvm/lib/Target/AArch64/AArch64SchedNeoverseN2.td
+++ b/llvm/lib/Target/AArch64/AArch64SchedNeoverseN2.td
@@ -1785,7 +1785,7 @@ def : InstRW<[N2Write_2c_1V1], (instregex "^[SU]XTB_ZPmZ_[HSD]",
                                           "^[SU]XTW_ZPmZ_[D]")>;
 
 // Extract
-def : InstRW<[N2Write_2c_1V], (instrs EXT_ZZI, EXT_ZZI_B)>;
+def : InstRW<[N2Write_2c_1V], (instrs EXT_ZZI, EXT_ZZZI, EXT_ZZI_B)>;
 
 // Extract narrow saturating
 def : InstRW<[N2Write_4c_1V1], (instregex "^[SU]QXTN[BT]_ZZ_[BHS]$",
diff --git a/llvm/lib/Target/AArch64/AArch64SchedNeoverseN3.td b/llvm/lib/Target/AArch64/AArch64SchedNeoverseN3.td
index ecfb1249cfc49..1dbd7135d7870 100644
--- a/llvm/lib/Target/AArch64/AArch64SchedNeoverseN3.td
+++ b/llvm/lib/Target/AArch64/AArch64SchedNeoverseN3.td
@@ -1757,7 +1757,7 @@ def : InstRW<[N3Write_2c_1V], (instregex "^[SU]XTB_ZPmZ_[HSD]",
                                          "^[SU]XTW_ZPmZ_[D]")>;
 
 // Extract
-def : InstRW<[N3Write_2c_1V], (instrs EXT_ZZI, EXT_ZZI_B)>;
+def : InstRW<[N3Write_2c_1V], (instrs EXT_ZZI, EXT_ZZZI, EXT_ZZI_B)>;
 
 // Extract narrow saturating
 def : InstRW<[N3Write_4c_1V1], (instregex "^[SU]QXTN[BT]_ZZ_[BHS]$",
diff --git a/llvm/lib/Target/AArch64/AArch64SchedNeoverseV1.td b/llvm/lib/Target/AArch64/AArch64SchedNeoverseV1.td
index 368665467859f..e885c42f66bfb 100644
--- a/llvm/lib/Target/AArch64/AArch64SchedNeoverseV1.td
+++ b/llvm/lib/Target/AArch64/AArch64SchedNeoverseV1.td
@@ -1575,7 +1575,7 @@ def : InstRW<[V1Write_2c_1V1], (instregex "^[SU]XTB_ZPmZ_[HSD]",
                                           "^[SU]XTW_ZPmZ_[D]")>;
 
 // Extract
-def : InstRW<[V1Write_2c_1V01], (instrs EXT_ZZI)>;
+def : InstRW<[V1Write_2c_1V01], (instrs EXT_ZZI, EXT_ZZZI)>;
 
 // Extract/insert operation, SIMD and FP scalar form
 def : InstRW<[V1Write_3c_1V1], (instregex "^LAST[AB]_VPZ_[BHSD]$",
diff --git a/llvm/lib/Target/AArch64/AArch64SchedNeoverseV2.td b/llvm/lib/Target/AArch64/AArch64SchedNeoverseV2.td
index b2c3da03b4b84..678e10b35513d 100644
--- a/llvm/lib/Target/AArch64/AArch64SchedNeoverseV2.td
+++ b/llvm/lib/Target/AArch64/AArch64SchedNeoverseV2.td
@@ -2272,7 +2272,7 @@ def : InstRW<[V2Write_2c_1V13], (instregex "^[SU]XTB_ZPmZ_[HSD]",
                                            "^[SU]XTW_ZPmZ_[D]")>;
 
 // Extract
-def : InstRW<[V2Write_2c_1V], (instrs EXT_ZZI, EXT_ZZI_B)>;
+def : InstRW<[V2Write_2c_1V], (instrs EXT_ZZI, EXT_ZZZI, EXT_ZZI_B)>;
 
 // Extract narrow saturating
 def : InstRW<[V2Write_4c_1V13], (instregex "^[SU]QXTN[BT]_ZZ_[BHS]",
diff --git a/llvm/lib/Target/AArch64/SVEInstrFormats.td b/llvm/lib/Target/AArch64/SVEInstrFormats.td
index a0320f919e8c5..4392e36e6819c 100644
--- a/llvm/lib/Target/AArch64/SVEInstrFormats.td
+++ b/llvm/lib/Target/AArch64/SVEInstrFormats.td
@@ -809,6 +809,11 @@ let hasNoSchedulingInfo = 1 in {
     Pseudo<(outs zprty:$Zd), (ins PPR3bAny:$Pg, zprty:$Zs1, zprty:$Zs2, zprty:$Zs3), []> {
     let FalseLanes = flags;
   }
+
+  class UnpredRegRegImmPseudo<string name, ZPRRegOp zprty, Operand immty>
+  : SVEPseudo2Instr<name, 0>,
+    Pseudo<(outs zprty:$Zd), (ins zprty:$Zs1, zprty:$Zs2, immty:$imm), []> {
+  }
 }
 
 //
@@ -1885,13 +1890,14 @@ class sve_int_perm_extract_i<string asm>
   let Inst{4-0}   = Zdn;
 
   let Constraints = "$Zdn = $_Zdn";
-  let DestructiveInstType = DestructiveOther;
+  let DestructiveInstType = DestructiveRegRegImmUnpred;
   let ElementSize = ElementSizeNone;
   let hasSideEffects = 0;
 }
 
-multiclass sve_int_perm_extract_i<string asm, SDPatternOperator op> {
-  def NAME : sve_int_perm_extract_i<asm>;
+multiclass sve_int_perm_extract_i<string asm, SDPatternOperator op, string pseudoInstrName> {
+  def NAME : sve_int_perm_extract_i<asm>,
+             SVEPseudo2Instr<pseudoInstrName, 1>;
 
   def : SVE_3_Op_Imm_Pat<nxv16i8, op, nxv16i8, nxv16i8, i32, imm0_255,
                          !cast<Instruction>(NAME)>;
diff --git a/llvm/test/CodeGen/AArch64/expand-constructive-zpzz.mir b/llvm/test/CodeGen/AArch64/expand-constructive-zpzz.mir
new file mode 100644
index 0000000000000..5e1c211ccb9a2
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/expand-constructive-zpzz.mir
@@ -0,0 +1,95 @@
+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py UTC_ARGS: --version 5
+# RUN: llc -mtriple=aarch64 -mattr=+sve -run-pass=aarch64-expand-pseudo -verify-machineinstrs %s -o - | FileCheck %s
+
+# Test the expansion of constructive binary operations into their
+# destructive counterparts.
+
+
+# BIC_ZPmZ_B
+
+---
+name:            test_bic_zpzz_unique_regs
+body:             |
+  bb.0:
+    ; CHECK-LABEL: name: test_bic_zpzz_unique_regs
+    ; CHECK: BUNDLE implicit-def $z2, implicit-def $q2, implicit-def $d2, implicit-def $s2, implicit-def $h2, implicit-def $b2, implicit-def $b2_hi, implicit-def $h2_hi, implicit-def $s2_hi, implicit-def $d2_hi, implicit-def $q2_hi, implicit killed $p0, implicit $z0, implicit killed $z1 {
+    ; CHECK-NEXT:   $z2 = MOVPRFX_ZPzZ_B $p0, $z0
+    ; CHECK-NEXT:   $z2 = BIC_ZPmZ_B killed $p0, internal killed $z2, killed $z1
+    ; CHECK-NEXT: }
+    ; CHECK-NEXT: RET undef $lr, implicit killed $z2
+    $z2 = BIC_ZPZZ_B_ZERO killed $p0, killed $z0, killed $z1
+    RET_ReallyLR implicit killed $z2
+...
+
+---
+name:            test_bic_zpzz_not_unique_regs
+body:             |
+  bb.0:
+    ; CHECK-LABEL: name: test_bic_zpzz_not_unique_regs
+    ; CHECK: BUNDLE implicit-def $z2, implicit-def $q2, implicit-def $d2, implicit-def $s2, implicit-def $h2, implicit-def $b2, implicit-def $b2_hi, implicit-def $h2_hi, implicit-def $s2_hi, implicit-def $d2_hi, implicit-def $q2_hi, implicit killed $p0, implicit $z0 {
+    ; CHECK-NEXT:   $z2 = MOVPRFX_ZPzZ_B $p0, $z0
+    ; CHECK-NEXT:   $z2 = LSL_ZPmI_B killed $p0, internal $z2, 0
+    ; CHECK-NEXT:   $z2 = BIC_ZPmZ_B killed $p0, internal killed $z2, internal killed $z2
+    ; CHECK-NEXT: }
+    ; CHECK-NEXT: RET undef $lr, implicit killed $z2
+    $z2 = BIC_ZPZZ_B_ZERO killed $p0, killed $z0, killed $z2
+    RET_ReallyLR implicit killed $z2
+...
+
+---
+name:            test_bic_zpzz_implicit_ops
+body:             |
+  bb.0:
+    ; CHECK-LABEL: name: test_bic_zpzz_implicit_ops
+    ; CHECK: BUNDLE implicit-def $z2, implicit-def $q2, implicit-def $d2, implicit-def $s2, implicit-def $h2, implicit-def $b2, implicit-def $b2_hi, implicit-def $h2_hi, implicit-def $s2_hi, implicit-def $d2_hi, implicit-def $q2_hi, implicit killed $p0, implicit $z0, implicit killed $q0, implicit killed $q1, implicit killed $z1 {
+    ; CHECK-NEXT:   $z2 = MOVPRFX_ZPzZ_B $p0, $z0, implicit killed $q0, implicit killed $q1
+    ; CHECK-NEXT:   $z2 = BIC_ZPmZ_B killed $p0, internal killed $z2, killed $z1, implicit-def $q2
+    ; CHECK-NEXT: }
+    ; CHECK-NEXT: RET undef $lr, implicit killed $z2, implicit killed $q2
+    $z2 = BIC_ZPZZ_B_ZERO killed $p0, killed $z0, killed $z1, implicit-def $q2, implicit killed $q0, implicit killed $q1
+    RET_ReallyLR implicit killed $z2, implicit killed $q2
+...
+
+---
+name:            test_bic_zpzz_undef
+body:             |
+  bb.0:
+    ; CHECK-LABEL: name: test_bic_zpzz_undef
+    ; CHECK: BUNDLE implicit-def $z2, implicit-def $q2, implicit-def $d2, implicit-def $s2, implicit-def $h2, implicit-def $b2, implicit-def $b2_hi, implicit-def $h2_hi, implicit-def $s2_hi, implicit-def $d2_hi, implicit-def $q2_hi, implicit $p0, implicit undef $z0, implicit undef $z1 {
+    ; CHECK-NEXT:   $z2 = MOVPRFX_ZPzZ_B $p0, undef $z0
+    ; CHECK-NEXT:   $z2 = BIC_ZPmZ_B undef $p0, internal killed $z2, undef $z1
+    ; CHECK-NEXT: }
+    ; CHECK-NEXT: RET undef $lr, implicit killed $z2
+    $z2 = BIC_ZPZZ_B_ZERO undef $p0, undef $z0, undef $z1
+    RET_ReallyLR implicit killed $z2
+...
+
+# ADD_ZPmZ_B
+
+---
+name:            test_add_zpzz_unique_regs
+body:             |
+  bb.0:
+    ; CHECK-LABEL: name: test_add_zpzz_unique_regs
+    ; CHECK: BUNDLE implicit-def $z2, implicit-def $q2, implicit-def $d2, implicit-def $s2, implicit-def $h2, implicit-def $b2, implicit-def $b2_hi, implicit-def $h2_hi, implicit-def $s2_hi, implicit-def $d2_hi, implicit-def $q2_hi, implicit killed $p0, implicit $z0, implicit killed $z1 {
+    ; CHECK-NEXT:   $z2 = MOVPRFX_ZPzZ_B $p0, $z0
+    ; CHECK-NEXT:   $z2 = ADD_ZPmZ_B killed $p0, internal killed $z2, killed $z1
+    ; CHECK-NEXT: }
+    ; CHECK-NEXT: RET undef $lr, implicit killed $z2
+    $z2 = ADD_ZPZZ_B_ZERO killed $p0, killed $z0, killed $z1
+    RET_ReallyLR implicit killed $z2
+...
+
+---
+name:            test_add_zpzz_not_unique_regs
+body:             |
+  bb.0:
+    ; CHECK-LABEL: name: test_add_zpzz_not_unique_regs
+    ; CHECK: BUNDLE implicit-def $z2, implicit-def $q2, implicit-def $d2, implicit-def $s2, implicit-def $h2, implicit-def $b2, implicit-def $b2_hi, implicit-def $h2_hi, implicit-def $s2_hi, implicit-def $d2_hi, implicit-def $q2_hi, implicit killed $p0, implicit $z2, implicit killed $z0 {
+    ; CHECK-NEXT:   $z2 = MOVPRFX_ZPzZ_B $p0, $z2
+    ; CHECK-NEXT:   $z2 = ADD_ZPmZ_B killed $p0, internal killed $z2, killed $z0
+    ; CHECK-NEXT: }
+    ; CHECK-NEXT: RET undef $lr, implicit killed $z2
+    $z2 = ADD_ZPZZ_B_ZERO killed $p0, killed $z0, killed $z2
+    RET_ReallyLR implicit killed $z2
+...
diff --git a/llvm/test/CodeGen/AArch64/expand-constructive-zzzi.mir b/llvm/test/CodeGen/AArch64/expand-constructive-zzzi.mir
new file mode 100644
index 0000000000000..70fa117a5b463
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/expand-constructive-zzzi.mir
@@ -0,0 +1,91 @@
+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py UTC_ARGS: --version 5
+# RUN: llc -mtriple=aarch64 -mattr=+sve -run-pass=aarch64-expand-pseudo -verify-machineinstrs %s -o - | FileCheck %s
+
+# Test the expansion of constructive ternary operations into their
+# destructive counterparts.
+
+
+# EXT_ZZZI
+
+---
+name:            test_ext_zzzi_unique
+body:             |
+  bb.0:
+    ; CHECK-LABEL: name: test_ext_zzzi_unique
+    ; CHECK: BUNDLE implicit-def $z2, implicit-def $q2, implicit-def $d2, implicit-def $s2, implicit-def $h2, implicit-def $b2, implicit-def $b2_hi, implicit-def $h2_hi, implicit-def $s2_hi, implicit-def $d2_hi, implicit-def $q2_hi, implicit $z0, implicit killed $z1 {
+    ; CHECK-NEXT:   $z2 = MOVPRFX_ZZ $z0
+    ; CHECK-NEXT:   $z2 = EXT_ZZI internal killed $z2, killed $z1, 1
+    ; CHECK-NEXT: }
+    ; CHECK-NEXT: RET undef $lr, implicit killed $z2
+    $z2 = EXT_ZZZI killed $z0, killed $z1, 1
+    RET_ReallyLR implicit killed $z2
+...
+
+# Here the destination register is the same as the second operand,
+# we cannot use movprfx.
+---
+name:            test_ext_zzzi_not_unique
+body:             |
+  bb.0:
+    ; CHECK-LABEL: name: test_ext_zzzi_not_unique
+    ; CHECK: BUNDLE implicit-def $z2, implicit-def $q2, implicit-def $d2, implicit-def $s2, implicit-def $h2, implicit-def $b2, implicit-def $b2_hi, implicit-def $h2_hi, implicit-def $s2_hi, implicit-def $d2_hi, implicit-def $q2_hi, implicit $z0 {
+    ; CHECK-NEXT:   $z2 = ORR_ZZZ $z0, $z0
+    ; CHECK-NEXT:   $z2 = EXT_ZZI internal killed $z2, internal killed $z2, 1
+    ; CHECK-NEXT: }
+    ; CHECK-NEXT: RET undef $lr, implicit killed $z2
+    $z2 = EXT_ZZZI killed $z0, killed $z2, 1
+    RET_ReallyLR implicit killed $z2
+...
+
+---
+name:            test_ext_zzzi_same_inputs
+body:             |
+  bb.0:
+    ; CHECK-LABEL: name: test_ext_zzzi_same_inputs
+    ; CHECK: BUNDLE implicit-def $z2, implicit-def $q2, implicit-def $d2, implicit-def $s2, implicit-def $h2, implicit-def $b2, implicit-def $b2_hi, implicit-def $h2_hi, implicit-def $s2_hi, implicit-def $d2_hi, implicit-def $q2_hi, implicit killed $z0 {
+    ; CHECK-NEXT:   $z2 = MOVPRFX_ZZ $z0
+    ; CHECK-NEXT:   $z2 = EXT_ZZI internal killed $z2, killed $z0, 1
+    ; CHECK-NEXT: }
+    ; CHECK-NEXT: RET undef $lr, implicit killed $z2
+    $z2 = EXT_ZZZI killed $z0, killed $z0, 1
+    RET_ReallyLR implicit killed $z2
+...
+
+---
+name:            test_ext_zzzi_already_destructive
+body:             |
+  bb.0:
+    ; CHECK-LABEL: name: test_ext_zzzi_already_destructive
+    ; CHECK: $z2 = EXT_ZZI killed $z2, killed $z1, 1
+    ; CHECK-NEXT: RET undef $lr, implicit killed $z2
+    $z2 = EXT_ZZZI killed $z2, killed $z1, 1
+    RET_ReallyLR implicit killed $z2
+...
+
+---
+name:            test_ext_zzzi_unique_implicit_ops
+body:             |
+  bb.0:
+    ; CHECK-LABEL: name: test_ext_zzzi_unique_implicit_ops
+    ; CHECK: BUNDLE implicit-def $z2, implicit-def $q2, implicit-def $d2, implicit-def $s2, implicit-def $h2, implicit-def $b2, implicit-def $b2_hi, implicit-def $h2_hi, implicit-def $s2_hi, implicit-def $d2_hi, implicit-def $q2_hi, implicit $z0, implicit killed $q0, implicit killed $q1, implicit killed $z1 {
+    ; CHECK-NEXT:   $z2 = MOVPRFX_ZZ $z0, implicit killed $q0, implicit killed $q1
+    ; CHECK-NEXT:   $z2 = EXT_ZZI internal killed $z2, killed $z1, 1, implicit-def $q2
+    ; CHECK-NEXT: }
+    ; CHECK-NEXT: RET undef $lr, implicit killed $q2
+    $z2 = EXT_ZZZI killed $z0, killed $z1, 1, implicit-def $q2, implicit killed $q0, implicit killed $q1
+    RET_ReallyLR implicit killed $q2
+...
+
+---
+name:            test_ext_zzzi_undef
+body:             |
+  bb.0:
+    ; CHECK-LABEL: name: test_ext_zzzi_undef
+    ; CHECK: BUNDLE implicit-def $z2, implicit-def $q2, implicit-def $d2, implicit-def $s2, implicit-def $h2, implicit-def $b2, implicit-def $b2_hi, implicit-def $h2_hi, implicit-def $s2_hi, implicit-def $d2_hi, implicit-def $q2_hi, implicit undef $z0, implicit undef $z1 {
+    ; CHECK-NEXT:   $z2 = MOVPRFX_ZZ undef $z0
+    ; CHECK-NEXT:   $z2 = EXT_ZZI internal killed $z2, undef $z1, 1
+    ; CHECK-NEXT: }
+    ; CHECK-NEXT: RET undef $lr, implicit killed $z2
+    $z2 = EXT_ZZZI undef $z0, undef $z1, 1
+    RET_ReallyLR implicit killed $z2
+...

github-actions · 2025-08-07T17:19:37Z

⚠️ C/C++ code formatter, clang-format found issues in your code. ⚠️

You can test this locally with the following command:

git-clang-format --diff HEAD~1 HEAD --extensions cpp,h -- llvm/lib/Target/AArch64/AArch64ExpandPseudoInsts.cpp llvm/lib/Target/AArch64/AArch64InstrInfo.h

View the diff from clang-format here.

diff --git a/llvm/lib/Target/AArch64/AArch64InstrInfo.h b/llvm/lib/Target/AArch64/AArch64InstrInfo.h
index b903cd90c..c65f59cba 100644
--- a/llvm/lib/Target/AArch64/AArch64InstrInfo.h
+++ b/llvm/lib/Target/AArch64/AArch64InstrInfo.h
@@ -810,18 +810,18 @@ enum ElementSizeType {
 };
 
 enum DestructiveInstType {
-  DestructiveInstTypeMask       = TSFLAG_DESTRUCTIVE_INST_TYPE(0xf),
-  NotDestructive                = TSFLAG_DESTRUCTIVE_INST_TYPE(0x0),
-  DestructiveOther              = TSFLAG_DESTRUCTIVE_INST_TYPE(0x1),
-  DestructiveUnary              = TSFLAG_DESTRUCTIVE_INST_TYPE(0x2),
-  DestructiveBinaryImm          = TSFLAG_DESTRUCTIVE_INST_TYPE(0x3),
-  DestructiveBinaryShImmUnpred  = TSFLAG_DESTRUCTIVE_INST_TYPE(0x4),
-  DestructiveBinary             = TSFLAG_DESTRUCTIVE_INST_TYPE(0x5),
-  DestructiveBinaryComm         = TSFLAG_DESTRUCTIVE_INST_TYPE(0x6),
-  DestructiveBinaryCommWithRev  = TSFLAG_DESTRUCTIVE_INST_TYPE(0x7),
+  DestructiveInstTypeMask = TSFLAG_DESTRUCTIVE_INST_TYPE(0xf),
+  NotDestructive = TSFLAG_DESTRUCTIVE_INST_TYPE(0x0),
+  DestructiveOther = TSFLAG_DESTRUCTIVE_INST_TYPE(0x1),
+  DestructiveUnary = TSFLAG_DESTRUCTIVE_INST_TYPE(0x2),
+  DestructiveBinaryImm = TSFLAG_DESTRUCTIVE_INST_TYPE(0x3),
+  DestructiveBinaryShImmUnpred = TSFLAG_DESTRUCTIVE_INST_TYPE(0x4),
+  DestructiveBinary = TSFLAG_DESTRUCTIVE_INST_TYPE(0x5),
+  DestructiveBinaryComm = TSFLAG_DESTRUCTIVE_INST_TYPE(0x6),
+  DestructiveBinaryCommWithRev = TSFLAG_DESTRUCTIVE_INST_TYPE(0x7),
   DestructiveTernaryCommWithRev = TSFLAG_DESTRUCTIVE_INST_TYPE(0x8),
-  Destructive2xRegImmUnpred     = TSFLAG_DESTRUCTIVE_INST_TYPE(0x9),
-  DestructiveUnaryPassthru      = TSFLAG_DESTRUCTIVE_INST_TYPE(0xa),
+  Destructive2xRegImmUnpred = TSFLAG_DESTRUCTIVE_INST_TYPE(0x9),
+  DestructiveUnaryPassthru = TSFLAG_DESTRUCTIVE_INST_TYPE(0xa),
 };
 
 enum FalseLaneType {

llvm/lib/Target/AArch64/AArch64InstrFormats.td

gbossu · 2025-08-07T17:25:43Z

llvm/test/CodeGen/AArch64/expand-constructive-zpzz.mir

+# RUN: llc -mtriple=aarch64 -mattr=+sve -run-pass=aarch64-expand-pseudo -verify-machineinstrs %s -o - | FileCheck %s
+
+# Test the expansion of constructive binary operations into their
+# destructive counterparts.


Note these aren't new instructions, but there are no MIR test for those "constructive pseudos", only end-to-end .ll tests. So I thought I would add a couple test cases for existing instructions to show the current behaviour, especially around implicit operands.

I have removed that test file as it was unrelated.

I'll probably create an NFC PR later where I document all the different types of "destructive instructions" and add tests for them, because I have done that locally for my own understanding anyway.

llvm/test/CodeGen/AArch64/expand-constructive-zzzi.mir

SamTebbs33

LGTM

llvm/test/CodeGen/AArch64/expand-constructive-zzzi.mir

llvm/lib/Target/AArch64/AArch64ExpandPseudoInsts.cpp

llvm/lib/Target/AArch64/AArch64InstrFormats.td

llvm/lib/Target/AArch64/AArch64ExpandPseudoInsts.cpp

llvm/lib/Target/AArch64/SVEInstrFormats.td

llvm/test/CodeGen/AArch64/expand-constructive-zzi.mir

It will get expanded into MOVPRFX_ZZ and EXT_ZZI by the AArch64ExpandPseudo pass. This instruction takes a single Z register as input, as opposed to the existing destructive EXT_ZZI instruction. Note this patch only defines the pseudo, it isn't used in any ISel pattern yet. It will later be used for vector.extract.

- More consistent .td - Clearer comment in AArch64ExpandPseudoInsts.cpp

Due to #142563

gbossu · 2025-08-14T12:27:42Z

Rebased and updated CHECK lines

llvmbot added the backend:AArch64 label Aug 7, 2025

This was referenced Aug 7, 2025

[AArch64][ISel] Add unary vector_splice tests (NFC) #152553

Merged

[AArch64][ISel] Select constructive EXT_ZZI pseudo instruction #152554

Merged

gbossu requested review from SamTebbs33, paulwalker-arm and sdesmalen-arm August 7, 2025 17:21

gbossu commented Aug 7, 2025

View reviewed changes

llvm/lib/Target/AArch64/AArch64InstrFormats.td Outdated Show resolved Hide resolved

gbossu commented Aug 7, 2025

View reviewed changes

llvm/test/CodeGen/AArch64/expand-constructive-zzzi.mir Outdated Show resolved Hide resolved

SamTebbs33 approved these changes Aug 8, 2025

View reviewed changes

sdesmalen-arm reviewed Aug 11, 2025

View reviewed changes

llvm/test/CodeGen/AArch64/expand-constructive-zzzi.mir Outdated Show resolved Hide resolved

paulwalker-arm reviewed Aug 11, 2025

View reviewed changes

llvm/lib/Target/AArch64/AArch64ExpandPseudoInsts.cpp Outdated Show resolved Hide resolved

gbossu force-pushed the users/gbossu.vector.extract.movprfx.1 branch from 1d4c1a8 to 1ccb570 Compare August 12, 2025 15:43

gbossu changed the title ~~[AArch64] Define constructive EXT_ZZZI pseudo instruction~~ [AArch64] Define constructive EXT_ZZI pseudo instruction Aug 12, 2025

paulwalker-arm reviewed Aug 13, 2025

View reviewed changes

gbossu added 3 commits August 14, 2025 10:29

Address comments

8a891a7

- More consistent .td - Clearer comment in AArch64ExpandPseudoInsts.cpp

Update test after rebase

3293d18

Due to #142563

gbossu force-pushed the users/gbossu.vector.extract.movprfx.1 branch from 7a7bbc3 to 3293d18 Compare August 14, 2025 12:27

paulwalker-arm approved these changes Aug 14, 2025

View reviewed changes

gbossu merged commit fdd2d4d into main Aug 15, 2025
8 of 9 checks passed

gbossu deleted the users/gbossu.vector.extract.movprfx.1 branch August 15, 2025 08:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[AArch64] Define constructive EXT_ZZI pseudo instruction #152552

[AArch64] Define constructive EXT_ZZI pseudo instruction #152552

Uh oh!

gbossu commented Aug 7, 2025 •

edited

Loading

Uh oh!

llvmbot commented Aug 7, 2025

Uh oh!

github-actions bot commented Aug 7, 2025 •

edited

Loading

Uh oh!

Uh oh!

gbossu Aug 7, 2025

Uh oh!

gbossu Aug 12, 2025

Uh oh!

Uh oh!

SamTebbs33 left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gbossu commented Aug 14, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

[AArch64] Define constructive EXT_ZZI pseudo instruction #152552

[AArch64] Define constructive EXT_ZZI pseudo instruction #152552

Uh oh!

Conversation

gbossu commented Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

llvmbot commented Aug 7, 2025

Uh oh!

github-actions bot commented Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

gbossu Aug 7, 2025

Choose a reason for hiding this comment

Uh oh!

gbossu Aug 12, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

SamTebbs33 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gbossu commented Aug 14, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

gbossu commented Aug 7, 2025 •

edited

Loading

github-actions bot commented Aug 7, 2025 •

edited

Loading