Skip to content

[AArch64] match TRN starting from undef elements #167955

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Nov 19, 2025
Merged

[AArch64] match TRN starting from undef elements #167955

merged 2 commits into from
Nov 19, 2025

Conversation

@ginsbach
Copy link
Contributor

@ginsbach ginsbach commented Nov 13, 2025 •
edited
Loading

When the first element of a trn mask is undef, the isTRNMask function assumes WhichResult = 1. That has a 50% chance of being wrong, so we fail to match some valid trn1/trn2.

This patch introduces a more precise test to determine the correct value of WhichResult, based on corresponding code in the isZIPMask and isUZPMask functions.

@github-actions
Copy link

github-actions bot commented Nov 13, 2025

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

@ginsbach ginsbach marked this pull request as ready for review November 13, 2025 21:22
@llvmbot
Copy link
Member

llvmbot commented Nov 13, 2025

@llvm/pr-subscribers-backend-aarch64

Author: Philip Ginsbach-Chen (ginsbach)

Changes

When the first element of a trn mask is undef, the isTRNMask function assumes that the value for "WhichResult" should be 1. That has a 50% chance of being wrong, so we fail to match some valid trn1/trn2. This patch introduces a more precise test to determine the correct value of "WhichResult", based on corresponding code in the isZIPMask and isUZPMask functions.

Patch is 26.93 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/167955.diff

4 Files Affected:

  • (modified) llvm/lib/Target/AArch64/AArch64PerfectShuffle.h (+17-2)
  • (modified) llvm/test/CodeGen/AArch64/insert-extend.ll (+10-10)
  • (modified) llvm/test/CodeGen/AArch64/reduce-shuffle.ll (+227-229)
  • (modified) llvm/test/CodeGen/AArch64/vldn_shuffle.ll (+10-10)
diff --git a/llvm/lib/Target/AArch64/AArch64PerfectShuffle.h b/llvm/lib/Target/AArch64/AArch64PerfectShuffle.h
index f7beca1b8b77e..c28cbf2bc63c2 100644
--- a/llvm/lib/Target/AArch64/AArch64PerfectShuffle.h
+++ b/llvm/lib/Target/AArch64/AArch64PerfectShuffle.h
@@ -6685,15 +6685,30 @@ inline bool isUZPMask(ArrayRef<int> M, unsigned NumElts,
 ///  <0, 8, 2, 10, 4, 12, 6, 14> or
 ///  <1, 9, 3, 11, 5, 13, 7, 15>
 inline bool isTRNMask(ArrayRef<int> M, unsigned NumElts,
-                      unsigned &WhichResult) {
+                      unsigned &WhichResultOut) {
   if (NumElts % 2 != 0)
     return false;
-  WhichResult = (M[0] == 0 ? 0 : 1);
+  // Check the first non-undef element for trn1 vs trn2.
+  unsigned WhichResult = 2;
+  for (unsigned i = 0; i != NumElts; i += 2) {
+    if (M[i] >= 0) {
+      WhichResult = ((unsigned)M[i] == i ? 0 : 1);
+      break;
+    }
+    if (M[i + 1] >= 0) {
+      WhichResult = ((unsigned)M[i + 1] == i + NumElts ? 0 : 1);
+      break;
+    }
+  }
+  if (WhichResult == 2)
+    return false;
+
   for (unsigned i = 0; i < NumElts; i += 2) {
     if ((M[i] >= 0 && (unsigned)M[i] != i + WhichResult) ||
         (M[i + 1] >= 0 && (unsigned)M[i + 1] != i + NumElts + WhichResult))
       return false;
   }
+  WhichResultOut = WhichResult;
   return true;
 }
 
diff --git a/llvm/test/CodeGen/AArch64/insert-extend.ll b/llvm/test/CodeGen/AArch64/insert-extend.ll
index 851fb0d03e8aa..e128abf4d7376 100644
--- a/llvm/test/CodeGen/AArch64/insert-extend.ll
+++ b/llvm/test/CodeGen/AArch64/insert-extend.ll
@@ -85,24 +85,24 @@ define i32 @large(ptr nocapture noundef readonly %p1, i32 noundef %st1, ptr noca
 ; CHECK-NEXT:    addp v2.4s, v3.4s, v2.4s
 ; CHECK-NEXT:    zip1 v16.4s, v5.4s, v4.4s
 ; CHECK-NEXT:    sub v7.4s, v3.4s, v7.4s
+; CHECK-NEXT:    trn1 v4.4s, v5.4s, v4.4s
 ; CHECK-NEXT:    zip2 v3.4s, v6.4s, v7.4s
 ; CHECK-NEXT:    mov v6.s[1], v7.s[0]
-; CHECK-NEXT:    ext v7.16b, v5.16b, v16.16b, #8
-; CHECK-NEXT:    mov v5.s[3], v4.s[2]
-; CHECK-NEXT:    ext v4.16b, v2.16b, v2.16b, #8
-; CHECK-NEXT:    mov v6.d[1], v7.d[1]
-; CHECK-NEXT:    mov v3.d[1], v5.d[1]
-; CHECK-NEXT:    uzp1 v1.4s, v4.4s, v0.4s
-; CHECK-NEXT:    uzp2 v4.4s, v4.4s, v0.4s
+; CHECK-NEXT:    ext v7.16b, v2.16b, v2.16b, #8
+; CHECK-NEXT:    ext v5.16b, v5.16b, v16.16b, #8
+; CHECK-NEXT:    mov v3.d[1], v4.d[1]
+; CHECK-NEXT:    uzp1 v1.4s, v7.4s, v0.4s
+; CHECK-NEXT:    uzp2 v4.4s, v7.4s, v0.4s
+; CHECK-NEXT:    mov v6.d[1], v5.d[1]
 ; CHECK-NEXT:    addp v0.4s, v2.4s, v0.4s
+; CHECK-NEXT:    sub v1.4s, v1.4s, v4.4s
+; CHECK-NEXT:    rev64 v7.4s, v0.4s
 ; CHECK-NEXT:    add v5.4s, v3.4s, v6.4s
 ; CHECK-NEXT:    sub v3.4s, v6.4s, v3.4s
-; CHECK-NEXT:    rev64 v7.4s, v0.4s
-; CHECK-NEXT:    sub v1.4s, v1.4s, v4.4s
+; CHECK-NEXT:    rev64 v2.4s, v1.4s
 ; CHECK-NEXT:    rev64 v4.4s, v5.4s
 ; CHECK-NEXT:    rev64 v6.4s, v3.4s
 ; CHECK-NEXT:    addp v16.4s, v0.4s, v5.4s
-; CHECK-NEXT:    rev64 v2.4s, v1.4s
 ; CHECK-NEXT:    sub v0.4s, v0.4s, v7.4s
 ; CHECK-NEXT:    zip1 v21.4s, v16.4s, v16.4s
 ; CHECK-NEXT:    sub v4.4s, v5.4s, v4.4s
diff --git a/llvm/test/CodeGen/AArch64/reduce-shuffle.ll b/llvm/test/CodeGen/AArch64/reduce-shuffle.ll
index 325ab444205bf..354edc4ff7ab4 100644
--- a/llvm/test/CodeGen/AArch64/reduce-shuffle.ll
+++ b/llvm/test/CodeGen/AArch64/reduce-shuffle.ll
@@ -4,126 +4,126 @@
 define i32 @v1(ptr nocapture noundef readonly %p1, i32 noundef %i1, ptr nocapture noundef readonly %p2, i32 noundef %i2) {
 ; CHECK-LABEL: v1:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    // kill: def $w3 killed $w3 def $x3
 ; CHECK-NEXT:    // kill: def $w1 killed $w1 def $x1
 ; CHECK-NEXT:    sxtw x8, w1
+; CHECK-NEXT:    // kill: def $w3 killed $w3 def $x3
 ; CHECK-NEXT:    sxtw x9, w3
 ; CHECK-NEXT:    ldr d0, [x0]
-; CHECK-NEXT:    ldr d1, [x2]
+; CHECK-NEXT:    ldr d5, [x2]
 ; CHECK-NEXT:    add x10, x0, x8
 ; CHECK-NEXT:    add x11, x2, x9
-; CHECK-NEXT:    ldr d2, [x10]
-; CHECK-NEXT:    add x10, x10, x8
-; CHECK-NEXT:    ldr d3, [x11]
-; CHECK-NEXT:    add x11, x11, x9
-; CHECK-NEXT:    ldr d4, [x10]
-; CHECK-NEXT:    ldr d6, [x10, x8]
-; CHECK-NEXT:    ldr d5, [x11]
-; CHECK-NEXT:    ldr d7, [x11, x9]
-; CHECK-NEXT:    usubl v0.8h, v0.8b, v1.8b
-; CHECK-NEXT:    usubl v1.8h, v2.8b, v3.8b
-; CHECK-NEXT:    usubl v2.8h, v4.8b, v5.8b
+; CHECK-NEXT:    add x12, x10, x8
+; CHECK-NEXT:    ldr d6, [x10]
+; CHECK-NEXT:    ldr d7, [x11]
+; CHECK-NEXT:    ldr d1, [x12, x8]
+; CHECK-NEXT:    add x8, x11, x9
+; CHECK-NEXT:    ldr d2, [x12]
+; CHECK-NEXT:    ldr d3, [x8, x9]
+; CHECK-NEXT:    ldr d4, [x8]
+; CHECK-NEXT:    usubl v0.8h, v0.8b, v5.8b
+; CHECK-NEXT:    usubl v2.8h, v2.8b, v4.8b
+; CHECK-NEXT:    usubl v1.8h, v1.8b, v3.8b
 ; CHECK-NEXT:    usubl v3.8h, v6.8b, v7.8b
-; CHECK-NEXT:    shll2 v4.4s, v0.8h, #16
-; CHECK-NEXT:    shll2 v5.4s, v1.8h, #16
-; CHECK-NEXT:    shll2 v6.4s, v3.8h, #16
-; CHECK-NEXT:    shll2 v7.4s, v2.8h, #16
-; CHECK-NEXT:    saddw v0.4s, v4.4s, v0.4h
-; CHECK-NEXT:    saddw v1.4s, v5.4s, v1.4h
-; CHECK-NEXT:    saddw v3.4s, v6.4s, v3.4h
-; CHECK-NEXT:    saddw v2.4s, v7.4s, v2.4h
-; CHECK-NEXT:    zip1 v4.4s, v1.4s, v0.4s
-; CHECK-NEXT:    zip2 v6.4s, v1.4s, v0.4s
-; CHECK-NEXT:    uzp2 v5.4s, v3.4s, v2.4s
-; CHECK-NEXT:    mov v7.16b, v2.16b
-; CHECK-NEXT:    ext v17.16b, v3.16b, v3.16b, #12
-; CHECK-NEXT:    zip2 v18.4s, v3.4s, v2.4s
-; CHECK-NEXT:    ext v16.16b, v1.16b, v4.16b, #8
-; CHECK-NEXT:    mov v1.s[3], v0.s[2]
-; CHECK-NEXT:    mov v7.s[1], v3.s[0]
-; CHECK-NEXT:    uzp2 v0.4s, v5.4s, v3.4s
-; CHECK-NEXT:    zip2 v5.4s, v2.4s, v3.4s
-; CHECK-NEXT:    mov v3.s[0], v2.s[1]
+; CHECK-NEXT:    shll2 v6.4s, v0.8h, #16
+; CHECK-NEXT:    shll2 v4.4s, v1.8h, #16
+; CHECK-NEXT:    shll2 v5.4s, v2.8h, #16
+; CHECK-NEXT:    shll2 v7.4s, v3.8h, #16
+; CHECK-NEXT:    saddw v0.4s, v6.4s, v0.4h
+; CHECK-NEXT:    saddw v1.4s, v4.4s, v1.4h
+; CHECK-NEXT:    saddw v2.4s, v5.4s, v2.4h
+; CHECK-NEXT:    saddw v3.4s, v7.4s, v3.4h
+; CHECK-NEXT:    uzp2 v4.4s, v1.4s, v2.4s
+; CHECK-NEXT:    zip1 v5.4s, v3.4s, v0.4s
+; CHECK-NEXT:    mov v6.16b, v2.16b
+; CHECK-NEXT:    trn1 v7.4s, v3.4s, v0.4s
+; CHECK-NEXT:    zip2 v0.4s, v3.4s, v0.4s
+; CHECK-NEXT:    ext v17.16b, v1.16b, v1.16b, #12
+; CHECK-NEXT:    zip2 v18.4s, v1.4s, v2.4s
+; CHECK-NEXT:    zip2 v16.4s, v2.4s, v1.4s
+; CHECK-NEXT:    mov v6.s[1], v1.s[0]
+; CHECK-NEXT:    uzp2 v4.4s, v4.4s, v1.4s
+; CHECK-NEXT:    ext v3.16b, v3.16b, v5.16b, #8
+; CHECK-NEXT:    mov v1.s[0], v2.s[1]
 ; CHECK-NEXT:    ext v2.16b, v2.16b, v17.16b, #12
-; CHECK-NEXT:    mov v18.d[1], v1.d[1]
-; CHECK-NEXT:    mov v7.d[1], v16.d[1]
-; CHECK-NEXT:    mov v0.d[1], v6.d[1]
-; CHECK-NEXT:    mov v3.d[1], v4.d[1]
-; CHECK-NEXT:    mov v5.d[1], v1.d[1]
-; CHECK-NEXT:    mov v2.d[1], v6.d[1]
-; CHECK-NEXT:    add v0.4s, v0.4s, v18.4s
-; CHECK-NEXT:    add v1.4s, v3.4s, v7.4s
-; CHECK-NEXT:    sub v3.4s, v7.4s, v3.4s
-; CHECK-NEXT:    sub v2.4s, v5.4s, v2.4s
+; CHECK-NEXT:    mov v18.d[1], v7.d[1]
+; CHECK-NEXT:    mov v16.d[1], v7.d[1]
+; CHECK-NEXT:    mov v4.d[1], v0.d[1]
+; CHECK-NEXT:    mov v6.d[1], v3.d[1]
+; CHECK-NEXT:    mov v1.d[1], v5.d[1]
+; CHECK-NEXT:    mov v2.d[1], v0.d[1]
+; CHECK-NEXT:    add v0.4s, v4.4s, v18.4s
+; CHECK-NEXT:    add v3.4s, v1.4s, v6.4s
+; CHECK-NEXT:    sub v1.4s, v6.4s, v1.4s
+; CHECK-NEXT:    sub v2.4s, v16.4s, v2.4s
 ; CHECK-NEXT:    rev64 v4.4s, v0.4s
-; CHECK-NEXT:    rev64 v6.4s, v1.4s
-; CHECK-NEXT:    sub v5.4s, v3.4s, v2.4s
-; CHECK-NEXT:    add v2.4s, v2.4s, v3.4s
+; CHECK-NEXT:    rev64 v5.4s, v3.4s
+; CHECK-NEXT:    sub v6.4s, v1.4s, v2.4s
+; CHECK-NEXT:    add v1.4s, v2.4s, v1.4s
 ; CHECK-NEXT:    mov v4.d[1], v0.d[1]
-; CHECK-NEXT:    mov v6.d[1], v1.d[1]
-; CHECK-NEXT:    rev64 v3.4s, v5.4s
-; CHECK-NEXT:    rev64 v7.4s, v2.4s
-; CHECK-NEXT:    sub v1.4s, v1.4s, v4.4s
-; CHECK-NEXT:    add v0.4s, v0.4s, v6.4s
-; CHECK-NEXT:    sub v3.4s, v5.4s, v3.4s
-; CHECK-NEXT:    addp v4.4s, v1.4s, v5.4s
-; CHECK-NEXT:    sub v5.4s, v2.4s, v7.4s
-; CHECK-NEXT:    addp v2.4s, v0.4s, v2.4s
-; CHECK-NEXT:    rev64 v6.4s, v0.4s
+; CHECK-NEXT:    mov v5.d[1], v3.d[1]
+; CHECK-NEXT:    rev64 v2.4s, v6.4s
 ; CHECK-NEXT:    rev64 v7.4s, v1.4s
-; CHECK-NEXT:    ext v16.16b, v4.16b, v3.16b, #4
-; CHECK-NEXT:    ext v17.16b, v2.16b, v5.16b, #4
+; CHECK-NEXT:    sub v3.4s, v3.4s, v4.4s
+; CHECK-NEXT:    add v0.4s, v0.4s, v5.4s
+; CHECK-NEXT:    sub v2.4s, v6.4s, v2.4s
+; CHECK-NEXT:    sub v5.4s, v1.4s, v7.4s
+; CHECK-NEXT:    addp v4.4s, v3.4s, v6.4s
+; CHECK-NEXT:    addp v1.4s, v0.4s, v1.4s
+; CHECK-NEXT:    rev64 v6.4s, v0.4s
+; CHECK-NEXT:    rev64 v7.4s, v3.4s
+; CHECK-NEXT:    ext v16.16b, v4.16b, v2.16b, #4
+; CHECK-NEXT:    ext v17.16b, v1.16b, v5.16b, #4
 ; CHECK-NEXT:    sub v0.4s, v0.4s, v6.4s
-; CHECK-NEXT:    sub v1.4s, v1.4s, v7.4s
-; CHECK-NEXT:    mov v7.16b, v3.16b
+; CHECK-NEXT:    sub v3.4s, v3.4s, v7.4s
+; CHECK-NEXT:    mov v7.16b, v2.16b
 ; CHECK-NEXT:    zip2 v6.4s, v16.4s, v4.4s
 ; CHECK-NEXT:    mov v16.16b, v5.16b
-; CHECK-NEXT:    zip2 v17.4s, v17.4s, v2.4s
-; CHECK-NEXT:    ext v18.16b, v0.16b, v2.16b, #4
+; CHECK-NEXT:    zip2 v17.4s, v17.4s, v1.4s
+; CHECK-NEXT:    ext v18.16b, v0.16b, v1.16b, #4
 ; CHECK-NEXT:    mov v7.s[2], v4.s[3]
-; CHECK-NEXT:    mov v21.16b, v1.16b
-; CHECK-NEXT:    mov v16.s[2], v2.s[3]
+; CHECK-NEXT:    mov v21.16b, v3.16b
+; CHECK-NEXT:    mov v16.s[2], v1.s[3]
 ; CHECK-NEXT:    ext v5.16b, v5.16b, v17.16b, #12
-; CHECK-NEXT:    zip1 v17.4s, v2.4s, v2.4s
-; CHECK-NEXT:    ext v3.16b, v3.16b, v6.16b, #12
+; CHECK-NEXT:    zip1 v17.4s, v1.4s, v1.4s
+; CHECK-NEXT:    ext v2.16b, v2.16b, v6.16b, #12
 ; CHECK-NEXT:    ext v18.16b, v18.16b, v18.16b, #4
 ; CHECK-NEXT:    mov v19.16b, v7.16b
-; CHECK-NEXT:    ext v6.16b, v1.16b, v4.16b, #8
+; CHECK-NEXT:    ext v6.16b, v3.16b, v4.16b, #8
 ; CHECK-NEXT:    mov v21.s[2], v4.s[1]
 ; CHECK-NEXT:    mov v20.16b, v16.16b
 ; CHECK-NEXT:    mov v19.s[1], v4.s[2]
 ; CHECK-NEXT:    trn2 v0.4s, v17.4s, v0.4s
 ; CHECK-NEXT:    sub v16.4s, v16.4s, v5.4s
 ; CHECK-NEXT:    mov v17.16b, v18.16b
-; CHECK-NEXT:    ext v1.16b, v6.16b, v1.16b, #4
-; CHECK-NEXT:    sub v7.4s, v7.4s, v3.4s
-; CHECK-NEXT:    mov v20.s[1], v2.s[2]
-; CHECK-NEXT:    mov v17.s[0], v2.s[1]
-; CHECK-NEXT:    mov v2.16b, v21.16b
-; CHECK-NEXT:    add v3.4s, v19.4s, v3.4s
-; CHECK-NEXT:    uzp2 v1.4s, v6.4s, v1.4s
+; CHECK-NEXT:    ext v3.16b, v6.16b, v3.16b, #4
+; CHECK-NEXT:    sub v7.4s, v7.4s, v2.4s
+; CHECK-NEXT:    mov v20.s[1], v1.s[2]
+; CHECK-NEXT:    mov v17.s[0], v1.s[1]
+; CHECK-NEXT:    mov v1.16b, v21.16b
+; CHECK-NEXT:    add v2.4s, v19.4s, v2.4s
+; CHECK-NEXT:    uzp2 v3.4s, v6.4s, v3.4s
 ; CHECK-NEXT:    add v5.4s, v20.4s, v5.4s
-; CHECK-NEXT:    mov v2.s[1], v4.s[0]
+; CHECK-NEXT:    mov v1.s[1], v4.s[0]
 ; CHECK-NEXT:    sub v4.4s, v0.4s, v18.4s
-; CHECK-NEXT:    mov v3.d[1], v7.d[1]
+; CHECK-NEXT:    mov v2.d[1], v7.d[1]
 ; CHECK-NEXT:    add v0.4s, v0.4s, v17.4s
 ; CHECK-NEXT:    mov v5.d[1], v16.d[1]
-; CHECK-NEXT:    sub v6.4s, v21.4s, v1.4s
-; CHECK-NEXT:    add v1.4s, v2.4s, v1.4s
+; CHECK-NEXT:    sub v6.4s, v21.4s, v3.4s
+; CHECK-NEXT:    add v1.4s, v1.4s, v3.4s
 ; CHECK-NEXT:    mov v0.d[1], v4.d[1]
-; CHECK-NEXT:    cmlt v4.8h, v3.8h, #0
-; CHECK-NEXT:    cmlt v2.8h, v5.8h, #0
+; CHECK-NEXT:    cmlt v4.8h, v2.8h, #0
+; CHECK-NEXT:    cmlt v3.8h, v5.8h, #0
 ; CHECK-NEXT:    mov v1.d[1], v6.d[1]
-; CHECK-NEXT:    add v3.4s, v4.4s, v3.4s
+; CHECK-NEXT:    add v2.4s, v4.4s, v2.4s
 ; CHECK-NEXT:    cmlt v6.8h, v0.8h, #0
-; CHECK-NEXT:    add v5.4s, v2.4s, v5.4s
-; CHECK-NEXT:    eor v3.16b, v3.16b, v4.16b
+; CHECK-NEXT:    add v5.4s, v3.4s, v5.4s
+; CHECK-NEXT:    eor v2.16b, v2.16b, v4.16b
 ; CHECK-NEXT:    cmlt v7.8h, v1.8h, #0
 ; CHECK-NEXT:    add v0.4s, v6.4s, v0.4s
-; CHECK-NEXT:    eor v2.16b, v5.16b, v2.16b
+; CHECK-NEXT:    eor v3.16b, v5.16b, v3.16b
 ; CHECK-NEXT:    add v1.4s, v7.4s, v1.4s
 ; CHECK-NEXT:    eor v0.16b, v0.16b, v6.16b
-; CHECK-NEXT:    add v2.4s, v2.4s, v3.4s
+; CHECK-NEXT:    add v2.4s, v3.4s, v2.4s
 ; CHECK-NEXT:    eor v1.16b, v1.16b, v7.16b
 ; CHECK-NEXT:    add v0.4s, v0.4s, v2.4s
 ; CHECK-NEXT:    add v0.4s, v0.4s, v1.4s
@@ -227,114 +227,112 @@ entry:
 define i32 @v2(ptr nocapture noundef readonly %p1, i32 noundef %i1, ptr nocapture noundef readonly %p2, i32 noundef %i2) {
 ; CHECK-LABEL: v2:
 ; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    // kill: def $w3 killed $w3 def $x3
 ; CHECK-NEXT:    // kill: def $w1 killed $w1 def $x1
 ; CHECK-NEXT:    sxtw x8, w1
-; CHECK-NEXT:    // kill: def $w3 killed $w3 def $x3
 ; CHECK-NEXT:    sxtw x9, w3
-; CHECK-NEXT:    ldr d4, [x0]
-; CHECK-NEXT:    ldr d5, [x2]
+; CHECK-NEXT:    ldr d0, [x0]
+; CHECK-NEXT:    ldr d1, [x2]
 ; CHECK-NEXT:    add x10, x0, x8
 ; CHECK-NEXT:    add x11, x2, x9
-; CHECK-NEXT:    add x12, x10, x8
+; CHECK-NEXT:    ldr d2, [x10]
+; CHECK-NEXT:    add x10, x10, x8
+; CHECK-NEXT:    ldr d3, [x11]
+; CHECK-NEXT:    add x11, x11, x9
+; CHECK-NEXT:    ldr d4, [x10, x8]
 ; CHECK-NEXT:    ldr d6, [x10]
+; CHECK-NEXT:    ldr d5, [x11, x9]
 ; CHECK-NEXT:    ldr d7, [x11]
-; CHECK-NEXT:    ldr d0, [x12, x8]
-; CHECK-NEXT:    add x8, x11, x9
-; CHECK-NEXT:    ldr d1, [x12]
-; CHECK-NEXT:    ldr d2, [x8, x9]
-; CHECK-NEXT:    ldr d3, [x8]
-; CHECK-NEXT:    usubl v1.8h, v1.8b, v3.8b
-; CHECK-NEXT:    usubl v0.8h, v0.8b, v2.8b
-; CHECK-NEXT:    usubl v3.8h, v6.8b, v7.8b
+; CHECK-NEXT:    usubl v0.8h, v0.8b, v1.8b
+; CHECK-NEXT:    usubl v1.8h, v2.8b, v3.8b
 ; CHECK-NEXT:    usubl v2.8h, v4.8b, v5.8b
+; CHECK-NEXT:    usubl v3.8h, v6.8b, v7.8b
 ; CHECK-NEXT:    shll2 v4.4s, v0.8h, #16
 ; CHECK-NEXT:    shll2 v5.4s, v1.8h, #16
-; CHECK-NEXT:    shll2 v7.4s, v3.8h, #16
 ; CHECK-NEXT:    shll2 v6.4s, v2.8h, #16
+; CHECK-NEXT:    shll2 v7.4s, v3.8h, #16
 ; CHECK-NEXT:    saddw v0.4s, v4.4s, v0.4h
 ; CHECK-NEXT:    saddw v1.4s, v5.4s, v1.4h
-; CHECK-NEXT:    saddw v3.4s, v7.4s, v3.4h
 ; CHECK-NEXT:    saddw v2.4s, v6.4s, v2.4h
-; CHECK-NEXT:    uzp2 v4.4s, v0.4s, v1.4s
-; CHECK-NEXT:    mov v7.16b, v3.16b
-; CHECK-NEXT:    mov v17.16b, v1.16b
-; CHECK-NEXT:    zip1 v5.4s, v3.4s, v2.4s
-; CHECK-NEXT:    zip2 v6.4s, v3.4s, v2.4s
-; CHECK-NEXT:    zip2 v16.4s, v0.4s, v1.4s
-; CHECK-NEXT:    ext v18.16b, v0.16b, v0.16b, #12
-; CHECK-NEXT:    mov v7.s[3], v2.s[2]
-; CHECK-NEXT:    mov v17.s[1], v0.s[0]
-; CHECK-NEXT:    uzp2 v2.4s, v4.4s, v0.4s
-; CHECK-NEXT:    mov v4.16b, v0.16b
+; CHECK-NEXT:    saddw v3.4s, v7.4s, v3.4h
+; CHECK-NEXT:    zip1 v4.4s, v1.4s, v0.4s
+; CHECK-NEXT:    trn1 v18.4s, v1.4s, v0.4s
 ; CHECK-NEXT:    zip2 v0.4s, v1.4s, v0.4s
-; CHECK-NEXT:    ext v3.16b, v3.16b, v5.16b, #8
-; CHECK-NEXT:    mov v4.s[0], v1.s[1]
+; CHECK-NEXT:    uzp2 v5.4s, v2.4s, v3.4s
+; CHECK-NEXT:    mov v6.16b, v2.16b
+; CHECK-NEXT:    mov v16.16b, v3.16b
+; CHECK-NEXT:    zip2 v7.4s, v2.4s, v3.4s
+; CHECK-NEXT:    mov v6.s[0], v3.s[1]
+; CHECK-NEXT:    ext v17.16b, v1.16b, v4.16b, #8
+; CHECK-NEXT:    mov v16.s[1], v2.s[0]
+; CHECK-NEXT:    uzp2 v1.4s, v5.4s, v2.4s
+; CHECK-NEXT:    ext v5.16b, v2.16b, v2.16b, #12
+; CHECK-NEXT:    zip2 v2.4s, v3.4s, v2.4s
+; CHECK-NEXT:    mov v7.d[1], v18.d[1]
+; CHECK-NEXT:    mov v6.d[1], v4.d[1]
+; CHECK-NEXT:    mov v16.d[1], v17.d[1]
+; CHECK-NEXT:    mov v1.d[1], v0.d[1]
+; CHECK-NEXT:    ext v3.16b, v3.16b, v5.16b, #12
+; CHECK-NEXT:    mov v2.d[1], v18.d[1]
+; CHECK-NEXT:    add v4.4s, v6.4s, v16.4s
+; CHECK-NEXT:    add v1.4s, v1.4s, v7.4s
+; CHECK-NEXT:    mov v3.d[1], v0.d[1]
+; CHECK-NEXT:    rev64 v5.4s, v4.4s
+; CHECK-NEXT:    rev64 v0.4s, v1.4s
+; CHECK-NEXT:    sub v2.4s, v2.4s, v3.4s
+; CHECK-NEXT:    sub v3.4s, v16.4s, v6.4s
+; CHECK-NEXT:    mov v5.d[1], v4.d[1]
+; CHECK-NEXT:    mov v0.d[1], v1.d[1]
+; CHECK-NEXT:    add v6.4s, v2.4s, v3.4s
+; CHECK-NEXT:    sub v2.4s, v3.4s, v2.4s
+; CHECK-NEXT:    add v1.4s, v1.4s, v5.4s
+; CHECK-NEXT:    sub v0.4s, v4.4s, v0.4s
+; CHECK-NEXT:    zip1 v3.4s, v1.4s, v6.4s
+; CHECK-NEXT:    uzp2 v4.4s, v1.4s, v6.4s
+; CHECK-NEXT:    zip2 v16.4s, v1.4s, v6.4s
+; CHECK-NEXT:    zip1 v5.4s, v0.4s, v2.4s
+; CHECK-NEXT:    trn1 v7.4s, v0.4s, v2.4s
+; CHECK-NEXT:    zip2 v2.4s, v0.4s, v2.4s
+; CHECK-NEXT:    trn2 v3.4s, v1.4s, v3.4s
+; CHECK-NEXT:    uzp2 v4.4s, v4.4s, v1.4s
+; CHECK-NEXT:    mov v1.s[1], v6.s[1]
+; CHECK-NEXT:    ext v0.16b, v0.16b, v5.16b, #8
 ; CHECK-NEXT:    mov v16.d[1], v7.d[1]
-; CHECK-NEXT:    ext v1.16b, v1.16b, v18.16b, #12
-; CHECK-NEXT:    mov v2.d[1], v6.d[1]
-; CHECK-NEXT:    mov v0.d[1], v7.d[1]
-; CHECK-NEXT:    mov v17.d[1], v3.d[1]
-; CHECK-NEXT:    mov v4.d[1], v5.d[1]
-; CHECK-NEXT:    mov v1.d[1], v6.d[1]
-; CHECK-NEXT:    add v2.4s, v2.4s, v16.4s
-; CHECK-NEXT:    add v3.4s, v4.4s, v17.4s
-; CHECK-NEXT:    rev64 v5.4s, v2.4s
-; CHECK-NEXT:    sub v0.4s, v0.4s, v1.4s
-; CHECK-NEXT:    sub v1.4s, v17.4s, v4.4s
-; CHECK-NEXT:    rev64 v6.4s, v3.4s
-; CHECK-NEXT:    mov v5.d[1], v2.d[1]
-; CHECK-NEXT:    sub v4.4s, v1.4s, v0.4s
-; CHECK-NEXT:    add v0.4s, v0.4s, v1.4s
-; CHECK-NEXT:    mov v6.d[1], v3.d[1]
-; CHECK-NEXT:    sub v3.4s, v3.4s, v5.4s
-; CHECK-NEXT:    add v1.4s, v2.4s, v6.4s
-; CHECK-NEXT:    zip1 v2.4s, v3.4s, v4.4s
-; CHECK-NEXT:    zip2 v7.4s, v3.4s, v4.4s
-; CHECK-NEXT:    zip1 v5.4s, v1.4s, v0.4s
-; CHECK-NEXT:    uzp2 v6.4s, v1.4s, v0.4s
-; CHECK-NEXT:    mov v18.16b, v1.16b
-; CHECK-NEXT:    ext v16.16b, v3.16b, v2.16b, #8
-; CHECK-NEXT:    zip2 v17.4s, v1.4s, v0.4s
-; CHECK-NEXT:    mov v3.s[3], v4.s[2]
-; CHECK-NEXT:    mov v18.s[1], v0.s[1]
-; CHECK-NEXT:    trn2 v4.4s, v1.4s, v5.4s
-; CHECK-NEXT:    uzp2 v1.4s, v6.4s, v1.4s
-; CHECK-NEXT:    mov v17.d[1], v3.d[1]
-; CHECK-NEXT:    mov v18.d[1], v2.d[1]
-; CHECK-NEXT:    mov v4.d[1], v16.d[1]
-; CHECK-NEXT:    mov v1.d[1], v7.d[1]
-; CHECK-NEXT:    add v0.4s, v17.4s, v1.4s
-; CHECK-NEXT:    add v2.4s, v18.4s, v4.4s
-; CHECK-NEXT:    sub v1.4s, v1.4s, v17.4s
-; CHECK-NEXT:    sub v3.4s, v4.4s, v18.4s
-; CHECK-NEXT:    ext v4.16b, v0.16b, v0.16b, #4
+; CHECK-NEXT:    mov v4.d[1], v2.d[1]
+; CHECK-NEXT:    mov v1.d[1], v5.d[1]
+; CHECK-NEXT:    mov v3.d[1], v0.d[1]
+; CHECK-NEXT:    add v0.4s, v16.4s, v4.4s
+; CHECK-NEXT:    sub v4.4s, v4.4s, v16.4s
+; CHECK-NEXT:    add v2.4s, v1.4s, v3.4s
+; CHECK-NEXT:    sub v1.4s, v3.4s, v1.4s
+; CHECK-NEXT:    ext v3.16b, v0.16b, v0.16b, #4
+; CHECK-NEXT:    zip2 v6.4s, v0.4s, v4.4s
+; CHECK-NEXT:    zip2 v7.4s, v4.4s, v0.4s
 ; CHECK-NEXT:    ext v5.16b, v2.16b, v2.16b, #4
-; CHECK-NEXT:    zip2 v6.4s, v0.4s, v1.4s
-; CHECK-NEXT:    zip2 v7.4s, v1.4s, v0.4s
-; CHECK-NEXT:    zip2 v16.4s, v3.4s, v2.4s
-; CHECK-NEXT:    zip2 v17.4s, v2.4s, v3.4s
-; CHECK-NEXT:    zip1 v0.4s, v0.4s, v1.4s
-; CHECK-NEXT:    ext v18.16b, v4.16b, v1.16b, #8
-; CHECK-NEXT:    ext v19.16b, v5.16b, v3.16b, #8
-; CHECK-NEXT:    zip1 v1.4s, v2.4s, v3.4s
+; CHECK-NEXT:    zip2 v16.4s, v1.4s, v2.4s
+; CHECK-NEXT:    zip2 v17.4s, v2.4s, v1.4s
+; CHECK-NEXT:    zip1 v0.4s, v0.4s, v4.4s
+; CHECK-NEXT:    ext v18.16b, v3.16b, v4.16b, #8
+; CHECK-NEXT:    ext v19.16b, v5.16b, v1.16b, #8
+; CHECK-NEXT:    zip1 v1.4s, v2.4s, v1.4s
 ; CHECK-NEXT:    add v2.4s, v16.4s, v7.4s
-; CHECK-NEXT:    sub v3.4s, v6.4s, v17.4s
-; CHECK-NEXT:    ext v4.16b, v18.16b, v4.16b, #4
+; CHECK-NEXT:    sub v4.4s, v6.4s, v17.4s
+; CHECK-NEXT:    ext v3.16b, v18.16b, v3.16b, #4
+; CHECK-NEXT:    cmlt v6.8h, v2.8h, #0
 ; CHECK-NEXT:    ext v5.16b, v19.16b, v5.16b, #4
 ; CHECK-NEXT:    sub v0.4s, v0.4s, v1.4s
-; CHECK-NEXT:    cmlt v1.8h, v3.8h, #0
-; CHECK-NEXT:    cmlt v6.8h, v2.8h, #0
-; CHECK-NEXT:    add v4.4s, v5.4s, v4.4s
-; CHECK-NEXT:    cmlt v5.8h, v0.8h, #0
+; CHECK-NEXT:    cmlt v1.8h, v4.8h, #0
 ; CHECK-NEXT:    add v2.4s, v6.4s, v2.4s
-; CHECK-NEXT:    add v3.4s, v1.4s, v3.4s
-; CHECK-NEXT:    cmlt v7.8h, v4.8h, #0
-; CHECK-NEXT:    add v0.4s, v5.4s, v0.4s
+; CHECK-NEXT:    add v4.4s, v1.4s, v4.4s
+; CHECK-NEXT:    add v3.4s, v5.4s, v3.4s
+; CHECK-NEXT:    cmlt v5.8h, v0.8h, #0
 ; CHECK-NEXT:    eor v2.16b, v2.16b, v6.16b
-; CHECK-NEXT:    eor v1.16b, v3.16b, v1.16b
-; CHECK-NEXT:    add v3.4s, v7.4s, v4.4s
-; CHECK-NEXT:    eor v0.16b, v0.16b, v5.16b
+; CHECK-NEXT:    eor v1.16b, v4.16b, v1.16b
+; CHECK-NEXT:    cmlt v7.8h, v3.8h, #0
+; CHECK-NEXT:    add v0.4s, v5.4s, v0.4s
 ; CHECK-NEXT:    add v1.4s, v2.4s, v1.4s
+; CHECK-NEXT:    add v3.4s, v7.4s, v3.4s
+; CHECK-NEXT:    eor v0.16b, v0.16b, v5.16b
 ; CHECK-NEXT:    eor v2.16b, v3.16b, v7.16b
 ; CHECK-NEXT:    add v0.4s, v0.4s, v1.4s
 ; CHECK-NEXT:    add v0.4s, v2.4s, v0.4s
@@ -478,73 +476,73 @@ define i32 @v3(ptr nocapture nou...
[truncated]

MacDue
MacDue reviewed Nov 15, 2025
View reviewed changes
llvm/test/CodeGen/AArch64/insert-extend.ll
; CHECK-NEXT: rev64 v7.4s, v3.4s
; CHECK-NEXT: sub v5.4s, v1.4s, v5.4s
; CHECK-NEXT: sub v6.4s, v2.4s, v6.4s
; CHECK-NEXT: addp v2.4s, v3.4s, v2.4s
Copy link
Member

@MacDue MacDue Nov 15, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it may be worth adding some smaller test cases for this PR (possibly in a new file), as the current test updates are fairly large and do not directly test this change.

Copy link
Contributor Author

@ginsbach ginsbach Nov 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, great suggestion. In the second commit, I have added additional tests to arm64-trn.ll, loosely based on equivalent tests in arm64-zip.ll.

@ginsbach ginsbach requested a review from MacDue November 15, 2025 20:41
MacDue
MacDue approved these changes Nov 18, 2025
View reviewed changes
Copy link
Member

@MacDue MacDue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems reasonable to me 👍 I'm just going to tag @davemgreen too since he worked on the ZIP patch.

@MacDue MacDue requested a review from davemgreen November 18, 2025 15:33
@github-actions
Copy link

github-actions bot commented Nov 18, 2025 •
edited
Loading

✅ With the latest revision this PR passed the undef deprecator.

@ginsbach
[AArch64] match TRN starting from undef elements (llvm#167955)
e30eaeb 
When the first element of a trn mask is undef, the `isTRNMask` function
assumes `WhichResult = 1`. That has a 50% chance of being wrong, so we
fail to match some valid trn1/trn2.

This patch introduces a more precise test to determine the correct value
of `WhichResult`, based on corresponding code in the `isZIPMask` and
`isUZPMask` functions.

- This change is based on llvm#89578. I'd like to follow it up with a
further change along the lines of llvm#167235.
@ginsbach ginsbach force-pushed the more-robust-trn-detection branch from 020b822 to e30eaeb Compare November 18, 2025 19:26
@ginsbach
Copy link
Contributor Author

ginsbach commented Nov 18, 2025

This seems reasonable to me 👍 I'm just going to tag @davemgreen too since he worked on the ZIP patch.

Thank you very much!

I have consolidated the changes into a single commit according to the guidelines, with message and title matching the PR. There were no other changes in the force-push.

I don't have write access, so won't be able to merge myself.

The following files introduce new uses of undef:

  • llvm/test/CodeGen/AArch64/arm64-trn.ll

I expect that this is fine to ignore, as it matches the approach in the remainder of this file?

@MacDue
Copy link
Member

MacDue commented Nov 18, 2025

I have consolidated the changes into a single commit according to the guidelines, with message and title matching the PR. There were no other changes in the force-push.

No worries, but I think those docs are a little outdated/misleading; there's no need for you to manually squash your commits before a PR is merged.

I expect that this is fine to ignore, as it matches the approach in the remainder of this file?

undef is being phased out. It would be better if your new tests used poison, since this will need to be changed eventually. I don't have a strong preference, though.

I don't have write access, so won't be able to merge myself.

I'll merge this in a day or so (if there's no further comments).

@ginsbach
Copy link
Contributor Author

ginsbach commented Nov 18, 2025

I have consolidated the changes into a single commit according to the guidelines, with message and title matching the PR. There were no other changes in the force-push.

No worries, but I think those docs are a little outdated/misleading; there's no need for you to manually squash your commits before a PR is merged.

Thanks for clarifying.

undef is being phased out. It would be better if your new tests used poison, since this will need to be changed eventually. I don't have a strong preference, though.

I have pushed another commit that changes these to poison values.

I'll merge this in a day or so (if there's no further comments).

Thank you!

@github-actions
Copy link

github-actions bot commented Nov 18, 2025

🐧 Linux x64 Test Results

  • 186344 tests passed
  • 4854 tests skipped

davemgreen
davemgreen approved these changes Nov 19, 2025
View reviewed changes
Copy link
Collaborator

@davemgreen davemgreen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds great.

@MacDue MacDue merged commit bdcaa00 into llvm:main Nov 19, 2025
10 checks passed
@github-actions
Copy link

github-actions bot commented Nov 19, 2025

@ginsbach Congratulations on having your first Pull Request (PR) merged into the LLVM Project!

Your changes will be combined with recent changes from other authors, then tested by our build bots. If there is a problem with a build, you may receive a report in an email or a comment on this PR.

Please check whether problems have been caused by your change specifically, as the builds can include changes from many authors. It is not uncommon for your change to be included in a build that fails due to someone else's changes, or infrastructure issues.

How to do this, and the rest of the post-merge process, is covered in detail here.

If your change does cause a problem, it may be reverted, or you can revert it yourself. This is a normal part of LLVM development. You can fix your changes and open a new PR to merge them again.

If you don't get any reports, no action is required from you. Your changes are working as expected, well done!

@ginsbach ginsbach deleted the more-robust-trn-detection branch November 19, 2025 17:36
@ginsbach ginsbach mentioned this pull request Nov 19, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@MacDue MacDue MacDue approved these changes

@davemgreen davemgreen davemgreen approved these changes

Assignees

No one assigned

Labels

backend:AArch64

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ginsbach @llvmbot @MacDue @davemgreen