[x86] Using `select` instead of `pblendvb` leads to very poor codegen on AVX2

[Sp00ph]

I tried this code:

define <32 x i8> @cond_double_blendv(<32 x i8> %a, <32 x i8> %mask) {
    %aa = add <32 x i8> %a, %a
    %ret = call <32 x i8> @llvm.x86.avx2.pblendvb(<32 x i8> %a, <32 x i8> %aa, <32 x i8> %mask)
    ret <32 x i8> %ret
}

define <32 x i8> @cond_double_select(<32 x i8> %a, <32 x i8> %mask) {
    %aa = add <32 x i8> %a, %a
    %bitmask = icmp slt <32 x i8> %mask, splat (i8 0)
    %ret = select <32 x i1> %bitmask, <32 x i8> %aa, <32 x i8> %a
    ret <32 x i8> %ret
}

Both functions have the same behavior, doubling lanes of %a if the MSB of the corresponding lane in %b is set. However, they generate wildly different assembly (using clang 21.1.0 with -O3 -march=x86-64-v3):

cond_double_blendv:
        vpaddb  ymm2, ymm0, ymm0
        vpblendvb       ymm0, ymm0, ymm2, ymm1
        ret

.LCPI1_0:
        .zero   32,252
.LCPI1_1:
        .zero   32,32
cond_double_select:
        vpsllw  ymm2, ymm0, 2
        vpand   ymm2, ymm2, ymmword ptr [rip + .LCPI1_0]
        vpsrlw  ymm1, ymm1, 2
        vpand   ymm1, ymm1, ymmword ptr [rip + .LCPI1_1]
        vpaddb  ymm1, ymm1, ymm1
        vpblendvb       ymm0, ymm0, ymm2, ymm1
        vpaddb  ymm2, ymm0, ymm0
        vpaddb  ymm1, ymm1, ymm1
        vpblendvb       ymm0, ymm0, ymm2, ymm1
        ret

The version using the @llvm.x86.avx2.pblendvb intrinsic emits the expected assembly. After staring at the version using select for a while, I can say that all instructions except vpaddb ymm2, ymm0, ymm0 and the last vpblendvb form an elaborate no-op. I have no idea however what the code generator's intent was with these instructions. I don't see any reason why these functions should not just emit the exact same assembly.

Note: I originally encountered this while using AVX2 intrinsics in Rust, where the output from rustc was much worse than the output from clang for an equivalent function, with the difference being that rustc lowers _mm256_blendv_epi8 to icmp slt + select, whereas clang lowers it to call @llvm.x86.avx2.pblendvb.

[llvmbot]

@llvm/issue-subscribers-backend-x86

Author: Sp00ph (Sp00ph)

I tried this code:

define &lt;32 x i8&gt; @<!-- -->cond_double_blendv(&lt;32 x i8&gt; %a, &lt;32 x i8&gt; %mask) {
    %aa = add &lt;32 x i8&gt; %a, %a
    %ret = call &lt;32 x i8&gt; @<!-- -->llvm.x86.avx2.pblendvb(&lt;32 x i8&gt; %a, &lt;32 x i8&gt; %aa, &lt;32 x i8&gt; %mask)
    ret &lt;32 x i8&gt; %ret
}

define &lt;32 x i8&gt; @<!-- -->cond_double_select(&lt;32 x i8&gt; %a, &lt;32 x i8&gt; %mask) {
    %aa = add &lt;32 x i8&gt; %a, %a
    %bitmask = icmp slt &lt;32 x i8&gt; %mask, splat (i8 0)
    %ret = select &lt;32 x i1&gt; %bitmask, &lt;32 x i8&gt; %aa, &lt;32 x i8&gt; %a
    ret &lt;32 x i8&gt; %ret
}

Both functions have the same behavior, doubling lanes of %a if the MSB of the corresponding lane in %b is set. However, they generate wildly different assembly (using clang 21.1.0 with -O3 -march=x86-64-v3):

cond_double_blendv:
        vpaddb  ymm2, ymm0, ymm0
        vpblendvb       ymm0, ymm0, ymm2, ymm1
        ret

.LCPI1_0:
        .zero   32,252
.LCPI1_1:
        .zero   32,32
cond_double_select:
        vpsllw  ymm2, ymm0, 2
        vpand   ymm2, ymm2, ymmword ptr [rip + .LCPI1_0]
        vpsrlw  ymm1, ymm1, 2
        vpand   ymm1, ymm1, ymmword ptr [rip + .LCPI1_1]
        vpaddb  ymm1, ymm1, ymm1
        vpblendvb       ymm0, ymm0, ymm2, ymm1
        vpaddb  ymm2, ymm0, ymm0
        vpaddb  ymm1, ymm1, ymm1
        vpblendvb       ymm0, ymm0, ymm2, ymm1
        ret

The version using the @<!-- -->llvm.x86.avx2.pblendvb intrinsic emits the expected assembly. After staring at the version using select for a while, I can say that all instructions except vpaddb ymm2, ymm0, ymm0 and the last vpblendvb form an elaborate no-op. I have no idea however what the code generator's intent was with these instructions. I don't see any reason why these functions should not just emit the exact same assembly.

Note: I originally encountered this while using AVX2 intrinsics in Rust, where the output from rustc was much worse than the output from clang for an equivalent function, with the difference being that rustc lowers _mm256_blendv_epi8 to icmp slt + select, whereas clang lowers it to call @<!-- -->llvm.x86.avx2.pblendvb.

[RKSimon]

instcombine is converting this to:

define <32 x i8> @cond_double_select(<32 x i8> %a, <32 x i8> %mask) {
  %1 = lshr <32 x i8> %mask, splat (i8 7)
  %ret = shl <32 x i8> %a, %1
  ret <32 x i8> %ret
}

and the v32i8 vector shift lowering codegen never quite manages to combine it back.

I haven't looked into whether there's any missing DAG value tracking / combines that would help, but its a shame that the v32i8 shl lowering fails to recognise that the shift %1 amount can only be 0 or 1.

[Sp00ph]

I suppose I also found it weird that it makes any difference at all whether you use icmp slt zeroinitializer + select compared to pblendvb. Feels to me like the optimizer should know they are exactly equivalent.

[RKSimon]

instcombine is purely responsible for canonicalizing code to a simpler optimized pattern - and the shift pair is definitely better than a select - its just unfortunate that x86 has such poor vXi8 shift handling - I've found a few combines to help this now, but I'm running out of time before the weekend :/

[Sp00ph]

What I was referring to is I would've expected pblendvb to be canonicalized to select (or vice versa).

[RKSimon]

Ah. We only do that if we recognize the blendv selection mask operand as coming from comparison results.

[Sp00ph]

pblendv only cares about the sign bit of the mask vector though, so I don't see why it would care whether all lanes are 0 or -1. AFAIK that's usually the reason to specialize on comparison results, right?