[MIPS] Incorrect expansion of sub-word signed atomic max

[Amanieu]

IR

define void @foo(ptr %a, i32 %x) {
  %val = trunc i32 %x to i8
  %1 = atomicrmw max ptr %a, i8 %val seq_cst
  ret void
}

Asm (-march=mipsel)

foo:                                    # @foo
        sync
        addiu   $1, $zero, -4
        and     $1, $4, $1
        andi    $2, $4, 3
        sll     $2, $2, 3
        ori     $3, $zero, 255
        sllv    $3, $3, $2
        nor     $4, $zero, $3
        sllv    $5, $5, $2
$BB0_1:                                 # =>This Inner Loop Header: Depth=1
        ll      $7, 0($1)
        and     $7, $7, $3 # <== WRONG, zero-extending
        and     $5, $5, $3 # <== WRONG, zero-extending 
        slt     $10, $7, $5
        move    $8, $7
        movn    $8, $5, $10
        and     $8, $8, $3
        and     $9, $7, $4
        or      $9, $9, $8
        sc      $9, 0($1)
        beqz    $9, $BB0_1
        nop
        and     $6, $7, $3
        srlv    $6, $6, $2
        sll     $6, $6, 16
        sra     $6, $6, 16
        sync
        jr      $ra
        nop

The problem

The expansion of i8 signed atomic max masks out all bits that are not involved, but fails to sign-extend the values before passing them to slt for a proper signed comparison.

Downstream bug: rust-lang/rust#100650

[llvmbot]

@llvm/issue-subscribers-backend-mips

[yingopq]

Hi @Amanieu,
What do you think a correct assembly should look like?
$3 has experienced sll, value was (0 | 255) << ((ptr andi 3) << 3). So does this mean expansion?
Thanks.

[Amanieu]

If you look at the assembly code, it is extracting the byte from the word by masking it (lines marked <== WRONG, zero-extending). The problem is that this is supposed to be a signed atomic max. The values need to be sign-extended before being compared with slt.

[Amanieu]

The current implementation will happen to work if the byte is the top byte in the word, since no sign-extension is required them. But in other cases the result will be incorrect.