Very poor codegen for switch statement

[resistor]

We are currently generating very poor code for this switch statement: https://github.com/resistor/cheriot-qoi/blob/97d7013479b2adb10ef1780717aa7325e4fb9dc1/lib/qoi_decode/qoi_decode.cc#L406

In particular, we decide to reload cra, cs0, and cs1 in every single case of the switch statement.

.LBB0_11:                               # %sw.bb24
        ct.cmove        ca1, cs0
        ct.clc  cra, 24(csp)                    # 8-byte Folded Reload
        ct.clc  cs0, 16(csp)                    # 8-byte Folded Reload
        ct.clc  cs1, 8(csp)                     # 8-byte Folded Reload
        ct.cincoffset   csp, csp, 32
        ct.ctail        _ZL27qoi_progress_await_channelsP17qoi_decoder_stateP10qoi_stream
.LBB0_12:                               # %sw.bb34
        ct.cmove        ca1, cs0
        ct.clc  cra, 24(csp)                    # 8-byte Folded Reload
        ct.clc  cs0, 16(csp)                    # 8-byte Folded Reload
        ct.clc  cs1, 8(csp)                     # 8-byte Folded Reload
        ct.cincoffset   csp, csp, 32
        ct.ctail        _ZL23qoi_progress_await_tailP17qoi_decoder_stateP10qoi_stream
...

In reduced versions of this the machine outliner is able to mostly save us from ourselves, but in the generalized version it is not able to.

[resistor]

foo.ll.txt

[resistor]

Reproducible with ./bin/llc --filetype=asm --mcpu=cheriot --mtriple=riscv32-unknown-unknown -target-abi cheriot -mattr=+xcheri,+cap-mode,+xcheriot --enable-machine-outliner=never < foo.ll

[resistor]

After some further investigation, it looks like forms of this are common across LLVM targets. Here's an example showing it happening on both RV32 and AArch64 in upstream: https://godbolt.org/z/sdWreMGxn

[resistor]

Reported upstream as llvm/llvm-project#141542