[Codegen] Drop the workaround from EmulateNarrowType pass.

[hanhanW]

llvm/llvm-project@2de936b is a reasonable support, but it breaks IREE. The reason is that it is conversative about offsets, while IREE usually tiles the dimensions to be aligned with native vector size. In this context, the stores are always aligned store. The upstream has assumeAligned mode, so we drop the workaround.

Below is an example (from #20645) that shows the dynamic offset %arg2 is always aligned with bytes.

func.func @main(%arg0: memref<8xf32>, %arg1: memref<8xi4>) {
  %c4 = arith.constant 4 : index
  %c8 = arith.constant 8 : index
  %c0 = arith.constant 0 : index
  scf.for %arg2 = %c0 to %c8 step %c4 {
    %0 = vector.load %arg0[%arg2] : memref<8xf32>, vector<4xf32>
    %1 = arith.fptoui %0 : vector<4xf32> to vector<4xi32>
    %2 = arith.trunci %1 : vector<4xi32> to vector<4xi4>
    vector.store %2, %arg1[%arg2] : memref<8xi4>, vector<4xi4>
  }
  return
}

Closes #20645

[hanhanW]

It requires llvm/llvm-project#178565

[MaheshRavishankar]

Are you sure you want to use AtomicRMW. That can be expensive? My understanding is that we should just vectorize to a size where we dont need atomics.

[hanhanW]

This is default option; we already use it. We can switch in a follow up.

[hanhanW]

I forgot to say that I agree with you and it should be happening on CPU side. We'll need the switch or expose the option.

I also have a local patch that makes VMVX happy for all the subtypes we have, and it doesn't use atomicRMW.

[MaheshRavishankar]

THis is common CPU/GPU path. We can tolerate some regression on CPU side, but regression on GPU side is more problematic. If there was a known correctness issue, that is one thing, but do we have a known correctness issue here on GPU?

[hanhanW]

I did not see any failure locally when I did the switch. I need to test more, but GPU test coverage is lower than I expected. I'm not aware of any correctness issue and I did not find any on Github issues. GPU does not work for other fp8 types, btw. See the table in #23238 (comment)

Type	gfx908/90a	gfx942	gfx950	gfx11xx	gfx12xx
f8E4M3FNUZ	emu	hw	emu	emu	emu
f8E5M2FNUZ	emu	hw	emu	emu	emu
f8E4M3FN	emu	emu	hw	emu	hw
f8E5M2	emu	emu	hw	emu	hw
f4E2M1FN	emu	emu	hw	emu	hw

emu means that it is not supported without the PR.

What I wanted to say is that it's been using AtomicRMW mode for a long time. I'll create a PR to do the experiment, let's move forward to drop the workaround?

I think the tolerance won't be impacted if the tiling config is correct. The heuristic should take it into account, like what I observed in #20645 (comment). If not, it is a bug to me and we need to fix it. Or we expose the option and we always enable the flag on GPU.

[hanhanW]

sharktank tests are red because of azure issue, other tests look okay: #23344