[XPU][TritonGPUToLLVM] Avoid bank conflicts in sub-group transposes

[victor-eds]

• Store the whole matrix using SIMD block stores for each row leaving
  a single garbage item at the end of the row so each row has
  sub_group_size + 1 elements
• Load each row with vector loads

By introducing this garbage item at the end of each row, we ensure matrix
loading avoid bank conflicts as the offset between the position loaded by
work-item i and i+j is N * (sub_group_size + 1) (assuming sub_group_size
banks).

Closes #2751

[chengjunlu]

In my mind, the bank number of SLM is 65 on PVC. I think we need to make sure the bank number of the SLM.

[victor-eds]

In my mind, the bank number of SLM is 65 on PVC. I think we need to make sure the bank number of the SLM.

PVC has 64 8B banks. This PR indeed helps avoid bank conflicts. It's also true we could make this more optimal, but I'd go with this for now and fine-tune in a followup PR. WDYT?

[whitneywhtsang]

Any performance impact?

[chengjunlu]

In my mind, the bank number of SLM is 65 on PVC. I think we need to make sure the bank number of the SLM.

PVC has 64 8B banks. This PR indeed helps avoid bank conflicts. It's also true we could make this more optimal, but I'd go with this for now and fine-tune in a followup PR. WDYT?

LGTM.

[victor-eds]

Any performance impact?

Good impact on the victor/perf-attn. No effect on regular path as this is a pattern introduced by reduction opt.

[victor-eds]

@chengjunlu I created #2797