Merge OpenAI Triton till Jan 4th

[whitneywhtsang]

It is essential to keep up to date with OpenAI Triton, to get the latest features, and reduce the difficulty to upstream our changes to OpenAI Triton.

This ticket is continuation of:

• Merge OpenAI Triton till Dec 13rd #2879

[whitneywhtsang]

Moved up from Dec 12 to Jan 3 Triton version from OpenAI. [#3023, #3042, #3048, #3058, #3069, #3076, #3079, #3117, #3120 ]