RMSNorm backward kernel implementaton (#709)

* the first version of rmsnorm bwd kernel implementation

* rename variables to be more representitive

* fix dg shared accross batch dimension

* add dg reduction kernel and restructure rmsnorm forward kernel and backward kernel

* change to wrapper approach

* replace wrapper with torch.autograd

* fix fwd benchmark

* add benchmark for rmsnorm backward

* use Michalel's new Integration script to test

* revert back to orginal, as it won't run

* change grad_sum init and remove epsilon from backward argument list

* add num_warps to backward kernel

* updates to do block reduction

* change grid_sum init

* fix rmsnomr tests for fp32

* dx, dg is the same type of x, and g, and fix unit test for backward kernel

* make sure norm_factor is tl.float32

* switch off fp32 test

Author: xiaohuguo2023

.github/workflows/amd_perf_kernel_Integration_tests.yml

@@ -126,6 +126,7 @@ jobs:
         run: |
           python ./python/perf-kernels/flash-attention.py
           python ./python/perf-kernels/softmax.py
-          python ./python/perf-kernels/rmsnorm.py
+          python ./python/perf-kernels/rmsnorm.py --mode fwd
+          python ./python/perf-kernels/rmsnorm.py --mode bwd
           python ./python/perf-kernels/layernorm.py
           python ./python/perf-kernels/multreduce_matmul_kernel.py bench