raise error for group_gemm_fp8_nt_groupwise then num_groups > 1 on sm120/121 (#1862)

## 📌 Description

- Raise a RuntimeError for group_gemm_fp8_nt_groupwise when num_groups >
1 on SM120/121.
- Skip the related tests.
- Tentatively rename tests/GEMM to tests/gemm for consistency with other
components that use lowercase directory names across the codebase
(correct me pls!!!).


## 🔍 Related Issues

<!-- Link any related issues here -->

## 🚀 Pull Request Checklist

Thank you for contributing to FlashInfer! Before we review your pull
request, please make sure the following items are complete.

### ✅ Pre-commit Checks

- [x] I have installed `pre-commit` by running `pip install pre-commit`
(or used your preferred method).
- [x] I have installed the hooks with `pre-commit install`.
- [x] I have run the hooks manually with `pre-commit run --all-files`
and fixed any reported issues.

> If you are unsure about how to set up `pre-commit`, see [the
pre-commit documentation](https://pre-commit.com/).

## 🧪 Tests

- [x] Tests have been added or updated as needed.
- [ ] All tests are passing (`unittest`, etc.).

## Reviewer Notes

@yzh119 @aleozlx @nvmbreughe @bkryu

Author: yongwww

flashinfer/gemm.py

@@ -2304,6 +2304,11 @@ def group_gemm_fp8_nt_groupwise(
         assert out.dtype == out_dtype
 
     if is_sm120a_supported(a.device) or is_sm121a_supported(a.device):
+        # it has correctness issues for num_groups > 1
+        if num_groups > 1:
+            raise RuntimeError(
+                "group_gemm_fp8_nt_groupwise has correctness issues for num_groups > 1 on SM120/121"
+            )
         # SM120/121 doesn't use mma_sm parameter
         get_gemm_sm120_module().group_gemm_fp8_nt_groupwise(
             int_workspace_buffer,

scripts/task_jit_run_tests_part1.sh

@@ -7,7 +7,7 @@ set -x
 
 pip install -e . -v
 
-# pytest -s tests/GEMM/test_group_gemm.py
+# pytest -s tests/gemm/test_group_gemm.py
 pytest -s tests/attention/test_logits_cap.py
 pytest -s tests/attention/test_sliding_window.py
 pytest -s tests/attention/test_tensor_cores_decode.py

scripts/task_jit_run_tests_part4.sh

@@ -9,7 +9,7 @@ pip install -e . -v
 
 export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True  # avoid memory fragmentation
 pytest -s tests/attention/test_deepseek_mla.py
-pytest -s tests/GEMM/test_group_gemm.py
+pytest -s tests/gemm/test_group_gemm.py
 pytest -s tests/attention/test_batch_prefill_kernels.py
 # NOTE(Zihao): need to fix tile size on KV dimension for head_dim=256 on small shared memory architecture (sm89)
 # pytest -s tests/attention/test_batch_attention.py

tests/GEMM/__init__.py renamed to tests/gemm/__init__.py

@@ -146,6 +146,12 @@ def test_fp8_groupwise_group_gemm(
     scale_major_mode,
     out_dtype,
 ):
+    if group_size > 1 and torch.cuda.get_device_capability()[0] in [
+        12,
+    ]:
+        pytest.skip(
+            "group_gemm_fp8_nt_groupwise has correctness issues for num_groups > 1 on SM120/121"
+        )
     torch.random.manual_seed(0)
     tile_size = 128