benchmark: Adding FP8 benchmark on attention and matmul testing (#1390)

<!-- .github/pull_request_template.md -->

## 📌 Description

<!-- What does this PR do? Briefly describe the changes and why they’re
needed. -->

Current PR extends benchmarking script in `flashinfer_benchmark.py` by
adding:
* MLA backend testing
* Attention FP8 Benchmarking Introduction for prefill, decode, and MLA. 
* bmm_fp8 and mm_fp4 testing. 
* Addition of a few backends where applicable (e.g. trtllm-gen for paged
prefill)
* Fix broken cuDNN prefill & decode benchmarking due to migration from
cubins to full cuDNN integration
* General minor benchmark code refactoring to reduce code redundancy.

## 🔍 Related Issues

<!-- Link any related issues here -->

## 🚀 Pull Request Checklist

Thank you for contributing to FlashInfer! Before we review your pull
request, please make sure the following items are complete.

### ✅ Pre-commit Checks

- [x] I have installed `pre-commit` by running `pip install pre-commit`
(or used your preferred method).
- [x] I have installed the hooks with `pre-commit install`.
- [x] I have run the hooks manually with `pre-commit run --all-files`
and fixed any reported issues.

> If you are unsure about how to set up `pre-commit`, see [the
pre-commit documentation](https://pre-commit.com/).

## 🧪 Tests

- [x] Tests have been added or updated as needed.
- [x] All tests are passing (`unittest`, etc.).

## Reviewer Notes

<!-- Optional: anything you'd like reviewers to focus on, concerns, etc.
-->

Author: bkryu

benchmarks/README.md

@@ -2,7 +2,11 @@
 import sys
 
 from routines.attention import parse_attention_args, run_attention_test
-from routines.flashinfer_benchmark_utils import full_output_columns, output_column_dict
+from routines.flashinfer_benchmark_utils import (
+    benchmark_apis,
+    full_output_columns,
+    output_column_dict,
+)
 from routines.gemm import parse_gemm_args, run_gemm_test
 
 
@@ -15,16 +19,9 @@ def run_test(args):
     """
 
     ## Depending on routine type, route to corresponding test routine
-    if args.routine in [
-        "BatchDecodeWithPagedKVCacheWrapper",
-        "BatchPrefillWithPagedKVCacheWrapper",
-        "BatchPrefillWithRaggedKVCacheWrapper",
-    ]:
+    if args.routine in benchmark_apis["attention"]:
         res = run_attention_test(args)
-    elif args.routine in [
-        "gemm_fp8_nt_groupwise",
-        "group_gemm_fp8_nt_groupwise",
-    ]:
+    elif args.routine in benchmark_apis["gemm"]:
         res = run_gemm_test(args)
     else:
         raise ValueError(f"Unsupported routine: {args.routine}")
@@ -63,13 +60,7 @@ def parse_args(line=sys.argv[1:]):
         "-R",
         type=str,
         required=True,
-        choices=[
-            "BatchDecodeWithPagedKVCacheWrapper",
-            "BatchPrefillWithPagedKVCacheWrapper",
-            "BatchPrefillWithRaggedKVCacheWrapper",
-            "gemm_fp8_nt_groupwise",
-            "group_gemm_fp8_nt_groupwise",
-        ],
+        choices=list(benchmark_apis["attention"]) + list(benchmark_apis["gemm"]),
     )
     args, _ = parser.parse_known_args(line[:])
 
@@ -122,16 +113,9 @@ def parse_args(line=sys.argv[1:]):
     )
 
     ## Check routine and pass on to routine-specific argument parser
-    if args.routine in [
-        "BatchDecodeWithPagedKVCacheWrapper",
-        "BatchPrefillWithPagedKVCacheWrapper",
-        "BatchPrefillWithRaggedKVCacheWrapper",
-    ]:
+    if args.routine in benchmark_apis["attention"]:
         args = parse_attention_args(line, parser)
-    elif args.routine in [
-        "gemm_fp8_nt_groupwise",
-        "group_gemm_fp8_nt_groupwise",
-    ]:
+    elif args.routine in benchmark_apis["gemm"]:
         args = parse_gemm_args(line, parser)
     else:
         raise ValueError(f"Unsupported routine: {args.routine}")