Purpose

This PR adds tuned Fused-MoE Triton configs for Qwen3-30B A3 (E=64) and A3B (E=128) on NVIDIA H100 80GB, covering both BF16 and FP8 (fp8_w8a8). It’s part of #22294.

New files (under vllm/model_executor/layers/fused_moe/configs/):

• E=64,N=8960,device_name=NVIDIA_H100_80GB_HBM3,dtype=bf16.json
• E=64,N=8960,device_name=NVIDIA_H100_80GB_HBM3,dtype=fp8_w8a8.json
• E=128,N=8960,device_name=NVIDIA_H100_80GB_HBM3,dtype=bf16.json
• E=128,N=8960,device_name=NVIDIA_H100_80GB_HBM3,dtype=fp8_w8a8.json

Each table is tuned per (E, dtype) and only keeps the “turning-point” batches to avoid bloat. For FP8, I used zero-init weights + unit scales (PyTorch can’t randn float8 yet); this doesn’t affect which kernel gets picked—selection depends on shapes and launch params.

Test Plan (short)

Env: H100 80GB (SXM) · CUDA 12.8 · PyTorch 2.8.0 · vLLM wheel (with Triton)

What I verified

• Loader picks up the tuned JSONs for each (E, dtype).
• If a file is hidden, vLLM falls back to default.
• The four JSONs are not identical.

Repro

python - <<'PY'
import logging, torch
from vllm.model_executor.layers.fused_moe.fused_moe import fused_topk, fused_experts
logging.getLogger("vllm.model_executor.layers.fused_moe.fused_moe").setLevel(logging.INFO)

H, N, top_k, B = 7168, 8960, 8, 32
def run(E, dtype):
    xdtype = torch.bfloat16 if dtype=="bf16" else torch.float16
    wdt    = xdtype if dtype=="bf16" else torch.float8_e4m3fn
    x  = torch.randn(B, H, dtype=xdtype, device="cuda")
    w1 = torch.zeros(E, N,   H, dtype=wdt, device="cuda")
    w2 = torch.zeros(E, H, N//2, dtype=wdt, device="cuda")
    gate = torch.randn(B, E, dtype=torch.float32, device="cuda")
    tw, ti, _ = fused_topk(x, gate, top_k, renormalize=True)
    fused_experts(x, w1, w2, tw, ti, inplace=True, quant_config=None)

for E in (64, 128):
  for dt in ("bf16", "fp8_w8a8"):
    print(f"\n=== E={E} {dt} ===")
    run(E, dt)
PY

Loader logs (proof of pickup)

INFO ... Using configuration from .../E=64,N=8960,...,dtype=bf16.json ...
INFO ... Using configuration from .../E=64,N=8960,...,dtype=fp8_w8a8.json ...
INFO ... Using configuration from .../E=128,N=8960,...,dtype=bf16.json ...
INFO ... Using configuration from .../E=128,N=8960,...,dtype=fp8_w8a8.json ...