perf: optimize Qwen3.5-122B SGLang serving for H200

[PierreLeGuen]

Summary

• Enable FP8 KV cache quantization (fp8_e4m3) to halve memory usage and allow ~2x concurrent requests
• Add chunked prefill (4096 tokens) to prevent OOM on 128K context prompts
• Set FlashInfer attention backend for optimized kernels on H200
• Increase mem-fraction-static to 0.88 and tune scheduler for better throughput

Test plan

• Deploy to a single H200 node and verify model loads without OOM
• Run load test comparing throughput vs previous config
• Validate 128K context prompts complete without memory errors
• Monitor /metrics endpoint for KV cache utilization improvements

Made with Cursor