Skip to content

perf: optimize Qwen3.5-122B SGLang serving for H200 #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
PierreLeGuen merged 3 commits into from
Mar 6, 2026
+6 −2
Merged

perf: optimize Qwen3.5-122B SGLang serving for H200 #5

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b914170
perf: optimize Qwen3.5-122B SGLang serving for H200
PierreLeGuen Mar 6, 2026
2be0dac
chore: increase context length to 262K (native max)
PierreLeGuen Mar 6, 2026
bc09379
fix: use H200 default chunked-prefill-size of 8192
PierreLeGuen Mar 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions Qwen3.5-122B.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,12 @@ x-sglang-qwen35-122b-common: &sglang-qwen35-122b-common
sglang serve
--model-path Qwen/Qwen3.5-122B-A10B
--tp 4
--mem-fraction-static 0.80
--context-length 131072
--mem-fraction-static 0.88
--context-length 262144
--kv-cache-dtype fp8_e4m3
--chunked-prefill-size 8192
--attention-backend flashinfer
--schedule-conservativeness 0.5
--reasoning-parser qwen3
--tool-call-parser qwen3_coder
--log-requests-level 0
Expand Down