Skip to content

Commit 4a56af1

Browse files
committed
update max-osl for sglang backend
1 parent 64c4ee9 commit 4a56af1

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

language/deepseek-r1/utils/backend_registry.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -95,8 +95,8 @@
9595
"host": "0.0.0.0",
9696
"api_key": None,
9797
"tensor_parallel_size": 8,
98-
# NOTE(vir): sg-lang crash without +2 additional
99-
"context_length": MAX_ISL + MAX_OSL + MAX_TEMPLATE_TOKS + 2,
98+
# NOTE(vir): sg-lang crash without +4 additional
99+
"context_length": MAX_ISL + MAX_OSL + MAX_TEMPLATE_TOKS + 4,
100100
"max_tokens": MAX_OSL,
101101
"mem_fraction_static": 0.85,
102102
"random_seed": 42,

0 commit comments

Comments
 (0)