Skip to content

Commit 2d7f5fc

Browse files
authored
fix: vllm disagg launch script should not enable kv routing (#4063)
Signed-off-by: PeaBrane <[email protected]>
1 parent 3555b72 commit 2d7f5fc

File tree

2 files changed

+3
-3
lines changed

2 files changed

+3
-3
lines changed

examples/backends/vllm/launch/disagg.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ set -e
55
trap 'echo Cleaning up...; kill 0' EXIT
66

77
# run ingress
8-
python -m dynamo.frontend --router-mode kv --http-port=8000 &
8+
python -m dynamo.frontend --http-port=8000 &
99

1010
# --enforce-eager is added for quick deployment. for production use, need to remove this flag
1111
CUDA_VISIBLE_DEVICES=0 python3 -m dynamo.vllm --model Qwen/Qwen3-0.6B --enforce-eager &

examples/backends/vllm/launch/disagg_kvbm.sh

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,8 @@
44
set -e
55
trap 'echo Cleaning up...; kill 0' EXIT
66

7-
# run ingress with KV router
8-
python -m dynamo.frontend --router-mode kv --http-port=8000 &
7+
# run ingress
8+
python -m dynamo.frontend --http-port=8000 &
99

1010
# run decode worker on GPU 0, without enabling KVBM
1111
# NOTE: remove --enforce-eager for production use

0 commit comments

Comments
 (0)