Skip to content

Commit 6c87081

Browse files
authored
fix: add --is-decode-worker flag (#5843)
Signed-off-by: Hannah Zhang <[email protected]>
1 parent b88cb59 commit 6c87081

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

examples/basics/kubernetes/Distributed_Inference/disagg_router.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ spec:
4242
- /bin/sh
4343
- -c
4444
args:
45-
- python3 -m dynamo.vllm --model meta-llama/Llama-3.1-70B-Instruct -tp 8
45+
- python3 -m dynamo.vllm --model meta-llama/Llama-3.1-70B-Instruct -tp 8 --is-decode-worker
4646
VllmPrefillWorker:
4747
envFromSecret: hf-token-secret
4848
componentType: worker

0 commit comments

Comments
 (0)