Skip to content

Commit 654feda

Browse files
authored
[data][llm] fix vllm ray data quickstart example (#58463)
Signed-off-by: Nikhil Ghosh <[email protected]>
1 parent 2691094 commit 654feda

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

doc/source/data/doc_code/working-with-llms/basic_llm_example.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@
2525
engine_kwargs={
2626
"enable_chunked_prefill": True,
2727
"max_num_batched_tokens": 4096, # Reduce if CUDA OOM occurs
28-
"max_model_len": 16384,
28+
"max_model_len": 4096, # Constrain to fit test GPU memory
2929
},
3030
concurrency=1,
3131
batch_size=64,

0 commit comments

Comments
 (0)