Skip to content

Commit b1340f9

Browse files
[V1] Bugfix: Validate Model Input Length (#12600)
SUMMARY: * avoid crashing the engine when we get an input longer than max_model_len FIX #12567(*link existing issues this PR will resolve*)
1 parent 44bbca7 commit b1340f9

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

vllm/v1/engine/processor.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -206,6 +206,11 @@ def _validate_model_inputs(self, inputs: ProcessorInputs):
206206
if prompt_ids is None or len(prompt_ids) == 0:
207207
raise ValueError("Prompt cannot be empty")
208208

209+
if len(prompt_ids) >= self.model_config.max_model_len:
210+
raise ValueError(
211+
f"Prompt length of {len(prompt_ids)} is longer than the "
212+
f"maximum model length of {self.model_config.max_model_len}.")
213+
209214
if self.model_config.is_multimodal_model:
210215
max_prompt_len = self.model_config.max_model_len
211216

0 commit comments

Comments
 (0)