Skip to content

Commit 54d3544

Browse files
Fix: Output text is always truncated in some models (#3016)
1 parent 703e42e commit 54d3544

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

vllm/engine/llm_engine.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -980,7 +980,10 @@ def _check_stop(self, seq: Sequence,
980980
def _finalize_sequence(self, seq: Sequence,
981981
sampling_params: SamplingParams,
982982
stop_string: str) -> None:
983-
if not sampling_params.include_stop_str_in_output and stop_string:
983+
if sampling_params.include_stop_str_in_output:
984+
return
985+
986+
if stop_string and seq.output_text.endswith(stop_string):
984987
# Truncate the output text so that the stop string is
985988
# not included in the output.
986989
seq.output_text = seq.output_text[:-len(stop_string)]

0 commit comments

Comments
 (0)