Skip to content

Commit 5e443b5

Browse files
authored
[Bugfix] Allow prefill of assistant response when using mistral_common (#9446)
1 parent 9d30a05 commit 5e443b5

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

vllm/transformers_utils/tokenizers/mistral.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -166,6 +166,10 @@ def apply_chat_template(self,
166166
tools: Optional[Dict[str, Any]] = None,
167167
**kwargs) -> List[int]:
168168

169+
last_message = messages[-1]
170+
if last_message["role"] == "assistant":
171+
last_message["prefix"] = True
172+
169173
request = ChatCompletionRequest(messages=messages,
170174
tools=tools) # type: ignore[type-var]
171175
encoded = self.mistral.encode_chat_completion(request)

0 commit comments

Comments
 (0)