Skip to content

Commit c857f9d

Browse files
committed
Yet another llama-specific bugfix
1 parent 647d7cc commit c857f9d

File tree

1 file changed

+1
-5
lines changed

1 file changed

+1
-5
lines changed

nemo/collections/nlp/modules/common/text_generation_server.py

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -290,11 +290,7 @@ def chat_completion(self, data):
290290
while not done:
291291
done = True
292292
for e in end_strings:
293-
# This code is meant to be somewhat generic (even if the above code is not):
294-
# - If we stop on "end_of_turn", then we strip "end_of_turn" (ex: "<|eot_id|>")
295-
# - If we stop on an end string that follows "end_of_turn", then we strip both "end_of_turn"
296-
# and that end string (ex: "\n<extra_id_1>")
297-
suffix = e if e == eot else (eot + e)
293+
suffix = e
298294
# The loop is very Llama-Instruct-specific, due to how "<|eot_id|>" is also the padding
299295
# EOS token => it may be present multiple times.
300296
while output_sentence.endswith(suffix):

0 commit comments

Comments
 (0)