Skip to content

Commit 13a3ade

Browse files
committed
Update on "fix eager run for cuda"
[ghstack-poisoned]
2 parents c88b312 + 99138a0 commit 13a3ade

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

examples/models/llama/runner/generation.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,8 @@ def generate( # noqa: C901
101101
)
102102
current_token = next_token(logits, temperature, top_p)
103103
if current_token == self.tokenizer.eos_id or (
104-
hasattr(self, "stop_tokens") and current_token in self.stop_tokens
104+
hasattr(self.tokenizer, "stop_tokens")
105+
and current_token in self.tokenizer.stop_tokens
105106
):
106107
break
107108
tokens.append(current_token)

0 commit comments

Comments
 (0)