Skip to content

Commit 9f34c1b

Browse files
authored
Fix typo in INFERENCE.md, change return_tensors and to correct usage of device (#98)
1 parent 6185106 commit 9f34c1b

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

INFERENCE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ model.generation_config.cache_implementation = "static"
6262
model.forward = torch.compile(model.forward, mode=compile_mode)
6363

6464
# warmup
65-
inputs = tokenizer("This is for compilation", return_tensors="pt", padding="max_length", max_length=max_length).to(device)
65+
inputs = tokenizer("This is for compilation", return_tensors="pt", padding="max_length", max_length=max_length).to(torch_device)
6666

6767
model_kwargs = {**inputs, "prompt_input_ids": inputs.input_ids, "prompt_attention_mask": inputs.attention_mask, }
6868

0 commit comments

Comments
 (0)