We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 0f9f05f commit 9f909bdCopy full SHA for 9f909bd
docs/features/models/vllm_offline.md
@@ -44,15 +44,15 @@ For instance:
44
45
```python
46
import outlines
47
-from vllm import LLM
+from vllm import LLM, SamplingParams
48
49
# Create the model
50
model = outlines.from_vllm_offline(
51
LLM("microsoft/Phi-3-mini-4k-instruct")
52
)
53
54
# Call it to generate text
55
-response = model("What's the capital of Latvia?", max_tokens=20)
+response = model("What's the capital of Latvia?", sampling_params=SamplingParams(max_tokens=20))
56
print(response) # 'Riga'
57
```
58
0 commit comments