We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 5f583b0 commit 952ba9eCopy full SHA for 952ba9e
README.md
@@ -64,12 +64,20 @@ This allows you to use llama.cpp compatible models with any OpenAI compatible cl
64
65
To install the server package and get started:
66
67
+Linux
68
```bash
69
pip install llama-cpp-python[server]
70
export MODEL=./models/7B/ggml-model.bin
71
python3 -m llama_cpp.server
72
```
73
74
+Windows
75
+```cmd
76
+pip install llama-cpp-python[server]
77
+SET MODEL=\models\7B\ggml-model.bin
78
+python3 -m llama_cpp.server
79
+```
80
+
81
Navigate to [http://localhost:8000/docs](http://localhost:8000/docs) to see the OpenAPI documentation.
82
83
## Docker image
0 commit comments