You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/advanced-usage/local-models.md
+26-2Lines changed: 26 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -53,6 +53,7 @@ Roo Code currently supports two main local model providers:
53
53
54
54
3. **Configure the Model:** by default, Ollama uses a context window size of 2048 tokens, which is too small for Roo Code requests. You need to have at least 12k to get decent results, ideally - 32k. To configure a model, you actually need to set its parameters and save a copy of it.
55
55
56
+
##### Using Ollama runtime
56
57
Load the model (we will use `qwen2.5-coder:32b` as an example):
57
58
58
59
```bash
@@ -70,13 +71,36 @@ Roo Code currently supports two main local model providers:
70
71
```bash
71
72
/save your_model_name
72
73
```
73
-
74
+
##### Using Ollama command line
75
+
Alternatively, you can write all your settings into a text file and generate the model in the command-line.
76
+
77
+
78
+
Create a text file with model settings, and save it (~/qwen2.5-coder-32k.txt). Here we've only used the `num_ctx` parameter, but you could include more parameters on the next line using the `PARAMETER name value` syntax.
79
+
80
+
```text
81
+
FROM qwen2.5-coder:32b
82
+
# sets the context window size to 32768, this controls how many tokens the LLM can use as context to generate the next token
83
+
PARAMETER num_ctx 32768
84
+
```
85
+
Change directory to the `.ollama/models` directory. On most Macs, thats `~/.ollama/models` by default (`%HOMEPATH%\.ollama` on Windows).
86
+
87
+
```bash
88
+
cd ~/.ollama/models
89
+
```
90
+
91
+
Create your model from the settings text file you created. The syntax is `ollama create (name of the model you want to see) -f (text file with settings)`
* Open the Roo Code sidebar (<Codicon name="rocket" /> icon).
77
101
* Click the settings gear icon (<Codicon name="gear" />).
78
102
* Select "ollama" as the API Provider.
79
-
* Enter the Model name from the previous step (e.g., `your_model_name`).
103
+
* Enter the Model name from the previous step (e.g., `your_model_name`) or choose it from the radio button list that should appear below `Model ID` if Ollama is currently running.
80
104
* (Optional) You can configure the base URL if you're running Ollama on a different machine. The default is `http://localhost:11434`.
81
105
* (Optional) Configure Model context size in Advanced settings, so Roo Code knows how to manage its sliding window.
0 commit comments