Skip to content

Commit 7ec8170

Browse files
chrarnoldusJean Carlo Nascimento
chrarnoldus
authored and
Jean Carlo Nascimento
committed
Merge pull request Kilo-Org#2170 from mcowger/mcowger/ollamaContext
Remove the forced override of the context limit for Ollama API Kilo-Org#2060
2 parents 6e52408 + d0efa75 commit 7ec8170

File tree

8 files changed

+206
-142
lines changed

8 files changed

+206
-142
lines changed

‎.changeset/twenty-tips-tell.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"kilo-code": patch
3+
---
4+
5+
Remove the forced override of the context limit for Ollama API

‎apps/kilocode-docs/docs/providers/ollama.md‎

Lines changed: 57 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -19,28 +19,25 @@ More trial and error will be required to find the right prompt.
1919
Local LLMs are usually also not very fast.
2020
Using simple prompts, keeping conversations short and disabling MCP tools can result in a speed-up.
2121

22-
2322
## Hardware Requirements
2423

2524
You will need a large amount of RAM (32GB or more) and a powerful CPU (e.g. Ryzen 9000 series) to run the models listed below.
2625
GPUs can run LLMs much faster, but a large amount of VRAM is required (24GB, if not more), which is not very common on consumer GPUs.
2726
Smaller models will run on more modest GPUs, but do not provide good results.
2827
MacBooks with a sufficient amount of unified memory can use GPU-acceleration, but do not outperform high-end desktop CPUs in our testing.
2928

30-
3129
## Selecting a Model
3230

3331
Ollama supports many different models.
3432
You can find a list of available models on the [Ollama website](https://ollama.com/library).
3533
Selecting a model that suits your use case, runs on your hardware configuration and achieves the desired speed requires some trial and error.
3634
The following rules and heuristics can be used to find a model:
3735

38-
* Must have at least a 32k context window (this is a requirement for Kilo Code).
39-
* Listed as supporting tools.
40-
* Number of parameters in the 7b to 24b range.
41-
* Prefer popular models.
42-
* Prefer newer models.
43-
36+
- Must have at least a 32k context window (this is a requirement for Kilo Code).
37+
- Listed as supporting tools.
38+
- Number of parameters in the 7b to 24b range.
39+
- Prefer popular models.
40+
- Prefer newer models.
4441

4542
### Recommendations for Kilo Code
4643

@@ -52,12 +49,12 @@ Create a simple web page with a button that greets the user when clicked.
5249

5350
A model is considered to pass if it produces a working result within a few tries. The models we found to work correctly are:
5451

55-
| Model name | Completion time |
56-
| --- | --- |
57-
| qwen2.5-coder:7b | 1x (baseline) |
58-
| devstral:24b | 2x |
59-
| gemma3:12b | 4x |
60-
| qwen3-8b | 12x |
52+
| Model name | Completion time |
53+
| ---------------- | --------------- |
54+
| qwen2.5-coder:7b | 1x (baseline) |
55+
| devstral:24b | 2x |
56+
| gemma3:12b | 4x |
57+
| qwen3-8b | 12x |
6158

6259
Our recommendation is to use **devstral:24b** if your hardware can handle it, because it makes fewer mistakes than qwen2.5-coder:7b.
6360
qwen2.5-coder:7b is worth considering because of its speed, if you can put up with its mistakes.
@@ -69,49 +66,56 @@ The result produced by devstral:24b is included below:
6966
```html
7067
<!DOCTYPE html>
7168
<html lang="en">
72-
<head>
73-
<meta charset="UTF-8">
74-
<meta name="viewport" content="width=device-width, initial-scale=1.0">
75-
<title>Greet User Button</title>
76-
<style>
77-
body {
78-
font-family: Arial, sans-serif;
79-
display: flex;
80-
justify-content: center;
81-
align-items: center;
82-
height: 100vh;
83-
margin: 0;
84-
}
85-
button {
86-
padding: 10px 20px;
87-
font-size: 16px;
88-
cursor: pointer;
89-
}
90-
</style>
91-
</head>
92-
<body>
93-
<button onclick="greetUser()">Greet Me!</button>
94-
95-
<script>
96-
function greetUser() {
97-
alert('Hello! Welcome to our website.');
98-
}
99-
</script>
100-
</body>
69+
<head>
70+
<meta charset="UTF-8" />
71+
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
72+
<title>Greet User Button</title>
73+
<style>
74+
body {
75+
font-family: Arial, sans-serif;
76+
display: flex;
77+
justify-content: center;
78+
align-items: center;
79+
height: 100vh;
80+
margin: 0;
81+
}
82+
button {
83+
padding: 10px 20px;
84+
font-size: 16px;
85+
cursor: pointer;
86+
}
87+
</style>
88+
</head>
89+
<body>
90+
<button onclick="greetUser()">Greet Me!</button>
91+
92+
<script>
93+
function greetUser() {
94+
alert("Hello! Welcome to our website.")
95+
}
96+
</script>
97+
</body>
10198
</html>
10299
```
103100

104101
The following models look like reasonable choices, but were found to **not** work properly with Kilo Code in its default configuration:
105102

106-
| Model name | Fail reason |
107-
| --- | --- |
108-
| deepseek-r1:7b | fails to use tools properly |
103+
| Model name | Fail reason |
104+
| -------------- | ------------------------------ |
105+
| deepseek-r1:7b | fails to use tools properly |
109106
| deepseek-r1:8b | gets stuck in a reasoning loop |
110107

108+
## Preventing prompt truncation
109+
110+
By default Ollama truncates prompts to a very short length.
111+
If you run into this problem, please see this FAQ item to resolve it:
112+
[How can I specify the context window size?](https://github.com/ollama/ollama/blob/4383a3ab7a075eff78b31f7dc84c747e2fcd22b8/docs/faq.md#how-can-i-specify-the-context-window-size)
113+
114+
If you decide to use the `OLLAMA_CONTEXT_LENGTH` environment variable, it needs to be visible to both the IDE and the Ollama server.
111115

112116
## Setting up Ollama
113117

114-
1. **Download and Install Ollama:** Download the Ollama installer for your operating system from the [Ollama website](https://ollama.com/). Follow the installation instructions and make sure Ollama is running:
118+
1. **Download and Install Ollama:** Download the Ollama installer for your operating system from the [Ollama website](https://ollama.com/). Follow the installation instructions and make sure Ollama is running:
115119

116120
```bash
117121
ollama serve
@@ -129,13 +133,12 @@ The following models look like reasonable choices, but were found to **not** wor
129133
ollama pull devstral:24b
130134
```
131135
132-
4. **Configure Kilo Code:**
133-
* Open the Kilo Code sidebar (<img src="/docs/img/kilo-v1.svg" width="12" /> icon).
134-
* Click the settings gear icon (<Codicon name="gear" />).
135-
* Select "ollama" as the API Provider.
136-
* Enter the Model name.
137-
* (Optional) You can configure the base URL if you're running Ollama on a different machine. The default is `http://localhost:11434`.
138-
136+
3. **Configure Kilo Code:**
137+
- Open the Kilo Code sidebar (<img src="/docs/img/kilo-v1.svg" width="12" /> icon).
138+
- Click the settings gear icon (<Codicon name="gear" />).
139+
- Select "ollama" as the API Provider.
140+
- Enter the Model name.
141+
- (Optional) You can configure the base URL if you're running Ollama on a different machine. The default is `http://localhost:11434`.
139142

140143
## Further Reading
141144

‎package.json‎

Lines changed: 71 additions & 71 deletions
Original file line numberDiff line numberDiff line change
@@ -1,72 +1,72 @@
11
{
2-
"name": "kilo-code",
3-
"packageManager": "[email protected]",
4-
"engines": {
5-
"node": "20.19.2"
6-
},
7-
"scripts": {
8-
"preinstall": "node scripts/bootstrap.mjs",
9-
"prepare": "husky",
10-
"install": "node scripts/bootstrap.mjs",
11-
"install:all": "node scripts/bootstrap.mjs",
12-
"lint": "turbo lint --log-order grouped --output-logs new-only",
13-
"check-types": "turbo check-types --log-order grouped --output-logs new-only",
14-
"test": "turbo test --log-order grouped --output-logs new-only",
15-
"format": "turbo format --log-order grouped --output-logs new-only",
16-
"build": "pnpm vsix",
17-
"bundle": "turbo bundle --log-order grouped --output-logs new-only",
18-
"bundle:nightly": "turbo bundle:nightly --log-order grouped --output-logs new-only",
19-
"vsix:production": "turbo vsix --log-order grouped --output-logs new-only --force",
20-
"vsix": "turbo vsix --log-order grouped --output-logs new-only",
21-
"vsix:nightly": "turbo vsix:nightly --log-order grouped --output-logs new-only",
22-
"clean": "turbo clean --log-order grouped --output-logs new-only && rimraf dist out bin .vite-port .turbo",
23-
"install:vsix": "pnpm install --frozen-lockfile && pnpm clean && pnpm vsix && node scripts/install-vsix.js",
24-
"changeset:version": "cp CHANGELOG.md src/CHANGELOG.md && changeset version && cp -vf src/CHANGELOG.md .",
25-
"knip": "knip --include files",
26-
"update-contributors": "node scripts/update-contributors.js",
27-
"evals": "dotenvx run -f packages/evals/.env.development packages/evals/.env.local -- docker compose -f packages/evals/docker-compose.yml --profile server --profile runner up --build --scale runner=0",
28-
"playwright": "turbo playwright",
29-
"npm:publish:types": "pnpm --filter @roo-code/types npm:publish",
30-
"link-workspace-packages": "tsx scripts/link-packages.ts",
31-
"unlink-workspace-packages": "tsx scripts/link-packages.ts --unlink",
32-
"docs:start": "pnpm --filter kilocode-docs start",
33-
"docs:build": "pnpm --filter kilocode-docs build"
34-
},
35-
"devDependencies": {
36-
"@changesets/changelog-github": "^0.5.1",
37-
"@changesets/cli": "^2.27.10",
38-
"@dotenvx/dotenvx": "^1.34.0",
39-
"@roo-code/config-typescript": "workspace:^",
40-
"@types/glob": "^9.0.0",
41-
"@types/node": "^24.1.0",
42-
"@vscode/vsce": "3.3.2",
43-
"esbuild": "^0.25.0",
44-
"eslint": "^9.27.0",
45-
"glob": "^11.0.3",
46-
"husky": "^9.1.7",
47-
"knip": "^5.44.4",
48-
"lint-staged": "^16.0.0",
49-
"mkdirp": "^3.0.1",
50-
"only-allow": "^1.2.1",
51-
"ovsx": "0.10.4",
52-
"prettier": "^3.4.2",
53-
"rimraf": "^6.0.1",
54-
"tsx": "^4.19.3",
55-
"turbo": "^2.5.3",
56-
"typescript": "^5.4.5"
57-
},
58-
"lint-staged": {
59-
"*.{js,jsx,ts,tsx,json,css,md}": [
60-
"prettier --write"
61-
]
62-
},
63-
"pnpm": {
64-
"overrides": {
65-
"tar-fs": ">=2.1.3",
66-
"esbuild": ">=0.25.0",
67-
"brace-expansion": ">=2.0.2",
68-
"form-data": ">=4.0.4",
69-
"bluebird": ">=3.7.2"
70-
}
71-
}
72-
}
2+
"name": "kilo-code",
3+
"packageManager": "[email protected]",
4+
"engines": {
5+
"node": "20.19.2"
6+
},
7+
"scripts": {
8+
"preinstall": "node preinstall.js",
9+
"prepare": "husky",
10+
"install": "node scripts/bootstrap.mjs",
11+
"install:all": "node scripts/bootstrap.mjs",
12+
"lint": "turbo lint --log-order grouped --output-logs new-only",
13+
"check-types": "turbo check-types --log-order grouped --output-logs new-only",
14+
"test": "turbo test --log-order grouped --output-logs new-only",
15+
"format": "turbo format --log-order grouped --output-logs new-only",
16+
"build": "pnpm vsix",
17+
"bundle": "turbo bundle --log-order grouped --output-logs new-only",
18+
"bundle:nightly": "turbo bundle:nightly --log-order grouped --output-logs new-only",
19+
"vsix:production": "turbo vsix --log-order grouped --output-logs new-only --force",
20+
"vsix": "turbo vsix --log-order grouped --output-logs new-only",
21+
"vsix:nightly": "turbo vsix:nightly --log-order grouped --output-logs new-only",
22+
"clean": "turbo clean --log-order grouped --output-logs new-only && rimraf dist out bin .vite-port .turbo",
23+
"install:vsix": "pnpm install --frozen-lockfile && pnpm clean && pnpm vsix && node scripts/install-vsix.js",
24+
"changeset:version": "cp CHANGELOG.md src/CHANGELOG.md && changeset version && cp -vf src/CHANGELOG.md .",
25+
"knip": "knip --include files",
26+
"update-contributors": "node scripts/update-contributors.js",
27+
"evals": "dotenvx run -f packages/evals/.env.development packages/evals/.env.local -- docker compose -f packages/evals/docker-compose.yml --profile server --profile runner up --build --scale runner=0",
28+
"playwright": "turbo playwright",
29+
"npm:publish:types": "pnpm --filter @roo-code/types npm:publish",
30+
"link-workspace-packages": "tsx scripts/link-packages.ts",
31+
"unlink-workspace-packages": "tsx scripts/link-packages.ts --unlink",
32+
"docs:start": "pnpm --filter kilocode-docs start",
33+
"docs:build": "pnpm --filter kilocode-docs build"
34+
},
35+
"devDependencies": {
36+
"@changesets/changelog-github": "^0.5.1",
37+
"@changesets/cli": "^2.27.10",
38+
"@dotenvx/dotenvx": "^1.34.0",
39+
"@roo-code/config-typescript": "workspace:^",
40+
"@types/glob": "^9.0.0",
41+
"@types/node": "^24.1.0",
42+
"@vscode/vsce": "3.3.2",
43+
"esbuild": "^0.25.0",
44+
"eslint": "^9.27.0",
45+
"glob": "^11.0.3",
46+
"husky": "^9.1.7",
47+
"knip": "^5.44.4",
48+
"lint-staged": "^16.0.0",
49+
"mkdirp": "^3.0.1",
50+
"only-allow": "^1.2.1",
51+
"ovsx": "0.10.4",
52+
"prettier": "^3.4.2",
53+
"rimraf": "^6.0.1",
54+
"tsx": "^4.19.3",
55+
"turbo": "^2.5.3",
56+
"typescript": "^5.4.5"
57+
},
58+
"lint-staged": {
59+
"*.{js,jsx,ts,tsx,json,css,md}": [
60+
"prettier --write"
61+
]
62+
},
63+
"pnpm": {
64+
"overrides": {
65+
"tar-fs": ">=2.1.3",
66+
"esbuild": ">=0.25.0",
67+
"brace-expansion": ">=2.0.2",
68+
"form-data": ">=4.0.4",
69+
"bluebird": ">=3.7.2"
70+
}
71+
}
72+
}

0 commit comments

Comments
 (0)