fix: use native Ollama API instead of OpenAI compatibility layer #7137
Conversation
- Implements native Ollama API using the official ollama npm package - Fixes issue #7070 where models like gpt-oss:120b failed with OpenAI routes - Based on the approach successfully used by Kilo-Org/kilocode - Maintains backward compatibility by keeping old handler available - Adds comprehensive tests for the new implementation Credits: Solution inspired by Kilo-Org/kilocode's implementation Fixes #7070
dosubot
bot
added
size:L
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
src/api/index.ts
Outdated
| @@ -13,7 +13,7 @@ import { | |||
| VertexHandler, | |||
| AnthropicVertexHandler, | |||
| OpenAiHandler, | |||
| OllamaHandler, | |||
| // OllamaHandler, // Replaced with NativeOllamaHandler | |||
There was a problem hiding this comment.
Remove the commented-out 'OllamaHandler' import if it’s no longer needed.
| // OllamaHandler, // Replaced with NativeOllamaHandler |
This comment was generated because it violated a code review rule: irule_Vw7dJWzvznOJagxS.
src/api/providers/native-ollama.ts
Outdated
| import { BaseProvider } from "./base-provider" | ||
| import type { ApiHandlerOptions } from "../../shared/api" | ||
| import { getOllamaModels } from "./fetchers/ollama" | ||
| import { getApiRequestTimeout } from "./utils/timeout-config" |
There was a problem hiding this comment.
Remove the unused import 'getApiRequestTimeout' to clean up the code.
| import { getApiRequestTimeout } from "./utils/timeout-config" |
![roomote[bot]](https://avatars.githubusercontent.com/in/1546624?s=60&v=4){kind=small}
src/api/providers/native-ollama.ts
Outdated
| import { BaseProvider } from "./base-provider" | ||
| import type { ApiHandlerOptions } from "../../shared/api" | ||
| import { getOllamaModels } from "./fetchers/ollama" | ||
| import { getApiRequestTimeout } from "./utils/timeout-config" |
There was a problem hiding this comment.
Is this import needed? I don't see getApiRequestTimeout being used anywhere in this file. Could we remove it to keep the imports clean?
src/api/providers/native-ollama.ts
Outdated
| return ollamaMessages | ||
| } | ||
|
|
||
| const OLLAMA_TIMEOUT_MS = 3_600_000 |
There was a problem hiding this comment.
This constant OLLAMA_TIMEOUT_MS is defined but never used. Is this intentional for future use, or should we either use it (perhaps pass it to the Ollama client configuration) or remove it?
| } | ||
| } | ||
|
|
||
| async fetchModel() { |
There was a problem hiding this comment.
I notice the fetchModel() method fetches models but doesn't seem to be called anywhere in the codebase. Is this intentional for future use, or should it be integrated into the initialization flow to populate the models cache?
src/api/providers/native-ollama.ts
Outdated
| if (part.type === "image") { | ||
| // Handle base64 images only (Anthropic SDK uses base64) | ||
| if ("source" in part && part.source.type === "base64") { | ||
| return `data:${part.source.media_type};base64,${part.source.data}` |
There was a problem hiding this comment.
The image handling here returns a data URL string directly in the content field. Could we verify this is the correct format expected by Ollama's API for image inputs? The comment mentions base64 handling, but I want to make sure this aligns with Ollama's expectations.
| expect(model.info).toBeDefined() | ||
| }) | ||
| }) | ||
| }) |
There was a problem hiding this comment.
Great test coverage! Consider adding a few more edge case tests:
- Image message conversion (testing the base64 image handling)
- Tool message handling
- Multiple concurrent requests
- Network timeout scenarios
These would help ensure robustness in production scenarios.
- Remove unused imports (getApiRequestTimeout) and constants (OLLAMA_TIMEOUT_MS) - Call fetchModel() in createMessage and completePrompt to load model info - Fix image handling to use raw base64 strings instead of data URLs - Remove commented-out OllamaHandler import - Properly separate text and images in message conversion
github-project-automation
bot
moved this from PR [Needs Review]
to Done
in Roo Code Roadmap
* main: (70 commits) fix: use native Ollama API instead of OpenAI compatibility layer (RooCodeInc#7137) feat: add support for OpenAI gpt-5-chat-latest model (RooCodeInc#7058) Make enhance with task history default to true (RooCodeInc#7140) Bump cloud version to 0.16.0 (RooCodeInc#7135) Release: v1.51.0 (RooCodeInc#7130) Add an API for resuming tasks by ID (RooCodeInc#7122) Add support for task page event population (RooCodeInc#7117) fix: add type check before calling .match() on diffItem.content (RooCodeInc#6905) (RooCodeInc#6906) Fix: Enable save button for provider dropdown and checkbox changes (RooCodeInc#7113) fix: Use cline.cwd as primary source for workspace path in codebaseSearchTool (RooCodeInc#6902) Hotfix multiple folder workspace checkpoint (RooCodeInc#6903) fix: prevent XML entity decoding in diff tools (RooCodeInc#7107) (RooCodeInc#7108) Refactor task execution system: improve call stack management (RooCodeInc#7035) Changeset version bump (RooCodeInc#7104) feat(web): fill missing SEO-related values (RooCodeInc#7096) Update contributors list (RooCodeInc#6883) Release v3.25.15 (RooCodeInc#7103) fix: add /evals page to sitemap generation (RooCodeInc#7102) feat: implement sitemap generation in TypeScript and remove XML file (RooCodeInc#6206) fix: reset condensing state when switching tasks (RooCodeInc#6922) ...
hannesrudolph
added
the
Issue/PR - Triage
Description
This PR fixes issue #7070 where Ollama models (like
gpt-oss:120b) were incorrectly using OpenAI-compatible routes instead of native Ollama API endpoints.Problem
When using models like
gpt-oss:120bwith Ollama, the plugin was trying to get completions from OpenAI routes (/v1) instead of the native Ollama API endpoint. This issue was reported by @LivioGama who noted that "Does not happen on Kilo Code".Solution
After investigating, I discovered that Kilo-Org/kilocode had already solved this by using the official
ollamanpm package for native API access. This PR adapts their approach to our codebase.Changes Made:
ollamanpm package (v0.5.17) as a dependencysrc/api/providers/native-ollama.ts- New handler using native Ollama SDKsrc/api/index.ts- Switched to useNativeOllamaHandlerfor ollama providersrc/api/providers/__tests__/native-ollama.spec.tsOllamaHandlerremains available but unusedKey Features:
Testing
Credits
This solution was inspired by Kilo-Org/kilocode's implementation.
Related
Breaking Changes
None - the change is transparent to users and maintains full backward compatibility.
Important
Replaces OpenAI compatibility layer with native Ollama API for Ollama models, introducing
NativeOllamaHandlerand adding comprehensive tests.src/api/index.ts.NativeOllamaHandlerinnative-ollama.tsfor direct API communication.OllamaHandler.NativeOllamaHandlerinnative-ollama.spec.ts.ollamanpm package (v0.5.17) topackage.json.This description was created by
for 5c83d3a. You can customize this summary. It will automatically update as commits are pushed.