[BUG] Codebase Search Tool Not Being Called with Local Ollama Setup

[dcroswell]

Problem (one or two sentences)

Issue:
The LLM never calls the codebase_search tool. Instead, it:

• Guesses file locations based on directory structure
• Hallucinates code content (generates fake FastAPI/Flask/Click code that doesn't exist)
• Enters an infinite loop, asking follow-up questions
• Treats codebase_search as a bash command when explicitly asked to use it

Additional Observations:
No "Roo: Index Codebase" or similar commands available in the command palette
Extension activates successfully, but doesn't appear in extension host activation logs

Context (who is affected and when)

Environment:

• Roo Code version: 3.28.14
• OS: Windows 11 with WSL2 (Ubuntu)
• VSCode: Running in WSL remote
• API Provider: Ollama (local)
• Model: llama3.1:8b-instruct-q4_K_M
• Embedder: Ollama all-minilm (384 dimensions)
• The code gets indexed into Qdrant and shows in the extension
• Vector DB: Qdrant (local Docker container)
• Ollama endpoint: http://localhost:11434/
• Qdrant endpoint: http://localhost:6333/

Verification Steps Completed:
✅ Qdrant has 4,561 indexed points in collection ws-dd9beb7f436a9121
✅ Ollama embedder works (generates correct 384-dim vectors)
✅ Direct Qdrant search returns correct results with proper file paths and code chunks
✅ WSL networking confirmed working (localhost resolves correctly)
✅ Extension shows as "Enabled on WSL: Ubuntu"

Reproduction steps

Configure Roo Code with local Ollama and Qdrant as described
Ask: "Find the main function in my codebase"
Observe LLM guessing instead of searching

Expected result

LLM should call the codebase_search tool and use the actual search results from Qdrant.

Actual result

LLM has no awareness of available tools. When asked "What tools do you have available?", it lists project files instead of its own tools. Extension host logs show no tool-related errors.

Variations tried (optional)

Tried different models for embedding and spent a while going over any configs that may be interfering with the extension. Indexing happens through the extension fine. The LLM seems to not get the results or know where to search even though the endpont for Qdrant was configured and tested.

App Version

3.28.14

API Provider (optional)

Ollama

Model Used (optional)

llama3.1:8b-instruct-q4_K_M

Roo Code Task Links (optional)

No response

Relevant logs or errors (optional)

The answers would be guesses, and then Roo Code goes into a loop of asking for more information.
After the hallucination of code it would display an API Request message containing:
[ERROR] You did not use a tool in your previous response! Please retry with a tool use.
# Reminder: Instructions for Tool Use
Tool uses are formatted using XML-style tags. The tool name itself becomes the XML tag name. Each parameter is enclosed within its own set of tags. Here's the structure:
<actual_tool_name>
<parameter1_name>value1</parameter1_name>
<parameter2_name>value2</parameter2_name>
...
</actual_tool_name>

[roomote]

I'm exploring a potential fix for the codebase_search tool not being called with local Ollama setup. I'll investigate the tool registration and invocation logic to ensure it works properly with local LLM providers.

[roomote]

Hi @dcroswell,

I've opened PR #8431 with a fix for this issue: #8431

The fix adds enhanced, explicit tool use instructions specifically for local models (Ollama and LMStudio) that struggle with the XML-formatted tool calls. The solution:

1. Detects local models by checking if the API provider is 'ollama' or 'lmstudio'

2. Provides stricter instructions with:

   • MANDATORY requirements that every response MUST contain exactly one tool use
   • Explicit warnings against adding explanatory text outside of tool XML tags
   • Clear examples of CORRECT vs INCORRECT responses
   • Emphasis on using codebase_search first when exploring code

3. Maintains backward compatibility - other providers continue to receive the standard instructions

The implementation has been tested with comprehensive unit tests and all CI checks are running. This should resolve the issues you were experiencing with llama3.1:8b-instruct-q4_K_M not calling the codebase_search tool properly.

Please feel free to test the changes once the PR is merged, or you can test from the branch directly if you'd like to verify the fix sooner. Feedback is welcome!

[dcroswell]

Update: Tested with v3.28.15 - Issue Still Present

I've tested the latest version 3.28.15 and can confirm the issue persists with Ollama models.

Testing Summary

Environment:

• Roo Code version: 3.28.15
• OS: Windows 11 with WSL2 (Ubuntu)
• Ollama endpoint: http://localhost:11434
• Qdrant: Running locally in Docker (http://localhost:6333)
• Qdrant collection: 4,561 indexed points confirmed working

Models Tested (all failed):

1. llama3.1:8b-instruct-q4_K_M - Original model from issue
2. llama3.2:latest - Attempted tool use but failed with "missing values for required parameters"
3. qwen2.5-coder:7b - Custom Modelfile with optimised parameters (temperature 0.2, repeat_penalty 1.2, num_ctx 32768)

Observed Behaviour:

• LLM does not recognise available tools (lists project files/services instead of tools when asked "what tools do you have available?")
• When explicitly asked to use the read_file tool, it treats it as a bash command
• Hallucinates code content instead of reading actual files
• Gets stuck in loops, asking for clarification or requesting follow-up information

Confirmation with Claude API:
Tested with Claude Sonnet 4.5 using the same Roo Code setup - tools work perfectly. Claude successfully:

• Used read_file tool without issues
• Retrieved actual file contents (not hallucinated)
• Properly understood and executed tool calls

Conclusion

This confirms the issue is specifically with Ollama model compatibility, not a general Roo Code bug. The tool system works correctly with Claude models but fails with local Ollama models, even with parameter optimisation.

The root cause appears to be that Roo Code uses a custom XML-based tool calling format that smaller local models struggle to follow reliably, despite Ollama supporting standard tool calling APIs.

[daniel-lxs]

Thanks for creating this issue! It appears that the model you're using can't handle large and complex prompts such as our system prompt. We are always looking into ways of improving performance on weaker models.