An excellent project. May I ask if there are any plans to support the use of open-source locally deployed LLM APIs, such as Qwen on Ollama?

[futurelighthouse]

[Thibault-Knobloch]

Hello, thank you for the interest!
Yes I was thinking about adding this. I was thinking about adding a command to setup a local llm ('tibo local') which requires setting LOCAL_LLM to True/False and providing a LOCAL_LLM_URL for the user to specify the exact url (including the local port) where the local model is running to connect to it instead of using openai GPT4o-mini in the indexing and fetching parts of the tool.
This would mean you would have to have running locally on some port that local model, and connect to to the tibo tool with the command.

Would this be an interesting addition that solves your issue?

[futurelighthouse]

It looks like a good plan. Looking forward to its launch.

[Thibault-Knobloch]

It looks like a good plan. Looking forward to its launch.

Hello! I have pushed a commit with the initial version of this feature. You can run 'tibo local' and follow the steps to use a locally running llm in the 'tibo index' and 'tibo fetch' commands. It is not yet in online pypi cli tool, but you can try it out by cloning this repo and running 'python -m pip install -e .' in the root folder to locally download the latest tibo cli tool.

For the 'tibo agent', there is no option to use a local llm as it would most likely not be capable enough to handle tool calling. That feature still requires a ANTHROPIC_API_KEY to be set up.

Note: I tried with Qwen2.5:32b model locally and the execution time is a lot longer, will vary depending on your local setup. You can run some model loally using FlaskAPI server or ollama, both are supported just need the live URL to be configured in 'tibo local'.
Also: ensure your model has enough context window size, some prompts can be 5-10k tokens input if the code files you have are longer. And for models that are not pre-trained to follow instructions, you may have to adjust the input prompt length in the code and simplify it. Important to keep in mind.

[futurelighthouse]

I think “qwen2.5 coder” might be more appropriate. The test found that the execution of “tibo local” failed, with a prompt indicating the need for an openai_api_key.

[Thibault-Knobloch]

I think “qwen2.5 coder” might be more appropriate. The test found that the execution of “tibo local” failed, with a prompt indicating the need for an openai_api_key.

Thank you for the model hint, I will try it out!

I tried to reproduce the issue you mention but 'tibo local' for me has no mention of the openai key, just a simple flow to set environment for local model variable. Could you share maybe the terminal command and output so I can debug further?

Also a question: are you using the 'tibo' tool from pypi, or are you using the tool locally after cloning the repo and building the tool?

[Harrywwq]

File "/home/codebase-intelligence/tibo/cli.py", line 9, in
from tibo.agent.agent import start_agent_shell
File "/home/codebase-intelligence/tibo/agent/agent.py", line 5, in
from tibo.agent.claude_agent import ClaudeAgent
File "/home/codebase-intelligence/tibo/agent/claude_agent.py", line 4, in
from .tools.search_tool import web_search
File "/home/codebase-intelligence/tibo/agent/tools/search_tool.py", line 4, in
client = openai.OpenAI(api_key=OPENAI_API_KEY)
File "/home/miniconda3/lib/python3.13/site-packages/openai/_client.py", line 114, in init
raise OpenAIError(
"The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable"
)
openai.OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable

OPENAI_API_KEY is needed in file codebase-intelligence/tibo/agent/tools/search_tool.py