feat/move-to-function-calling

[olimorris]

Description

This PR will see the plugin move to native function calling (sometimes called tool use), in the adapters listed in the checklist section below. This will result in much more accurate tool usage for those adapters and likely result in more useful responses from models.

This PR will see the plugin move away from the XML schemas implementation that I built c. 12 months ago. The impact of this is that adapters that aren't listed below will no longer have access to tools within CodeCompanion.

As a solo maintainer, I can't support two different versions of a tools implementation at the same time, so I will be asking those affected by this PR to use a separate branch. I will keep the branch live and open to PRs from the community that add features from the main branch, keeping the two in sync (somewhat).

This PR is scoped to just add function calling for the adapters below although this is subject to change.

Checklist

Function calling added for:

• Anthropic
• Copilot
• DeepSeek
• Gemini
  Ollama
• OpenAI

[cloudflare-workers-and-pages]

Deploying codecompanion with    Cloudflare Pages

Latest commit:	6bafcaf
Status:	✅  Deploy successful!
Preview URL:	https://9d7c6d3d.codecompanion.pages.dev
Branch Preview URL:	https://feat-move-to-function-callin.codecompanion.pages.dev

View logs

[ravitemer]

Hi @olimorris. Nice to see that you have started function calling.

From the code, I see that tools no longer support system prompt. I understand that this keeps the tools super simple and easy to build. But will you consider keeping the system prompt? For e.g coming to mcphub.nvim's @mcp tool, whenever the user needs to use the tool we need to let the LLM know about the current active mcp servers to be called upon. Without the system prompt per tool if we need additional information specific for a tool we might need to add this additional information to the global system prompt. Currently we have to do some conditional logic inside the strategies.chat.system_prompt to check if the chat has mcp tool and if so add this additional information. Users may have their own custom prompts where they need to add this logic too in order for the mcp tool to work. If we can keep the system_prompt per tool as it is before, we can dynamically add additional info there. And also if we can evaluate the type = "system" messages from the messages array everytime we do chat:submit() this helps in for e.g in mcphub's case to let the user dynamically include or exclude necessary mcp servers and tools themselves so LLM will always have the currently active servers. Currently when the user does @mcp do something the servers active at the time are being stored in the system_prompt.

Are there any issues with keeping the architecture this way? I know you have to consider wider use cases. Eager to hear your thoughts on this.

Thank you

[olimorris]

Are there any issues with keeping the architecture this way? I know you have to consider wider use cases. Eager to hear your thoughts on this.

@ravitemer - none at all. I'll revert those changes back later today. For you and @Davidyz, I'm hoping the only changes will be on the schema side of things and I'm keen to keep it that way.

As I mentioned yesterday, this PR will not be merged until you both are happy with it.

[ravitemer]

Thank you 😃.

[Davidyz]

From my side, I'd also prefer to have a system prompt (whatever it's called, as long as there's a way for me to write instructions to teach the LLM how to call the tool). Otherwise, I'm quite happy with the existing implementation, so if there are no significant changes in the interface, it should be easy for me to keep up with.

To be completely honest, I think the VectorCode codecompanion tool should eventually be deprecated in favour of MCP. I've been receiving feedback from CopilotChat devs, and it looks like they're doing their own tool stuff. It'll be a lot easier if I could just maintain one tool implementation instead of multiple ones for each AI plugin. The MCP server is not quite ready though, so I'll still keep maintaining the tool for codecompanion and copilotchat for the time being.

[olimorris]

I think the VectorCode codecompanion tool should eventually be deprecated in favour of MCP

This certainly feels like the direction of travel. I can't open a newsletter or browse Reddit without seeing reference to MCP.

[S1M0N38]

A common pattern in LLM-related frameworks, when implementing function calling, is to leverage the type annotation to automatically parse the code of a function into the JSON representation of the function call.

For example:

• @agent.tool pydantic-ai
• @tool in smolagent
• @mcp.tool in mcp python-sdk

With the high level of code introspection offered by Python with the syntactic sugar offered by function decorators, this translates to the "@tool" decorator expected to be applied to a typed annotated function.

---

I wonder if something similar is achievable in Lua through Lua-language-server (which is already extensively used in this project judging by the Lua CATS scatter in the code base).

This would be considered simply a parser from Lua code to JSON/Lua-table LLM tool API.

flowchart LR
 subgraph s1["code2tool"]
    n1["lua function<br>with string annotation<br>(source code)"]
    n2["OpenAI tool repr. <br>(json or lua table)"] 
    n5["Anthropic tool repr. <br>(json or lua table)"] 
    n6["Other tool repr. <br>(json or lua table)"] 
    n1 --> n2
    n1 --> n5
    n1 --> n6
  end
    n2 -- opeai adapter --> n7
    n5 -- anthropic adapter --> n7
    n6 -- other adapter --> n7
    n7["codecompanion.nvim"]

Loading

---

This would simplify the creation of new tools: they are just Lua functions with annotations.

[olimorris]

@S1M0N38 thanks for sharing those links.

My plan will be to get OpenAI working first, executing the tools as expected, ensure i've got test coverage etc. Then I'll investigate the other adapters. I'm hoping that I can have a transformation class that can map between the adapters although Gemini's looks suitably different that this may be troublesome. I'll hopefully know early next week.

[olimorris]

Docs updated. Video re-recorded and edit. This PR is complete and once #1290 is ready then we'll merge.

@Davidyz - You happy from a VectorCode perspective?

[Davidyz]

Yep and I've merged the has check for compatibility. Still need to move some things around for the tool calling branch, but nothing major (I mean the issue with some providers not doing tool use properly persists but that's their fault). I'm happy where we are about this PR.

[Davidyz]

I've done the PR for the function-calling VectorCode tool and I'm looking into openai-compatible adapters, and it looks like there's no tools field in the request body. The tool is correctly picked up in the chat:

## Me

> Context:
> - <tool>vectorcode</tool>

@vectorcode reranker implementation in this project?

but in the request body there's no tools field

{
	"model":"Qwen/Qwen2.5-72B-Instruct-128K",
	"stream_options":{"include_usage":true},
	"stream":true,
	"messages":[
		// some messages here
	]
}

[olimorris]

I've done the PR for the function-calling VectorCode tool and I'm looking into openai-compatible adapters, and it looks like there's no tools field in the request body. The tool is correctly picked up in the chat:

Should be good now

[Davidyz]

Sadly no, but the following patch from gemini did the trick:

diff --git a/lua/codecompanion/adapters/openai_compatible.lua b/lua/codecompanion/adapters/openai_compatible.lua
index ce57f2f..9a12bb1 100644
--- a/lua/codecompanion/adapters/openai_compatible.lua
+++ b/lua/codecompanion/adapters/openai_compatible.lua
@@ -123,6 +123,9 @@ return {
     form_messages = function(self, messages)
       return openai.handlers.form_messages(self, messages)
     end,
+    form_tools = function(self, tools)
+      return openai.handlers.form_tools(self, tools)
+    end,
     chat_output = function(self, data, tools)
       return openai.handlers.chat_output(self, data, tools)
     end,

with this patch the gemini 2.0 flash exp on openrouter works flawlessly.
I also tried with qwen2.5 72b from a different openai-compatible provider, and the wrong tool call format froze the chat (Couldn't decode the tool arguments error).

[olimorris]

Ah so dumb of me. Included that in the latest commit.

I also tried with qwen2.5 72b from a different openai-compatible provider, and the wrong tool call format froze the chat

So the provider isn't OpenAI-compatible when it comes to tools? Or is there an issue in how we're putting the tool calls back in the messages request?

[Davidyz]

So the provider isn't OpenAI-compatible when it comes to tools? Or is there an issue in how we're putting the tool calls back in the messages request?

[ERROR] 2025-04-28 17:16:24
Couldn't decode the tool arguments:  {"command": "query", "options": {"query": ["vectorcode", "reranker", "implementation"], "count": 15, "project_root": "/home/davidyz/git/VectorCode"}}}

There was an extra } in the response, which broke the json syntax. I remember in the xml-based tools when there's a decoding error the error message will just be added to the chat (prob. invisible?) so that the model knows it's wrong.

EDIT: deepseek from the same provider worked fine so it's just qwen2.5 being dumb. It'll be nice to have the aforementioned fail-safe when the tool putout can't be decoded tho.

btw, with the outcome we're getting from this PR, I'll merge Davidyz/VectorCode#59 by tmr. It'll work for both function-calling and xml-based tools, depending on codecompanion.has.

[olimorris]

There was an extra } in the response, which broke the json syntax. I remember in the xml-based tools when there's a decoding error the error message will just be added to the chat (prob. invisible?) so that the model knows it's wrong.

Okay and when you say "froze the chat", I assume the chat buffer wasn't unlocked, ready for you to add another user prompt?

I must have run 100+ tests now and I haven't had a model produce incorrect JSON (I thought they were fine-tuned to be able to call functions ffs 😆). So, I'll make sure we let the LLM know of the error in the chat.

[Davidyz]

Okay and when you say "froze the chat", I assume the chat buffer wasn't unlocked, ready for you to add another user prompt?

It's stuck here:

## Me

> Context:
> - <tool>vectorcode</tool>

@vectorcode reranker implementation?

## CodeCompanion (OpenAI Compatible)

To find the implementation of the reranker in the VectorCode project, I will perform a semantic search with the following keywords: "reranker implementation". This should retrieve files that are relevant to the reranker functionality.

<tool_call>

with the buffer lock and no ## Me header for me to type under. Funny enough, I was able to consistently reproduce this with qwen2.5-72b from this provider (siliconflow). I'll try this with some other providers/models and make some final adjustments to the system prompt, and then I'll merge the updated vectorcode tool.

[olimorris]

Okay and when you say "froze the chat", I assume the chat buffer wasn't unlocked, ready for you to add another user prompt?

It's stuck here:

## Me

> Context:
> - <tool>vectorcode</tool>

@vectorcode reranker implementation?

## CodeCompanion (OpenAI Compatible)

To find the implementation of the reranker in the VectorCode project, I will perform a semantic search with the following keywords: "reranker implementation". This should retrieve files that are relevant to the reranker functionality.

<tool_call>

with the buffer lock and no ## Me header for me to type under. Funny enough, I was able to consistently reproduce this with qwen2.5-72b from this provider (siliconflow). I'll try this with some other providers/models and make some final adjustments to the system prompt, and then I'll merge the updated vectorcode tool.

I've addressed this now. If the JSON can't be decoded then an error's thrown, the chat buffer's updated and the lock is released. I'll handle any additional errors that arise as users raise them after the merge.

[jacobdaddario]

🚀