Feature Request: Support for function calling in llama-server

[vijaysaayi]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

Could you add support for function calling supported in llama.cpp ?

• https://github.com/ggml-org/llama.cpp/blob/master/docs/function-calling.md

Currently it is not supported.

• ik_llama.cpp/examples/server/utils.hpp

  Line 394 in 36e6e88

  static const std::vector<std::string> unsupported_params { "tools", "tool_choice" };

Motivation

Tool calling will enable agent scenarios.

Possible Implementation

ggml-org/llama.cpp#9639

[ikawrakow]

I have never used function calling myself, so I'm not familiar with this feature.

Help will be appreciated.

[vijaysaayi]

Thanks for all the efforts on this. Would it be possible to update to latest llama.cpp (These functionalities are implemented)

[ikawrakow]

The code here has not been synced with llama.cpp since last August, and as a result the two code bases have totally diverged. Almost nothing is just a copy/paste from upstream.

[ubergarm]

@vijaysaayi Check out this wrapper/reverse-proxy which might be able to do what you want: #403 (comment)

[vijaysaayi]

Thanks for sharing this. I will check this out.

[mtcl]

@vijaysaayi let me know if you need any help with the function calling wrapper. Here is the video walkthrough of it. https://www.youtube.com/watch?v=JGo9HfkzAmc

[saood06]

With #723 merged, is this solved for you?

[ikawrakow]

Stale issue -> closing