Skip to content

Commit 68eeff1

Browse files
author
ochafik
committed
Update function-calling.md
1 parent 36ecb01 commit 68eeff1

File tree

1 file changed

+11
-8
lines changed

1 file changed

+11
-8
lines changed

docs/function-calling.md

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -325,6 +325,9 @@ To get the official template from original HuggingFace repos, you can use [scrip
325325
> [!TIP]
326326
> If there is no official `tool_use` Jinja template, you may want to set `--chat-template chatml` to use a default that works with many models (YMMV!), or write your own (e.g. we provide a custom [llama-cpp-deepseek-r1.jinja](../models/templates/llama-cpp-deepseek-r1.jinja) for DeepSeek R1 distills)
327327
328+
> [!CAUTION]
329+
> Beware of extreme KV quantizations (e.g. `-ctk q4_0`), they can substantially degrade the model's tool calling performance.
330+
328331
Test in CLI (or with any library / software that can use OpenAI-compatible API backends):
329332

330333
```bash
@@ -370,14 +373,14 @@ curl http://localhost:8080/v1/chat/completions -d '{
370373
"name":"get_current_weather",
371374
"description":"Get the current weather in a given location",
372375
"parameters":{
373-
"type":"object",
374-
"properties":{
375-
"location":{
376-
"type":"string",
377-
"description":"The city and country/state, e.g. `San Francisco, CA`, or `Paris, France`"
378-
}
379-
},
380-
"required":["location"]
376+
"type":"object",
377+
"properties":{
378+
"location":{
379+
"type":"string",
380+
"description":"The city and country/state, e.g. `San Francisco, CA`, or `Paris, France`"
381+
}
382+
},
383+
"required":["location"]
381384
}
382385
}
383386
}]

0 commit comments

Comments
 (0)