`tool-call`: fix Qwen 2.5 Coder support, add micro benchmarks, support trigger patterns for lazy grammars#12034

Merged

ochafik merged 48 commits intoggml-org:masterfrom

ochafik:tool-bench-prod

Mar 5, 2025

Commits on Feb 21, 2025

sampler: turn lazy grammar trigger words to regexes
ochafik
committed

Commits on Feb 22, 2025

add scripts/tool_bench.sh & .py
ochafik
committed
optionally allow any spaces in json schema grammars (useful for llama3 8b tool outputs)
ochafik
committed
constrain llama json output regardless of function name if matches at beginning
ochafik
committed
better error when wrong function called
ochafik
committed
improve error message in weather test
ochafik
committed
add more models to tool_bench.sh
ochafik
committed

Commits on Feb 23, 2025

benchmark other sizes of qwen 2.5 coder
ochafik
committed
rm duplicate in tool_bench.sh
ochafik
committed
add missing <variant> include
ochafik
committed
fix lints
ochafik
committed
improve "bad" qwen triggers
ochafik
committed
add cast to please some gccs
ochafik
committed
ditch server test request retry logic
ochafik
committed
fix flake8 lints
ochafik
committed
nits
ochafik
committed
remove any_spaces grammar option, allow extra line for airy llama json outputs
ochafik
committed
Update test_tool_call.py
ochafik
committed
test w/ beefier qwen 2.5 coder 3b
ochafik
committed
revert some test_hello_world diffs
ochafik
committed
diff
ochafik
committed
Update test_tool_call.py
ochafik
committed
add requirements for tool_bench
ochafik
committed
fix test_thoughts deepseek test expectation
ochafik
committed
Update README.md
ochafik
committed
update relaxed newline space rule in grammar tests
ochafik
committed

Commits on Feb 25, 2025

support add_generation_prompt query parameter (useful for /apply_template)
ochafik
committed
Merge remote-tracking branch 'origin/master' into tool-bench-prod
Olivier Chafik
committed
token cast tweak for gcc
Olivier Chafik
committed
fix warning on gcc13 w/ uninitialized variant
ochafik
committed
fix python lints
ochafik
committed
fix gcc13 warning
ochafik
committed
fix pyright lints in tool_bench.py
Olivier Chafik
committed
Merge remote-tracking branch 'origin/master' into tool-bench-prod
Olivier Chafik
committed

Commits on Feb 27, 2025

update readme w/ link to tool call
Olivier Chafik
committed
tool-bench: add --ctk, --ctv, --fa flags
Olivier Chafik
committed

Commits on Mar 4, 2025

Merge remote-tracking branch 'origin/master' into tool-bench-prod
Olivier Chafik
committed
common_grammar_trigger: always use string value (+ optional token)
ochafik
committed

Commits on Mar 5, 2025

add llama_grammar_trigger_pattern
Olivier Chafik
committed
add common_grammar_trigger.{to_json,from_json}
ochafik
committed
fix crashing typo
ochafik
committed
avoid returning optional from parse_json
ochafik
committed
disable slow hello Llama-3.1-8B (chopped unescaped string witin string problem)
ochafik
committed
fix nit eol at eof
ochafik
committed
Update src/llama-grammar.cpp

ochafik
and
ggerganov
authored
Merge remote-tracking branch 'origin/master' into tool-bench-prod
Olivier Chafik
committed
avoid ggml_assert in server for grammar triggers inconsistency
Olivier Chafik
committed
add comment on limits to common_grammar_trigger.to/from json specializations
Olivier Chafik
committed