`grammars`: cache decoded token codepoints for faster sampling#6811

Closed

ochafik wants to merge 19 commits intoggml-org:masterfrom

ochafik:grammar-fast

Commits on Apr 21, 2024

grammars: early exit when no next_candidates to reject
ochafik
committed
grammars: cache decoded tokens
ochafik
committed
grammars: faster llama_grammar_copy
ochafik
committed
Merge remote-tracking branch 'origin/master' into grammar-fast
ochafik
committed
grammars: fix bad merge
ochafik
committed

Commits on Apr 24, 2024

grammars: keep llama_grammar_copy non-quadratic optim for later
Olivier Chafik
committed

Commits on Apr 27, 2024

Merge remote-tracking branch 'origin/master' into grammar-fast
ochafik
committed

Commits on Apr 28, 2024

grammars: move token caches to llama_context
ochafik
committed
grammars: cache codepoints in llama_new_context_with_model
ochafik
committed
grammar: nit (layout)
ochafik
committed
grammars: nits (revert const grammar sig, fix comment)
ochafik
committed

Commits on Apr 30, 2024

Merge remote-tracking branch 'origin/master' into grammar-fast
ochafik
committed

Commits on May 1, 2024

Update llama.cpp

ochafik
and
HanClinto
authored

Commits on May 18, 2024

Commits on May 21, 2024

Merge remote-tracking branch 'origin/master' into grammar-fast
ochafik
committed

Commits on Jun 4, 2024

Merge remote-tracking branch 'origin/master' into grammar-fast
ochafik
committed

Commits on Jun 10, 2024

Merge remote-tracking branch 'origin/master' into grammar-fast
ochafik
committed