LLamaCausalLM add support for tokenizer.json#9967

robbiemu wants to merge 7 commits intoggml-org:masterggml-org/llama.cpp:masterfrom

robbiemu:llamacasuallm-sp-bperobbiemu/llama.cpp:llamacasuallm-sp-bpeCopy head branch name to clipboard