vocab : minor tokenization optimizations#11160

ggerganov merged 1 commit intogg/llama-refactor-7ggml-org/llama.cpp:gg/llama-refactor-7from

gg/llama-tokenizer-optggml-org/llama.cpp:gg/llama-tokenizer-optCopy head branch name to clipboard