Port universal assisted decoding to llama-server

[g2mt]

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High

See ggml-org/llama.cpp#12635 for equivalent PR in mainline. Related to #645

[ikawrakow]

It would be nice to have an example telling us how to test and use (which models to use, command line, etc.). For people like me who don't have the hardware to run giant models, it would be nice if there was an examples with main_draft models that can be loaded in 64 GB RAM + 16 GB VRAM.

[g2mt]

It should work the same as passing a compatible draft model. I also added the --spec-replace argument for translating the template tags.

Here are my prompt/generation speeds for a simple repeat prompt. I'm using Devstral Small with Qwen2 coder 0.5 as the draft model.

without speculative decoding:

"ik_llama.cpp/build/bin/Release/llama-server.exe"  "-m" "Devstral-Small-2507-UD-Q4_K_XL.gguf" "-c" "32768" "--run-time-repack" "-t" "12"

with it enabled:

"ik_llama.cpp/build/bin/Release/llama-server.exe"  "--port" "10006" "-m" "Devstral-Small-2507-UD-Q4_K_XL.gguf" "-c" "32768" "--run-time-repack" "-t" "12" "-md" "Qwen2.5-Coder-0.5B-Instruct-Q8_0.gguf" "--spec-replace" "[INST]" "<|im_begin|>user\n" "--spec-replace" "[/INST]" "<|im_end|><|im_begin|>assistant" "--spec-replace" "</s>" "<|im_begin|>user\n"