Discussions

• 2 You must be logged in to vote

  💬

  GPT-OSS performance

  ikawrakow started Sep 4, 2025 in General

  

  

  0
• 3 You must be logged in to vote

  🙏

  Tool Calling in GLM-4.5 - Anyone get it to work?

  jordandevai asked Sep 2, 2025 in Q&A · Unanswered

  

  

  

  

  10
• 1 You must be logged in to vote

  💡

  ~0.5% performance boost with mitigations=off?

  magikRUKKOLA started Aug 30, 2025 in Ideas

  

  

  1
• 1 You must be logged in to vote

  🙌

  Perplexity vs Size Graphs for the recent quants (Deepseek-R1, Kimi-K2, Chimera etc.)

  magikRUKKOLA started Aug 21, 2025 in Show and tell

  

  

  

  

  8
• 1 You must be logged in to vote

  🙏

  Help w/ Faster prefill on CPU-MoE?

  QuantumPlayDev asked Aug 22, 2025 in Q&A · Unanswered

  

  

  

  

  14
• 1 You must be logged in to vote

  🙏

  ggml_backend_metal_set_n_cb: n_cb = 24, using n_cb > 2 is not recommended

  CantHackTheHack asked Aug 25, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙌

  DeepSeek-R1-0528 ik quants!

  ubergarm started May 30, 2025 in Show and tell

  

  

  

  

  286
• 1 You must be logged in to vote

  💡

  Add --no-webui to llama-server

  yellowbadbeast1 started Aug 16, 2025 in Ideas

  

  

  0
• 6 You must be logged in to vote

  💬

  GPT-OSS?

  ikawrakow started Aug 7, 2025 in General

  

  

  

  

  3
• 1 You must be logged in to vote

  💬

  Options for supporting multiple users wrt. slots, KV-cache, --prompt-cache, --slot-save

  usrlocalben started Aug 8, 2025 in General

  

  

  3
• 5 You must be logged in to vote

  💬

  AMD GPU Vulkan & ROCm/HIP Discussion

  ubergarm started Jun 28, 2025 in General

  

  

  

  

  45
• 4 You must be logged in to vote

  🙏

  Help about optimizing inference speed on QWEN3-30B-2507 on ryzen 7 8745HS

  delta-whiplash asked Aug 1, 2025 in Q&A · Unanswered

  

  

  

  

  9
• 1 You must be logged in to vote

  💬

  NUMA and CPU selection

  joshuakoh1 started Jul 25, 2025 in General

  

  

  

  

  18
• 1 You must be logged in to vote

  🙏

  Best settings for Maverick - Dual CPU Xeon 8480+ - RTX 3090

  justinjja asked May 7, 2025 in Q&A · Unanswered

  

  

  

  

  3
• 2 You must be logged in to vote

  💬

  Intel Auto-round quants - how do they compare?

  Thireus started Jul 27, 2025 in General

  

  

  

  

  32
• 1 You must be logged in to vote

  💬

  ik_llama.cpp for Armv8.0

  NotAHero04 started Jun 25, 2025 in General

  

  

  

  

  6
• 1 You must be logged in to vote

  💡

  ik_llama.cpp memory managment improvements

  xkmire started Jul 28, 2025 in Ideas

  

  

  2
• 1 You must be logged in to vote

  🙏

  Is IQ4_KSS superior to IQ4_XS?

  tarruda asked Jul 27, 2025 in Q&A · Answered

  

  

  

  

  4
• 13 You must be logged in to vote

  🗳️

  How important is Vulkan back-end development?

  ikawrakow started Jul 6, 2025 in Polls

  

  

  

  

  43
• 1 You must be logged in to vote

  🙏

  Row-interleaved (R4) Quant Performance on Dual Xeon CPU-only Inference

  rkozuch asked Jul 20, 2025 in Q&A · Unanswered

  

  

  

  

  4
• 1 You must be logged in to vote

  💬

  gpu p2p utilization

  magikRUKKOLA started Jul 16, 2025 in General

  

  

  1
• 5 You must be logged in to vote

  💬

  Diverging from llama.cpp

  arnfaldur started Mar 14, 2025 in General

  

  

  

  

  7
• 8 You must be logged in to vote

  📣

  Unsuspended

  ikawrakow announced Jul 22, 2025 in Announcements

  

  

  

  

  12
• 2 You must be logged in to vote

  📣

  Unsuspended

  ikawrakow announced Jul 22, 2025 in Announcements · Closed

  

  

  0
• 2 You must be logged in to vote

  📣

  Unsuspended

  ikawrakow announced Jul 22, 2025 in Announcements · Closed

  

  

  0