Vulkan: GGML_ASSERT failed on Kimi-Linear-48B with large context - maxComputeWorkGroupCount exceeded

[iz0eyj]

Name and Version

federico@Sogliola:~$ llama-server --version
load_backend: loaded RPC backend from /home/federico/.local/share/llamacpp/libggml-rpc.so
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon Graphics (RADV GFX1150) (radv) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
load_backend: loaded Vulkan backend from /home/federico/.local/share/llamacpp/libggml-vulkan.so
load_backend: loaded CPU backend from /home/federico/.local/share/llamacpp/libggml-cpu-zen4.so
version: 7966 (8872ad2)
built with GNU 11.4.0 for Linux x86_64

Operating systems

Linux

GGML backends

Vulkan

Hardware

AMD Ryzen AI 9 HX370 (Strix Halo) with integrated Radeon 890M GPU

Models

https://huggingface.co/bartowski/Kimi-Linear-48B-Instruct-GGUF (model-00001-of-00002.gguf, Q8_0 quantization)

Problem description & steps to reproduce

Problem Description

When running Kimi-Linear-48B-Instruct Q8_0 with llama-server and Vulkan backend, the server crashes with a GGML_ASSERT failure during prompt processing. The assertion fails because the computed workgroup dimensions exceed the GPU's maxComputeWorkGroupCount limits.

Steps to Reproduce

1. Start llama-server with these parameters:

llama-server -c 65536 --context-shift -b 8192 -ub 2048 -fa on --no-mmap --jinja --host 0.0.0.0 --port 1234 -m /path/to/Kimi-Linear-48B-Instruct-Q8_0.gguf

2. Load a moderately sized text file (~38KB, approximately 13609 tokens)

3. Server crashes during prompt processing after processing 8192 tokens

Error Output

slot update_slots: id  3 | task 0 | prompt processing progress, n_tokens = 8192, batch.n_tokens = 8192, progress = 0.601955
/home/runner/work/llama.cpp/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:6225: GGML_ASSERT(wg0 <= ctx->device->properties.limits.maxComputeWorkGroupCount[0] && wg1 <= ctx->device->properties.limits.maxComputeWorkGroupCount[1] && wg2 <= ctx->device->properties.limits.maxComputeWorkGroupCount[2]) failed

Additional Context

• Same test with Qwen3-VL-30B-8bit works perfectly with identical parameters
• Issue appears specific to Kimi-Linear-48B model with large batch sizes
• Crash consistently happens at the same point during prompt processing
• Model works fine with small inputs (chat mode), only fails with larger context

First Bad Commit

No response

Relevant log output

slot update_slots: id  3 | task 0 | prompt processing progress, n_tokens = 8192, batch.n_tokens = 8192, progress = 0.601955
/home/runner/work/llama.cpp/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:6225: GGML_ASSERT(wg0 <= ctx->device->properties.limits.maxComputeWorkGroupCount[0] && wg1 <= ctx->device->properties.limits.maxComputeWorkGroupCount[1] && wg2 <= ctx->device->properties.limits.maxComputeWorkGroupCount[2]) failed

/home/runner/work/llama.cpp/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:6225: GGML_ASSERT(wg0 <= ctx->device->properties.limits.maxComputeWorkGroupCount[0] && wg1 <= ctx->device->properties.limits.maxComputeWorkGroupCount[1] && wg2 <= ctx->device->properties.limits.maxComputeWorkGroupCount[2]) failed

[iz0eyj]

Workaround Found

Reducing the batch size resolves the issue:

Working parameters:

llama-server -c 65536 --context-shift -b 4096 -ub 1024 -fa on --no-mmap --jinja --host 0.0.0.0 --port 1234 -m /path/to/Kimi-Linear-48B-Instruct-Q8_0.gguf

With -b 4096 and -ub 1024, the same 38KB file (~13609 tokens) that previously crashed now processes successfully without hitting the maxComputeWorkGroupCount limit.

This confirms the issue is specifically related to large batch sizes with the Vulkan backend when processing this model.

[vvverily]

I also noticed a large buffer size when i set -ub to 4096 on CUDA:
2026-02-09T17:01:03 [info] ggml_backend_cuda_buffer_type_alloc_buffer: allocating 9201.03 MiB on device 0: cudaMalloc failed: out of memory

[jeffbolznv]

I was able to reproduce this with the command line llama-bench.exe -m C:\models\moonshotai_Kimi-Linear-48B-A3B-Instruct-Q8_0-00001-of-00002.gguf -p 8192 -ub 2048 -fa 1. It's coming from ggml_vk_mul_mat_vec_q_f16, the y coordinate is 64k (ne12 == 64, ne13 == 1024). I'll look into this soon.

[iz0eyj]

@jeffbolznv Thank you for quickly reproducing and identifying the issue!

Once the workaround is in place, the model performs exceptionally well on long contexts. With the reduced batch parameters (-b 4096 -ub 1024), I'm getting:

• 63,529 tokens in context
• 12.3 tokens/sec generation speed
• Stable performance with no degradation

The Kimi-Linear architecture really shines with extended contexts on this hardware.

Question: Are there any plans to optimize Vulkan backend specifically for RDNA 3.5 (Radeon 890M/Strix Halo)? This APU is becoming popular for local inference, and any RDNA 3.5-specific optimizations would be valuable for the community.