UPSTREAM PR #17514: vulkan: use a fixed 1KB buffer for the add_rms_fusion opt

[loci-dev]

Mirrored from ggml-org/llama.cpp#17514

See ggml-org/llama.cpp#16900 (comment). I did verify that this resolves the failure.

I see qwen3moe and deepseek2 using in the range 400-800 bytes, so 1KB should be fine for now.

[loci-agentic-ai]

Explore the complete analysis inside the Version Insights

[loci-agentic-ai]

Explore the complete analysis inside the Version Insights

Performance Analysis Summary: PR #332

Overview

This PR implements a fixed 1KB buffer allocation for Vulkan RMS normalization operations, replacing dynamic sizing logic. The change modifies ggml-vulkan.cpp with 2 additions and 2 deletions across two functions: ggml_vk_init and ggml_backend_vk_graph_compute.

Performance Impact

Analysis shows 0.0% change across all performance metrics. The modified functions are part of the Vulkan backend initialization and graph computation paths, not the core inference tokenization pipeline. Functions responsible for tokens per second (llama_decode, llama_encode, llama_tokenize) remain unmodified with no measurable response time or throughput changes.

The removal of the std::max comparison in ggml_backend_vk_graph_compute reduces instruction count by approximately 3-5 operations per graph execution, but this translates to sub-nanosecond improvements that fall below measurement thresholds.

Power consumption analysis confirms 0.0% change across all binaries, including build.bin.libggml-cpu.so where the modifications occur.

Inference Impact

No impact on tokens per second. The change addresses buffer allocation for RMS partial results during Vulkan operations, which occurs outside the critical tokenization and decoding paths. Models using Vulkan acceleration will experience identical inference performance with improved reliability through deterministic buffer sizing.