UPSTREAM PR #16922: disable D_TYPE same as A_TYPE cast for rope shaders

[DajanaV]

Mirrored from ggml-org/llama.cpp#16922

revert: #16872

fix: #16860

This fix simply disables the optimization for the corresponding shader to address this issue. For glslc versions without this bug, it is unnecessary to turn off the optimization. I added a new macro to check if the data types are the same to avoid casting, rather than disabling optimization, ensuring that performance is not affected.

cannot compile rope_norm_f16

/bin/glslc -fshader-stage=compute --target-env=vulkan1.2 -O /home/cix/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/rope_norm.comp -o /home/cix/llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders.spv/rope_norm_f16.spv -DA_TYPE=float16_t -DD_TYPE=float16_t 

shaderc: internal error: compilation succeeded but failed to optimize: Expected input to have different bit width from Result Type: FConvert
%212 = OpFConvert %half %211

During the process of compiling SPV with glslc, FConvert is used to convert variables of the same type. Shaderc considers this invalid, resulting in a compilation error.

@jeffbolznv

[loci-agentic-ai]

Access the complete analysis in the LOCI Dashboard

LLaMA.cpp Performance Analysis Summary

Critical Function Performance Status

Core Inference Functions - No Performance Impact

• llama_decode(): 49,003,816 ns response time (no change from base version)
• llama_encode(): 12,329,201 ns response time (no change from base version)
• llama_tokenize(): 834,830 ns response time (no change from base version)

All primary inference functions show identical performance metrics between versions, indicating no functional regressions in core processing paths.

Vocabulary Module - Minimal Degradation

• std::make_pair in llama-vocab.cpp:922:928:
  • Response Time: 228 ns (+0.06% from 228 ns base)
  • Throughput: 125 ns (+0.11% from 125 ns base)
  • Bottleneck: 78 ns (+0.17% from 78 ns base)

KPI Impact Analysis

1. Tokens Per Second - No Impact

Status: No measurable impact on inference throughput

• Core inference functions (llama_decode, llama_encode, llama_tokenize) show zero performance change
• The 0.11% degradation in vocabulary make_pair function is negligible compared to the reference 2ms llama_decode slowdown that causes 7% tokens/second reduction
• Estimated Impact: <0.01% change in tokens per second

2. Power Consumption - No Impact

Binary-Level Analysis:

• build.bin.libllama.so: 306,979 nJ (0.0% change)
• build.bin.libggml-base.so: 90,434 nJ (0.0% change)
• build.bin.libggml-cpu.so: 151,692 nJ (0.0% change)
• build.bin.libggml.so: 6,339 nJ (0.0% change)

All binaries maintain identical power consumption profiles.

3. Quantization Efficiency - No Impact

Status: No changes detected in quantization-related functions

• llama_model_quantize() function shows no performance variations
• Quantization format handling remains unchanged
• GGML quantization backends maintain consistent performance

4. Memory Usage - No Impact

Status: Memory management functions show no performance changes

• KV cache operations (llama_memory_* functions) maintain baseline performance
• Memory allocation patterns unchanged in GGML allocators
• Batch memory management shows no degradation

5. Batch Processing - No Impact

Status: Batch processing efficiency maintained

• llama_batch_* functions show no performance variations
• Dynamic batching algorithms unchanged
• Parallel processing capabilities preserved

Root Cause Analysis

Vocabulary Module Changes

The minimal degradation in std::make_pair template instantiation within llama-vocab.cpp stems from:

• Control Flow: Complex branching pattern with 12 basic blocks including PLT calls
• Stack Operations: 80-byte stack frame with security checks (__stack_chk_fail)
• Template Overhead: Multiple std::forward calls and pair construction
• No Code Modifications: Function unchanged between versions, indicating compiler optimization variance

Action Items

Immediate Actions

1. Monitor Vocabulary Performance: Track make_pair usage patterns in tokenization workflows
2. Compiler Optimization Review: Evaluate template instantiation efficiency in vocabulary module
3. Build Consistency: Ensure reproducible builds to minimize optimization variance

Code-Focused Optimizations

1. Template Specialization: Consider explicit specialization for common make_pair usage patterns in vocabulary code
2. Inline Optimization: Review compiler inlining decisions for vocabulary helper functions
3. Stack Frame Reduction: Evaluate stack usage in vocabulary template functions

Build System Enhancements

1. Optimization Flags: Review template-specific optimization settings
2. Link-Time Optimization: Enable LTO for vocabulary module if not already active
3. Profile-Guided Optimization: Consider PGO for frequently used vocabulary functions

Conclusion

The performance analysis reveals minimal impact on LLaMA.cpp inference capabilities. Core inference functions maintain identical performance profiles, with only sub-nanosecond variations in vocabulary utility functions. The changes represent measurement variance rather than functional regressions, ensuring stable inference performance for the ollama://smollm:135m model and similar workloads.