UPSTREAM PR #16936: Fix segfault in moe-expert-reduce test in support mode and coverage

[DajanaV]

Mirrored from ggml-org/llama.cpp#16936

This PR fixes a segmentation fault that occurs while running the test-backend-ops tool in support mode or with --show-coverage flag. This will also allow docs/ops.md to be updated for tracking ggml-org/llama.cpp#14909 as it needs the results from support mode.

Root Cause

Testing does not initialize gf (ggml_cgraph), it calls build_graph method for each test case. The test_moe_expert_reduce test case calls ggml_build_forward_expand(gf, ...) inside its build_graph method but gf is a nullptr in this flow which causes a seg fault.

Solution

Wrap the ggml_build_forward_expand call in a gf null check.

[loci-agentic-ai]

Access the complete analysis in the LOCI Dashboard

LLaMA.cpp Performance Analysis Summary

Critical Function Performance Status

Core Inference Functions

All critical inference functions show no measurable performance changes:

• llama_decode: 49,003,504 ns (no change) - Primary token processing function
• llama_encode: 12,329,123 ns (no change) - Encoder model processing
• llama_tokenize: 834,823 ns (no change) - Text-to-token conversion

Supporting Functions

• llama_model_quantize: 6,891,713 ns (no change) - Model compression
• llama_batch_init: 257 ns (no change) - Batch initialization
• llama_memory_clear: 49 ns (no change) - Memory management

Only Detected Change

• std::make_pair (vocabulary processing): +0.13 ns (+0.059% response time degradation)

Key Performance Indicator Impact Analysis

1. Tokens Per Second: No Impact

• Status: No changes detected in inference-critical functions
• Reference Baseline: 7% tokens/sec reduction when llama_decode increases by 2ms
• Current Impact: llama_decode shows 0 ns change (49,003,504 ns vs 49,003,848 ns baseline)
• Affected Functions: None - all tokenization/inference functions unchanged

2. Power Consumption: Negligible Impact

• libllama.so: -0.0003% change (-0.93 nJ reduction)
• Other binaries: No measurable changes
  • libggml-base.so: 0.0% change
  • libggml-cpu.so: 0.0% change
  • libggml.so: 0.0% change
• Root Cause: Minimal std::make_pair optimization in vocabulary processing

3. Quantization Efficiency: No Impact

• llama_model_quantize: No performance changes detected
• Quantization pipeline: All related functions show identical performance
• Status: Quantization workflows maintain baseline efficiency

4. Memory Usage: No Impact

• llama_memory_clear: No changes (49 ns)
• Memory management functions: All show identical performance
• KV cache operations: No detected changes in memory allocation patterns

5. Batch Processing: No Impact

• llama_batch_init: No changes (257 ns)
• Batch processing pipeline: All functions maintain baseline performance
• Parallel processing efficiency: No degradation detected

Root Cause Analysis

Performance Change Source

The minimal degradation in std::make_pair appears to stem from:

• Build environment variations: Compiler optimization differences
• Template instantiation: Subtle changes in STL code generation
• Branch prediction: Minor instruction ordering changes

Code Modification Status

• All critical functions: is_modified: false
• No functional changes: Performance variations are build artifacts, not code changes

Action Items

Build System Optimization

1. Compiler consistency: Verify identical optimization flags between builds
2. Link-time optimization: Enable LTO to reduce PLT overhead in STL functions
3. Profile-guided optimization: Use PGO for template-heavy vocabulary processing

Performance Monitoring

1. Baseline validation: Confirm build environment consistency
2. Template optimization: Consider explicit specialization for common std::pair types
3. Instruction cache: Optimize function placement for better cache utilization

Conclusion

The analysis reveals no functional performance regressions in critical LLaMA.cpp inference pathways. All core functions (llama_decode, llama_encode, llama_tokenize) maintain identical performance characteristics. The minimal std::make_pair degradation represents a build artifact rather than a functional regression, with negligible impact on overall inference performance.