Implement SparseK Attention mechanism — new GGML operator with CPU backend (GPU planned next)

[yael-works]

New Attention Mechanism: SparseK Attention (CPU Backend)

This PR introduces a new attention mechanism called SparseK Attention, implemented from scratch as a new operator within the GGML framework, currently with CPU backend support.

---

Overview

SparseK Attention is a selective and efficient attention mechanism inspired by Flash Attention, but introduces additional sparsity through:

• Top-K filtering – keeps only the strongest attention weights.
• Local windowing – limits attention to a configurable local context.
• Global stride – adds periodic global connections between tokens.

---

Implementation Details

• Added new operator: GGML_OP_SPARSEK_ATTN defined in ggml.h and ggml.c.
• Implemented construction function ggml_sparsek_attn() that creates a computation node with parameters (k_top, win_local, stride_global).
• Added full CPU backend implementation in:
  • ggml-cpu/ops.h
  • ggml-cpu/ops.cpp
  • ggml-cpu.c

The CPU version includes:

• Scaled dot-product computation QKᵀ / √d
• Dynamic Top-K filtering
• Softmax normalization
• Multiplication with V

---

Next Steps

Our next goal is to extend SparseK Attention to the SYCL (GPU) backend in order to:

• Measure and compare performance between CPU and GPU implementations.
• Optimize kernel execution for sparse attention patterns.
• Validate correctness and scaling on Intel GPUs.

We are submitting this initial CPU implementation first to ensure review, integration, and baseline correctness before introducing GPU acceleration.

---

Co-Authors

Co-authored-by: Yael Shuker ([email protected])
Co-authored-by: Gitty Burstein ([email protected])

[GittyBurstein]

Hi @CISC and @NeoZhangJianyu,

We’d appreciate it if you could review our PR implementing the new SPARSEK Attention operator.
We ran internal validation tests we created ourselves, and all passed successfully.

This contribution was developed jointly by both of us (@yael-works and @GittyBurstein ).
Please make sure the PR reflects both contributors — if needed, we can adjust the commit authors accordingly.

Thanks in advance for your time and feedback!

[CISC]

We are talking about this SparseK, right?

[yael-works]

yes! @CISC

[CISC]

You need to rebase to fix Server CI failures, also please fix whitespaces:
https://github.com/ggml-org/llama.cpp/actions/runs/18935125175/job/54060021809