sync : ggml#3071

Merged

ggerganov merged 80 commits intomasterfrom

sync-ggml-25-04-24

Apr 24, 2025

Commits on Apr 24, 2025

CUDA: don't convert BF16 weights to FP32 (ggml/1174)

CISC
authored and
ggerganov
committed
ggml : simplify Arm fp16 CPU logic (ggml/1177)
ggerganov
committed
llama : add option to override model tensor buffers (llama/11397)

slaren
authored and
ggerganov
committed
Vulkan: Fix mmq int dot float cache size (llama/12722)

0cc4m
authored and
ggerganov
committed
vulkan: Implement grouped query attention in the coopmat2 FA shader (llama/12559)

jeffbolznv
authored and
ggerganov
committed
cmake: remove caching from vulkan coopmat checks (llama/12719)

bandoti
authored and
ggerganov
committed
vulkan: Implement split_k for coopmat2 flash attention. (llama/12627)

jeffbolznv
authored and
ggerganov
committed
opencl: use `max_alloc_size` in backend ctx instead of querying again (llama/12705)

lhez
authored and
ggerganov
committed
CANN: Fix failed test cases (llama/12708)

hipudding
authored and
ggerganov
committed
Simplify and improve CUDA graphs through use of indirect copy pointers (llama/9017)

authored and
ggerganov
committed
CANN: Support operator SIN COS ARGMAX (llama/12709)

authored and
ggerganov
committed
fix MUSA compiler warning (llama/12704)

A3shTnT
authored and
ggerganov
committed
vulkan: Fix missing cmake logic for dot product extension (llama/12721)

jeffbolznv
authored and
ggerganov
committed
CUDA: Prefer vector flash decoding kernel for Gemma models (llama/12738)

authored and
ggerganov
committed
vulkan: set cmake minimum and project name in vulkan-shaders (llama/12744)

jeffbolznv
authored and
ggerganov
committed
vulkan: Hybrid waitForFences/getFenceStatus to reduce fence latency (llama/12630)

jeffbolznv
authored and
ggerganov
committed
cmake: fix ggml-shaders-gen compiler paths containing spaces (llama/12747)

hydroo
authored and
ggerganov
committed
sycl: allow ggml-sycl configuration and compilation using Visual Studio project/solution (llama/12625)

s-Nick
authored and
ggerganov
committed
Vulkan: Tune Vulkan mmq int dot shader for performance (llama/12767)

0cc4m
authored and
ggerganov
committed
vulkan: Use unclamped loads for flash attention mask (llama/12720)

jeffbolznv
authored and
ggerganov
committed
vulkan: fix NaN issue in flash attention shader (llama/12776)

jeffbolznv
authored and
ggerganov
committed
musa: fix compilation warnings in mp_22/31 (llama/12780)

yeahdongcn
authored and
ggerganov
committed
CANN: Refactor to reduce duplicate code (llama/12731)

hipudding
authored and
ggerganov
committed
CANN: fix typo in ggml-cann (llama/12733)

zhouwg
authored and
ggerganov
committed
sycl: remove redundant memcopy in function ggml_backend_sycl_buffer_set_tensor (llama/12734)

zhouwg
authored and
ggerganov
committed
cuda : fix HIP and MUSA BF16 (llama/0)
ggerganov
committed
opencl: better identify Adreno GPU (llama/12760)

lhez
authored and
ggerganov
committed
Revert "sycl:remove redundant memcopy in function ggml_backend_sycl_buffer_set_tensor" (llama/12812)

NeoZhangJianyu
authored and
ggerganov
committed
ggml : add more generic custom op, remove deprecated custom ops (ggml/1183)

slaren
authored and
ggerganov
committed
ggml : add bilinear upscale support (ggml/1185)

slaren
authored and
ggerganov
committed
ggml: don't include arm_neon.h when using CUDA 12 with ARM Neon (ggml/1187)

cmdr2
authored and
ggerganov
committed
llama : fix FA when KV cache is not used (i.e. embeddings) (llama/12825)
ggerganov
committed
cuda : add f32 to bf16 copy op (llama/12806)

CISC
authored and
ggerganov
committed
vulkan: Use fp16 for the flash attention P*V multiplication (llama/12783)

jeffbolznv
authored and
ggerganov
committed
vulkan: In coopmat2 mmq, load q4_k/q5_k scales through shared memory (llama/12833)

jeffbolznv
authored and
ggerganov
committed
CANN: Support Opt CONV_TRANSPOSE_1D and ELU (llama/12786)

noemotiovon
authored and
ggerganov
committed
ggml-impl.h: fix build on POWER9 (llama/12855)

pkubaj
authored and
ggerganov
committed
ggml-cpu-impl.h: do not redefine bool on POWER9 (llama/12856)

pkubaj
authored and
ggerganov
committed
Fixes #12823 (llama/12830)

mehendarkarprajwal
authored and
ggerganov
committed
CANN: Support more ops (llama/12841)

noemotiovon
authored and
ggerganov
committed
cpu: fix cpu backend's supports-op for GET_ROWS_BACK. fixes a fatal when running test-backend-ops with only the CPU backend (ggml/1190)

cmdr2
authored and
ggerganov
committed
ggml: fix compilation error s390x (llama/12848)

authored and
ggerganov
committed
SYCL: Add fp16 type support to unary op kernels (llama/12788)

qnixsynapse
authored and
ggerganov
committed
sycl: Support sycl_ext_oneapi_limited_graph (llama/12873)

EwanC
authored and
ggerganov
committed
vulkan: use aligned loads for flash attention mask (llama/12853)

jeffbolznv
authored and
ggerganov
committed
ggml: disable CUDA graphs for unsupported DUP and CONT node types (llama/12891)

agray3
authored and
ggerganov
committed
ggml: use _mm[512/256]_dpbusd[_avx]_epi32 to directly accumulate into the result register (llama/12773)

SongXiaoXi
authored and
ggerganov
committed
ggml : Depthwise 2D convolution (ggml/1152)

Acly
authored and
ggerganov
committed
rpc : use ggml_context_ptr (llama/12938)

rgerganov
authored and
ggerganov
committed
SYCL: Fix im2col (llama/12910)

qnixsynapse
authored and
ggerganov
committed
CANN: Optimize CANN buffer pool memory management (llama/12875)

bachelor-dou
authored and
ggerganov
committed
CANN: Opt ROPE optimization (llama/12865)

noemotiovon
authored and
ggerganov
committed
ggml : Add AVX512 implementation of GEMM - Q4_Kx8 (llama/12829)

Srihari-mcw
authored and
ggerganov
committed
SYCL: Add ROPE vision kernel (llama/12887)

qnixsynapse
authored and
ggerganov
committed
CUDA/HIP: Share the same unified memory allocation logic. (llama/12934)

hjc4869
authored and
ggerganov
committed
CANN: Add x86 build ci (llama/12950)

hipudding
authored and
ggerganov
committed
metal : add FA-vec kernels for head size 96 (llama/12952)
ggerganov
committed
CANN: Add 310P operator support check (llama/12962)

noemotiovon
authored and
ggerganov
committed
vulkan: enable coopmat2 FA gqa and split_k optimizations more often (llama/12931)

jeffbolznv
authored and
ggerganov
committed
opencl: fix incorrect local_size index in profiling log (llama/12868)

kimminsu38oo
authored and
ggerganov
committed
CANN: Add support for async operator submission (llama/12864)

hipudding
authored and
ggerganov
committed
ggml: Re-enable CUDA graphs in presence of CONT and DUP nodes (llama/12970)

agray3
authored and
ggerganov
committed
graph : make FA compatible with MLA + add initial Metal kernels (llama/12953)
ggerganov
committed
rpc : add RPC_CMD_HELLO (llama/12955)

rgerganov
authored and
ggerganov
committed
SYCL: Refactor and enable FP16 in binary broadcast OPs (llama/12975)

qnixsynapse
authored and
ggerganov
committed
metal: add neg operator (llama/13029)

jmorganca
authored and
ggerganov
committed
vulkan: support noncontiguous rms_norm (llama/13031)

jeffbolznv
authored and
ggerganov
committed
SYCL: Add non-contiguous support in ROPE (llama/12993)

qnixsynapse
authored and
ggerganov
committed
ggml : add SSE 4.2 and x64 base variant for CPUs without AVX (llama/12871)

slaren
authored and
ggerganov
committed
CUDA: noncont MMVQ + batched bs1 MUL_MAT_ID (llama/13014)

JohannesGaessler
authored and
ggerganov
committed
vulkan: matmul gcn tuning (llama/13016)

authored and
ggerganov
committed
metal : fix floating-point range of attention scores in FA kernels (llama/13090)
ggerganov
committed
CUDA: use switch statements in constexpr functions (llama/13095)

JohannesGaessler
authored and
ggerganov
committed
ggml : fix trailing whitespaces (llama/0)
ggerganov
committed
opencl: split ggml-opencl.cl into multiple files and cleanup (llama/12886)

authored and
ggerganov
committed
sync : ggml
ggerganov
committed
opencl : remove obsolete files (skip) (ggml/1200)
ggerganov
committed
sync : ggml
ggerganov
committed
cuda : fix unused variable compile warning (#0)
ggerganov
committed
ruby : add cmake option (#0)
ggerganov
committed