Sync master with upstream release b5857#162

Merged

Minh141120 merged 188 commits intodevfrom

update-dev-from-master-2025-07-10-00-09

Jul 10, 2025

Commits on Jul 10, 2025

add geglu activation function (#14074)

authored and
qnixsynapse
committed
sycl: Add reorder to Q6_K mmvq implementation (#13885)

s-Nick
authored and
qnixsynapse
committed
webui: fix sidebar being covered by main content (#14082)

yeahdongcn
authored and
qnixsynapse
committed
CANN: Simplify the environment variable setting(#13104)

bachelor-dou
authored and
qnixsynapse
committed
graph : fix geglu (#14077)

ggerganov
authored and
qnixsynapse
committed
ggml-cpu : split arch-specific implementations (#13892)

authored and
qnixsynapse
committed
llama : allow building all tests on windows when not using shared libs (#13980)

authored and
qnixsynapse
committed
sync : ggml

ggerganov
authored and
qnixsynapse
committed
Vulkan: Don't default to CPU device (like llvmpipe), even if no other device is available, to allow fallback to CPU backend (#14099)

0cc4m
authored and
qnixsynapse
committed
ggml : fix weak alias win32 (whisper/0)

ggerganov
authored and
qnixsynapse
committed
sync : ggml

ggerganov
authored and
qnixsynapse
committed
vulkan: force device 0 in CI (#14106)

jeffbolznv
authored and
qnixsynapse
committed
llama : support GEGLU for jina-bert-v2 (#14090)

CISC
authored and
qnixsynapse
committed
convert : fix duplicate key DeepSeek-R1 conversion error (#14103)

CISC
authored and
qnixsynapse
committed
kv-cache : avoid modifying recurrent cells when setting inputs (#13834)

compilade
authored and
qnixsynapse
committed
opencl: add `mul_mv_id_q4_0_f32_8x_flat` (#14003)

lhez
authored and
qnixsynapse
committed
vulkan: Track descriptor pools/sets per-context (#14109)

jeffbolznv
authored and
qnixsynapse
committed
kv-cache : add LLAMA_KV_CACHE_DEBUG environment variable (#14121)

ggerganov
authored and
qnixsynapse
committed
kv-cache : relax SWA masking condition (#14119)

ggerganov
authored and
qnixsynapse
committed
webui: Wrap long numbers instead of infinite horizontal scroll (#14062)

am17an
authored and
qnixsynapse
committed
vulkan: Better thread-safety for command pools/buffers (#14116)

jeffbolznv
authored and
qnixsynapse
committed
tests : add test-tokenizers-repo (#14017)

CISC
authored and
qnixsynapse
committed
chore : clean up relative source dir paths (#14128)

CISC
authored and
qnixsynapse
committed
Implement GGML_CPU_ALL_VARIANTS for ARM (#14080)

ckastner
authored and
qnixsynapse
committed
kv-cache : fix split_equal handling in unified implementation (#14130)

ggerganov
authored and
qnixsynapse
committed
batch : remove logits_all flag (#14141)

ggerganov
authored and
qnixsynapse
committed
context : simplify output counting logic during decode (#14142)

ggerganov
authored and
qnixsynapse
committed
cmake : Improve build-info.cpp generation (#14156)

ckastner
authored and
qnixsynapse
committed
pooling : make cls_b and cls_out_b optional (#14165)

authored and
qnixsynapse
committed
cmake: Add ability to pass in LLAMA_BUILD_NUMBER/COMMIT (#14167)

ckastner
authored and
qnixsynapse
committed
batch : rework llama_batch_allocr (#14153)

ggerganov
authored and
qnixsynapse
committed
batch : add LLAMA_BATCH_DEBUG environment variable (#14172)

ggerganov
authored and
qnixsynapse
committed
Merge commit from fork

authored and
qnixsynapse
committed
vocab : fix build (#14175)

ggerganov
authored and
qnixsynapse
committed
batch : auto-gen positions + verify multi-sequence input (#14177)

ggerganov
authored and
qnixsynapse
committed
cparams : rename LLAMA_MAX_PARALLEL_SEQUENCES to LLAMA_MAX_SEQ (#14188)

ggerganov
authored and
qnixsynapse
committed
model : add dots.llm1 architecture support (#14044) (#14118)

Noeda
authored and
qnixsynapse
committed
kv-cache : fix use-after-move of defrag info (#14189)

ggerganov
authored and
qnixsynapse
committed
model : Add support for Arcee AI's upcoming AFM model (#14185)

authored and
qnixsynapse
committed
ggml-cpu : rework weak alias on apple targets (#14146)

xctan
authored and
qnixsynapse
committed
vulkan: mutex around vkQueueSubmit (#14127)

jeffbolznv
authored and
qnixsynapse
committed
convert : remove arcee change in convert_hf_to_gguf_update.py (#14207)

bartowski1182
authored and
qnixsynapse
committed
ggml: Add Android support for GGML_CPU_ALL_VARIANTS (#14206)

chaxu01
authored and
qnixsynapse
committed
llama : rework embeddings logic (#14208)

ggerganov
authored and
qnixsynapse
committed
model : add NeoBERT (#14164)

authored and
qnixsynapse
committed
cmake: clean up external project logic for vulkan-shaders-gen (#14179)

bandoti
authored and
qnixsynapse
committed
llama : add thread safety test (#14035)

authored and
qnixsynapse
committed
server : fix incorrect usage of llama_get_embeddings() (#14225)

ggerganov
authored and
qnixsynapse
committed
ggml-cpu : remove the weak alias trick (#14221)

xctan
authored and
qnixsynapse
committed
cmake: remove shader-gen step-targets from ggml-vulkan (#14226)

bandoti
authored and
qnixsynapse
committed
examples : include examples in msvc disable warn (ggml/1270)

danbev
authored and
qnixsynapse
committed
ggml : disable warnings for tests when using MSVC (ggml/1273)

danbev
authored and
qnixsynapse
committed
sync : ggml

ggerganov
authored and
qnixsynapse
committed
convert : fix null head_dim AutoConfig regression (#14248)

CISC
authored and
qnixsynapse
committed
ggml: Add Apple support for GGML_CPU_ALL_VARIANTS (#14258)

chaxu01
authored and
qnixsynapse
committed
docs: add s390x build documentation (#14264)

taronaeo
authored and
qnixsynapse
committed
metal : add mean kernel (#14267)

ggerganov
authored and
qnixsynapse
committed
memory : Hybrid recurrent cache (#13979)

authored and
qnixsynapse
committed
Vulkan: Set device max size for host memory to avoid OOM warning and fallback to CPU buffer (#14249)

0cc4m
authored and
qnixsynapse
committed
llamafile : support s390x SIMD instruction set (#14273)

taronaeo
authored and
qnixsynapse
committed
convert : fix remote option in Windows (#14100)

pqnet
authored and
qnixsynapse
committed
build : suppress gcc15 compile warnings (#14261)

fanyang89
authored and
qnixsynapse
committed
server : add server parameters for draft model cache type (#13782)

aa956
authored and
qnixsynapse
committed
ggml-cpu : remove unnecesary arm feature detection (#14281)

slaren
authored and
qnixsynapse
committed
CUDA: add conv_2d_dw (#14265)

am17an
authored and
qnixsynapse
committed
ubatch : new splitting logic (#14217)

ggerganov
authored and
qnixsynapse
committed
model : more uniform output id handling (#14275)

ggerganov
authored and
qnixsynapse
committed
ggml: Update KleidiAI to v1.9.0 (#14277)

chaxu01
authored and
qnixsynapse
committed
ggml : fix repack work size for mul_mat_id (#14292)

ggerganov
authored and
qnixsynapse
committed
cuda : synchronize graph capture and cublas handle destruction (#14288)

slaren
authored and
qnixsynapse
committed
llama : improve sep token handling (#14272)

CISC
authored and
qnixsynapse
committed
Implement GGML_CPU_ALL_VARIANTS for PowerPC (#14286)

authored and
qnixsynapse
committed
sycl: add usage of enqueue_functions extension (#14244)

s-Nick
authored and
qnixsynapse
committed
vocab : prevent tokenizer overflow (#14301)

retr0reg
authored and
qnixsynapse
committed
lint : remove trailing whitepace (#14304)

CISC
authored and
qnixsynapse
committed
CUDA: add conv_2d_transpose (#14287)

am17an
authored and
qnixsynapse
committed
Add `ggml_roll` (ggml/1274)

Acly
authored and
qnixsynapse
committed
sync : ggml

ggerganov
authored and
qnixsynapse
committed
convert : fix Llama 4 conversion (#14311)

danielhanchen
authored and
qnixsynapse
committed
memory : rename interface to llama_memory_context_i (#14296)

ggerganov
authored and
qnixsynapse
committed
metal : fix thread-safety (#14300)

ggerganov
authored and
qnixsynapse
committed
gguf-py : fix TemplateProcessing pair when bos/eos is missing (#14312)

CISC
authored and
qnixsynapse
committed
Add support for VK_EXT_debug_utils to add labels to Vulkan objects. (#13792)

mtavenrath
authored and
qnixsynapse
committed
gguf-py : fix Qwen3-Embedding eos token (#14314)

CISC
authored and
qnixsynapse
committed
CUDA: add mean operation (#14313)

am17an
authored and
qnixsynapse
committed
HIP: enable vec fattn on RDNA4 (#14323)

IMbackK
authored and
qnixsynapse
committed
examples : fix is_first logic for tokenization (#14329)

ggerganov
authored and
qnixsynapse
committed
run : avoid double tokenization (#14327)

retr0reg
authored and
qnixsynapse
committed
gguf-py : fix SpecialVocab parsing when post_processor is null (#14330)

CISC
authored and
qnixsynapse
committed
quantize : handle user-defined pruning of whole layers (blocks) (#13037)

EAddario
authored and
qnixsynapse
committed
vulkan: update windows SDK in CI (#14334)

jeffbolznv
authored and
qnixsynapse
committed
kv-cells : fix tracking of seq_pos (#14339)

ggerganov
authored and
qnixsynapse
committed
CUDA: mul_mat_v support for batch sizes > 1 (#14262)

JohannesGaessler
authored and
qnixsynapse
committed
ci: add workflow for relocatable cmake package (#14346)

bandoti
authored and
qnixsynapse
committed
CUDA/HIP: optimize mmv paths taken for HIP devices (#14324)

authored and
qnixsynapse
committed
cmake : use LLAMA_BUILD_NUMBER when defining LLAMA_INSTALL_VERSION (#14362)

mbaudier
authored and
qnixsynapse
committed
batch : fix check for empty sequences in memory (#14364)

ggerganov
authored and
qnixsynapse
committed
opencl: ref count `ggml_backend_opencl_context` and refactor profiling (#14254)

lhez
authored and
qnixsynapse
committed
ggml-cpu: enable IBM NNPA Vector Intrinsics (#14317)

authored and
qnixsynapse
committed
musa: enable fp16 mma (all) and cublas on qy2 (#13842)

authored and
qnixsynapse
committed
docs: update s390x documentation + add faq (#14389)

taronaeo
authored and
qnixsynapse
committed
metal : batch rows copy in a single threadgroup (#14384)

ggerganov
authored and
qnixsynapse
committed
metal : add special-case mat-vec mul for ne00 == 4 (#14385)

ggerganov
authored and
qnixsynapse
committed
llama : return mistral-v7-tekken as default template only (#14390)

CISC
authored and
qnixsynapse
committed
cmake: regen vulkan shaders when shaders-gen sources change (#14398)

bandoti
authored and
qnixsynapse
committed
model : gemma3n text-only (#14400)

ngxson
authored and
qnixsynapse
committed
convert : fix broken sentencepiece vocab (#14416)

CISC
authored and
qnixsynapse
committed
ggml : add ggml_set_rows (#14274)

authored and
qnixsynapse
committed
recurrent : call balloc split_reset() in init_batch() (#14414)

ggerganov
authored and
qnixsynapse
committed
graph : make llm_graph_context destructor virtual (#14410)

ggerganov
authored and
qnixsynapse
committed
vulkan: Fix GGML_VULKAN_SHADER_DEBUG_INFO (#14427)

jeffbolznv
authored and
qnixsynapse
committed
ci : fix windows build and release (#14431)

CISC
authored and
qnixsynapse
committed
fix async_mode bug (#14432)

bachelor-dou
authored and
qnixsynapse
committed
model : add support for ERNIE 4.5 0.3B model (#14408)

ownia
authored and
qnixsynapse
committed
vulkan: lock accesses of pinned_memory vector (#14333)

jeffbolznv
authored and
qnixsynapse
committed
vulkan: handle noncontig in the final case of ggml_vk_get_cpy_pipeline (#14378)

jeffbolznv
authored and
qnixsynapse
committed
CUDA: add bf16 and f32 support to cublas_mul_mat_batched (#14361)

am17an
authored and
qnixsynapse
committed
vulkan: Add fusion support for RMS_NORM+MUL (#14366)

authored and
qnixsynapse
committed
ggml : implement REGLU/GEGLU/SWIGLU ops (#14158)

committed
ggml : fix unmerged GGML_FPxx_TO_FPxx refactoring (#14443)

CISC
authored and
qnixsynapse
committed
SYCL: disable faulty fp16 exp kernel (#14395)
qnixsynapse
committed
server : fix appearance of the chats list context menu for Safari (#14322)

rntk
authored and
qnixsynapse
committed
server : support jinja extra template kwargs (Qwen3 enable_thinking feature), from command line and from client (#13196)

authored and
qnixsynapse
committed
scripts : make the shell scripts cross-platform (#14341)

vedranmiletic
authored and
qnixsynapse
committed
cmake : Remove redundant include path in CMakeLists.txt (#14452)

xiaobing318
authored and
qnixsynapse
committed
test-backend-ops : disable llama test (#14461)

slaren
authored and
qnixsynapse
committed
ggml-cpu: sycl: Re-enable exp f16 (#14462)

Rbiessy
authored and
qnixsynapse
committed
metal : disable fast-math for some cpy kernels (#14460)

ggerganov
authored and
qnixsynapse
committed
memory : correctly handle failure in apply() (#14438)

ggerganov
authored and
qnixsynapse
committed
Add Conv2d for CPU (#14388)

am17an
authored and
qnixsynapse
committed
opencl : add GEGLU, REGLU, SWIGLU (#14456)

lhez
authored and
qnixsynapse
committed
ggml-cpu : "align corners" for bilinear upscale/downscale (ggml/1285)

Acly
authored and
qnixsynapse
committed
sync : ggml

ggerganov
authored and
qnixsynapse
committed
ggml : remove trailing whitespace (#0)

ggerganov
authored and
qnixsynapse
committed
add GELU_ERF (#14455)

CISC
authored and
qnixsynapse
committed
vulkan: Split large mul_mat_id to fit in shared memory (#14451)

jeffbolznv
authored and
qnixsynapse
committed
ci : disable fast-math for Metal GHA CI (#14478)

ggerganov
authored and
qnixsynapse
committed
ggml : Callback before abort (#14481)

authored and
qnixsynapse
committed
github : add OpenCL backend to issue templates (#14492)

EZForever
authored and
qnixsynapse
committed
opencl : update upscale to support align corners (#14488)

lhez
authored and
qnixsynapse
committed
opencl : skip empty nodes on cgraph compute (#14491)

EZForever
authored and
qnixsynapse
committed
opencl : fix possible buffer overflow in dump_tensor (#14490)

jeffzhou2000
authored and
qnixsynapse
committed
ggml : support bcast ggml_soft_max_ext, ggml_flash_attn_ext (#14435)

ggerganov
authored and
qnixsynapse
committed
vulkan: support softmax/FA batch and broadcast (#14449)

jeffbolznv
authored and
qnixsynapse
committed
CUDA: add softmax broadcast (#14475)

am17an
authored and
qnixsynapse
committed
ggml : add version function to get lib version (ggml/1286)

authored and
qnixsynapse
committed
sync : ggml

ggerganov
authored and
qnixsynapse
committed
llama : initial Mamba-2 support (#9126)

compilade
authored and
qnixsynapse
committed
gguf-py : add support for chat template jinja files (#14508)

CISC
authored and
qnixsynapse
committed
CUDA: add dynamic shared mem to softmax, refactor general usage (#14497)

am17an
authored and
qnixsynapse
committed
ggml : remove kompute backend (#14501)

ggerganov
authored and
qnixsynapse
committed
ggml : fix FA mask dim 2 and 3 (#14505)

ggerganov
authored and
qnixsynapse
committed
kv-cache : use ggml_set_rows (#14285)

ggerganov
authored and
qnixsynapse
committed
ggml: backward pass for split swiglu (#14483)

JohannesGaessler
authored and
qnixsynapse
committed
vulkan: support mixed/deepseekR1 FA head sizes (#14509)

jeffbolznv
authored and
qnixsynapse
committed
opencl : broadcast for soft_max (#14510)

lhez
authored and
qnixsynapse
committed
ggml : implement GEGLU_ERF and GEGLU_QUICK ops (#14445)

CISC
authored and
qnixsynapse
committed
batch : add n_used count (#14512)

ggerganov
authored and
qnixsynapse
committed
graph : prepare for 4D mask (#14515)

ggerganov
authored and
qnixsynapse
committed
batch : add optional for sequential equal split (#14511)

ggerganov
authored and
qnixsynapse
committed
metal : disable fast math in all quantize kernels (#14528)

ggerganov
authored and
qnixsynapse
committed
test-backend-ops: add support for specifying output format (#14368)

authored and
qnixsynapse
committed
opencl: add GELU_ERF (#14476)

CISC
authored and
qnixsynapse
committed
vulkan: Handle updated FA dim2/3 definition (#14518)

jeffbolznv
authored and
qnixsynapse
committed
vulkan: fix rms_norm+mul fusion (#14545)

jeffbolznv
authored and
qnixsynapse
committed
vulkan: increase LOAD_VEC_A to 8 (IQ1/IQ2) or 4 (IQ3) (#14485)

authored and
qnixsynapse
committed
CUDA: add bf16 and i32 to getrows (#14529)

am17an
authored and
qnixsynapse
committed
llama : remove ggml_cont where possible (#14568)

CISC
authored and
qnixsynapse
committed
llama : fix incorrect minicpm3 v_states shape (#14571)

CISC
authored and
qnixsynapse
committed
musa: fix build warnings (unused variable) (#14561)

yeahdongcn
authored and
qnixsynapse
committed
CUDA: add bilinear interpolation for upscale (#14563)

am17an
authored and
qnixsynapse
committed
cuda : fix rope with partial rotation and non-cont src (#14580)

ggerganov
authored and
qnixsynapse
committed
vulkan: increase timeout for CI (#14574)

jeffbolznv
authored and
qnixsynapse
committed
model : add hunyuan moe (#14425)

authored and
qnixsynapse
committed
server: Add ability to mount server at prefix (#14544)

oluwandabira
authored and
qnixsynapse
committed
vulkan : fix rope with partial rotation and non-cont src (#14582)

jeffbolznv
authored and
qnixsynapse
committed
memory : fix broken batch splits for recurrent cache (#14575)

compilade
authored and
qnixsynapse
committed
model : add SmolLM3 (#14581)

authored and
qnixsynapse
committed
model : fix hunyuan moe chat template (#14584)

stevenkuang-tencent
authored and
qnixsynapse
committed
vulkan: optimize flash attention split_k_reduce (#14554)

jeffbolznv
authored and
qnixsynapse
committed
convert : fix smollm3 jinja template (#14586)

ngxson
authored and
qnixsynapse
committed
model : add support for Falcon-H1 family (#14534)

authored and
qnixsynapse
committed
llama : remove unintended whitespace (#14592)

CISC
authored and
qnixsynapse
committed
model : add skt/A.X-4.0 model vocabulary (#14589)

Bing-su
authored and
qnixsynapse
committed
ggml : prevent integer overflow in gguf tensor size calculation (#14595)

Yuuoniy
authored and
qnixsynapse
committed
ggml : add ggml_scale_bias (#14417)

ngxson
authored and
qnixsynapse
committed
llama : support Jamba hybrid Transformer-Mamba models (#7531)

authored and
qnixsynapse
committed
llama : remove llm_graph_input_one (#14603)

ngxson
authored and
qnixsynapse
committed

Sync master with upstream release b5857#162Minh141120 merged 188 commits intodevjanhq/llama.cpp:devfrom update-dev-from-master-2025-07-10-00-09janhq/llama.cpp:update-dev-from-master-2025-07-10-00-09Copy head branch name to clipboard

Commits

Commits on Jul 10, 2025

Sync master with upstream release b5857#162
Minh141120 merged 188 commits intodevfrom
update-dev-from-master-2025-07-10-00-09