Sync master with upstream release b5833#152

Closed

jan-service-account wants to merge 378 commits intoarchive-devfrom

update-dev-from-master-2025-07-05-15-07

This pull request is big! We're only showing the most recent 250 commits

Commits on Jun 23, 2025

Commits on Jun 25, 2025

sycl: GGML_SYCL_DISABLE_OPT on by default for all Intel Devices (#13973)
ShanoToni
authored
ggml : do not output unprintable characters on GGUF load failure (#14381)
CISC
authored
ggml-cpu: enable IBM NNPA Vector Intrinsics (#14317)

taronaeo
and
slaren
authored

Commits on Jun 26, 2025

Commits on Jul 1, 2025

Commits on Jul 2, 2025

Commits on Jul 3, 2025

Commits on Jul 5, 2025

test-backend-ops: add support for specifying output format (#14368)

yeahdongcn
and
slaren
authored
eval-callback : check for empty input (#14539)
ggerganov
authored
opencl: add GELU_ERF (#14476)
CISC
authored
server : fix assistant prefilling when content is an array (#14360)
CISC
authored
vulkan: Handle updated FA dim2/3 definition (#14518)
jeffbolznv
authored
add geglu activation function (#14074)

authored and
Minh141120
committed
sycl: Add reorder to Q6_K mmvq implementation (#13885)

s-Nick
authored and
Minh141120
committed
CANN: Simplify the environment variable setting(#13104)

bachelor-dou
authored and
Minh141120
committed
graph : fix geglu (#14077)

ggerganov
authored and
Minh141120
committed
ggml-cpu : split arch-specific implementations (#13892)

authored and
Minh141120
committed
llama : allow building all tests on windows when not using shared libs (#13980)

authored and
Minh141120
committed
sync : ggml

ggerganov
authored and
Minh141120
committed
Vulkan: Don't default to CPU device (like llvmpipe), even if no other device is available, to allow fallback to CPU backend (#14099)

0cc4m
authored and
Minh141120
committed
ggml : fix weak alias win32 (whisper/0)

ggerganov
authored and
Minh141120
committed
sync : ggml

ggerganov
authored and
Minh141120
committed
llama : support GEGLU for jina-bert-v2 (#14090)

CISC
authored and
Minh141120
committed
convert : fix duplicate key DeepSeek-R1 conversion error (#14103)

CISC
authored and
Minh141120
committed
kv-cache : avoid modifying recurrent cells when setting inputs (#13834)

compilade
authored and
Minh141120
committed
opencl: add `mul_mv_id_q4_0_f32_8x_flat` (#14003)

lhez
authored and
Minh141120
committed
vulkan: Track descriptor pools/sets per-context (#14109)

jeffbolznv
authored and
Minh141120
committed
kv-cache : add LLAMA_KV_CACHE_DEBUG environment variable (#14121)

ggerganov
authored and
Minh141120
committed
kv-cache : relax SWA masking condition (#14119)

ggerganov
authored and
Minh141120
committed
vulkan: Better thread-safety for command pools/buffers (#14116)

jeffbolznv
authored and
Minh141120
committed
tests : add test-tokenizers-repo (#14017)

CISC
authored and
Minh141120
committed
chore : clean up relative source dir paths (#14128)

CISC
authored and
Minh141120
committed
Implement GGML_CPU_ALL_VARIANTS for ARM (#14080)

ckastner
authored and
Minh141120
committed
kv-cache : fix split_equal handling in unified implementation (#14130)

ggerganov
authored and
Minh141120
committed
batch : remove logits_all flag (#14141)

ggerganov
authored and
Minh141120
committed
context : simplify output counting logic during decode (#14142)

ggerganov
authored and
Minh141120
committed
cmake : Improve build-info.cpp generation (#14156)

ckastner
authored and
Minh141120
committed
cmake: Add ability to pass in LLAMA_BUILD_NUMBER/COMMIT (#14167)

ckastner
authored and
Minh141120
committed
batch : rework llama_batch_allocr (#14153)

ggerganov
authored and
Minh141120
committed
batch : add LLAMA_BATCH_DEBUG environment variable (#14172)

ggerganov
authored and
Minh141120
committed
Merge commit from fork

authored and
Minh141120
committed
vocab : fix build (#14175)

ggerganov
authored and
Minh141120
committed
batch : auto-gen positions + verify multi-sequence input (#14177)

ggerganov
authored and
Minh141120
committed
cparams : rename LLAMA_MAX_PARALLEL_SEQUENCES to LLAMA_MAX_SEQ (#14188)

ggerganov
authored and
Minh141120
committed
model : add dots.llm1 architecture support (#14044) (#14118)

Noeda
authored and
Minh141120
committed
model : Add support for Arcee AI's upcoming AFM model (#14185)

authored and
Minh141120
committed
ggml-cpu : rework weak alias on apple targets (#14146)

xctan
authored and
Minh141120
committed
vulkan: mutex around vkQueueSubmit (#14127)

jeffbolznv
authored and
Minh141120
committed
convert : remove arcee change in convert_hf_to_gguf_update.py (#14207)

bartowski1182
authored and
Minh141120
committed
ggml: Add Android support for GGML_CPU_ALL_VARIANTS (#14206)

chaxu01
authored and
Minh141120
committed
llama : rework embeddings logic (#14208)

ggerganov
authored and
Minh141120
committed
model : add NeoBERT (#14164)

authored and
Minh141120
committed
cmake: clean up external project logic for vulkan-shaders-gen (#14179)

bandoti
authored and
Minh141120
committed
server : fix incorrect usage of llama_get_embeddings() (#14225)

ggerganov
authored and
Minh141120
committed
ggml-cpu : remove the weak alias trick (#14221)

xctan
authored and
Minh141120
committed
cmake: remove shader-gen step-targets from ggml-vulkan (#14226)

bandoti
authored and
Minh141120
committed
examples : include examples in msvc disable warn (ggml/1270)

danbev
authored and
Minh141120
committed
ggml : disable warnings for tests when using MSVC (ggml/1273)

danbev
authored and
Minh141120
committed
sync : ggml

ggerganov
authored and
Minh141120
committed
convert : fix null head_dim AutoConfig regression (#14248)

CISC
authored and
Minh141120
committed
ggml: Add Apple support for GGML_CPU_ALL_VARIANTS (#14258)

chaxu01
authored and
Minh141120
committed
docs: add s390x build documentation (#14264)

taronaeo
authored and
Minh141120
committed
metal : add mean kernel (#14267)

ggerganov
authored and
Minh141120
committed
memory : Hybrid recurrent cache (#13979)

authored and
Minh141120
committed
llamafile : support s390x SIMD instruction set (#14273)

taronaeo
authored and
Minh141120
committed
ggml-cpu : remove unnecesary arm feature detection (#14281)

slaren
authored and
Minh141120
committed
CUDA: add conv_2d_dw (#14265)

am17an
authored and
Minh141120
committed
ubatch : new splitting logic (#14217)

ggerganov
authored and
Minh141120
committed
model : more uniform output id handling (#14275)

ggerganov
authored and
Minh141120
committed
ggml : fix repack work size for mul_mat_id (#14292)

ggerganov
authored and
Minh141120
committed
llama : improve sep token handling (#14272)

CISC
authored and
Minh141120
committed
Implement GGML_CPU_ALL_VARIANTS for PowerPC (#14286)

authored and
Minh141120
committed
sycl: add usage of enqueue_functions extension (#14244)

s-Nick
authored and
Minh141120
committed
vocab : prevent tokenizer overflow (#14301)

retr0reg
authored and
Minh141120
committed
lint : remove trailing whitepace (#14304)

CISC
authored and
Minh141120
committed
CUDA: add conv_2d_transpose (#14287)

am17an
authored and
Minh141120
committed
Add `ggml_roll` (ggml/1274)

Acly
authored and
Minh141120
committed
sync : ggml

ggerganov
authored and
Minh141120
committed
memory : rename interface to llama_memory_context_i (#14296)

ggerganov
authored and
Minh141120
committed
gguf-py : fix TemplateProcessing pair when bos/eos is missing (#14312)

CISC
authored and
Minh141120
committed
Add support for VK_EXT_debug_utils to add labels to Vulkan objects. (#13792)

mtavenrath
authored and
Minh141120
committed
gguf-py : fix Qwen3-Embedding eos token (#14314)

CISC
authored and
Minh141120
committed
CUDA: add mean operation (#14313)

am17an
authored and
Minh141120
committed
HIP: enable vec fattn on RDNA4 (#14323)

IMbackK
authored and
Minh141120
committed
examples : fix is_first logic for tokenization (#14329)

ggerganov
authored and
Minh141120
committed
run : avoid double tokenization (#14327)

retr0reg
authored and
Minh141120
committed
gguf-py : fix SpecialVocab parsing when post_processor is null (#14330)

CISC
authored and
Minh141120
committed
kv-cells : fix tracking of seq_pos (#14339)

ggerganov
authored and
Minh141120
committed
CUDA: mul_mat_v support for batch sizes > 1 (#14262)

JohannesGaessler
authored and
Minh141120
committed
CUDA/HIP: optimize mmv paths taken for HIP devices (#14324)

authored and
Minh141120
committed
cmake : use LLAMA_BUILD_NUMBER when defining LLAMA_INSTALL_VERSION (#14362)

mbaudier
authored and
Minh141120
committed
batch : fix check for empty sequences in memory (#14364)

ggerganov
authored and
Minh141120
committed
opencl: ref count `ggml_backend_opencl_context` and refactor profiling (#14254)

lhez
authored and
Minh141120
committed
ggml-cpu: enable IBM NNPA Vector Intrinsics (#14317)

authored and
Minh141120
committed
musa: enable fp16 mma (all) and cublas on qy2 (#13842)

authored and
Minh141120
committed
docs: update s390x documentation + add faq (#14389)

taronaeo
authored and
Minh141120
committed
metal : batch rows copy in a single threadgroup (#14384)

ggerganov
authored and
Minh141120
committed
cmake: regen vulkan shaders when shaders-gen sources change (#14398)

bandoti
authored and
Minh141120
committed
model : gemma3n text-only (#14400)

ngxson
authored and
Minh141120
committed
convert : fix broken sentencepiece vocab (#14416)

CISC
authored and
Minh141120
committed
ggml : add ggml_set_rows (#14274)

authored and
Minh141120
committed
recurrent : call balloc split_reset() in init_batch() (#14414)

ggerganov
authored and
Minh141120
committed
graph : make llm_graph_context destructor virtual (#14410)

ggerganov
authored and
Minh141120
committed
ci : fix windows build and release (#14431)

CISC
authored and
Minh141120
committed
fix async_mode bug (#14432)

bachelor-dou
authored and
Minh141120
committed
model : add support for ERNIE 4.5 0.3B model (#14408)

ownia
authored and
Minh141120
committed
vulkan: lock accesses of pinned_memory vector (#14333)

jeffbolznv
authored and
Minh141120
committed
CUDA: add bf16 and f32 support to cublas_mul_mat_batched (#14361)

am17an
authored and
Minh141120
committed
vulkan: Add fusion support for RMS_NORM+MUL (#14366)

authored and
Minh141120
committed
ggml : implement REGLU/GEGLU/SWIGLU ops (#14158)

authored and
Minh141120
committed
ggml : fix unmerged GGML_FPxx_TO_FPxx refactoring (#14443)

CISC
authored and
Minh141120
committed
SYCL: disable faulty fp16 exp kernel (#14395)

qnixsynapse
authored and
Minh141120
committed
scripts : make the shell scripts cross-platform (#14341)

vedranmiletic
authored and
Minh141120
committed
cmake : Remove redundant include path in CMakeLists.txt (#14452)

xiaobing318
authored and
Minh141120
committed
test-backend-ops : disable llama test (#14461)

slaren
authored and
Minh141120
committed
ggml-cpu: sycl: Re-enable exp f16 (#14462)

Rbiessy
authored and
Minh141120
committed
metal : disable fast-math for some cpy kernels (#14460)

ggerganov
authored and
Minh141120
committed
memory : correctly handle failure in apply() (#14438)

ggerganov
authored and
Minh141120
committed
Add Conv2d for CPU (#14388)

am17an
authored and
Minh141120
committed
opencl : add GEGLU, REGLU, SWIGLU (#14456)

lhez
authored and
Minh141120
committed
ggml-cpu : "align corners" for bilinear upscale/downscale (ggml/1285)

Acly
authored and
Minh141120
committed
sync : ggml

ggerganov
authored and
Minh141120
committed
ggml : remove trailing whitespace (#0)

ggerganov
authored and
Minh141120
committed
vulkan: Split large mul_mat_id to fit in shared memory (#14451)

jeffbolznv
authored and
Minh141120
committed
github : add OpenCL backend to issue templates (#14492)

EZForever
authored and
Minh141120
committed
ci : add OpenCL to labeler workflow (#14496)

CISC
authored and
Minh141120
committed
opencl : update upscale to support align corners (#14488)

lhez
authored and
Minh141120
committed
opencl : skip empty nodes on cgraph compute (#14491)

EZForever
authored and
Minh141120
committed
simple-chat : fix context-exceeded condition (#14494)

ggerganov
authored and
Minh141120
committed
opencl : fix possible buffer overflow in dump_tensor (#14490)

jeffzhou2000
authored and
Minh141120
committed
ggml : support bcast ggml_soft_max_ext, ggml_flash_attn_ext (#14435)

ggerganov
authored and
Minh141120
committed
vulkan: support softmax/FA batch and broadcast (#14449)

jeffbolznv
authored and
Minh141120
committed
CUDA: broadcasting for FlashAttention mask (#14500)

JohannesGaessler
authored and
Minh141120
committed
CUDA: add softmax broadcast (#14475)

am17an
authored and
Minh141120
committed
Set RPATH to "@loader_path" / "$ORIGIN" to ensure executables and dynamic libraries search for dependencies in their origin directory. (#14309)

rotemdan
authored and
Minh141120
committed
ggml : add version function to get lib version (ggml/1286)

authored and
Minh141120
committed
sync : ggml

ggerganov
authored and
Minh141120
committed
llama : initial Mamba-2 support (#9126)

compilade
authored and
Minh141120
committed
gguf-py : add support for chat template jinja files (#14508)

CISC
authored and
Minh141120
committed
CUDA: add dynamic shared mem to softmax, refactor general usage (#14497)

am17an
authored and
Minh141120
committed
ggml : remove kompute backend (#14501)

ggerganov
authored and
Minh141120
committed
ggml : fix FA mask dim 2 and 3 (#14505)

ggerganov
authored and
Minh141120
committed
kv-cache : use ggml_set_rows (#14285)

ggerganov
authored and
Minh141120
committed
convert : correct gemma 3n conversion (#14450)

ngxson
authored and
Minh141120
committed
Fix conditional enabling following arch checks for ggml-sycl (#14504)

s-Nick
authored and
Minh141120
committed
ggml: backward pass for split swiglu (#14483)

JohannesGaessler
authored and
Minh141120
committed
vulkan: support mixed/deepseekR1 FA head sizes (#14509)

jeffbolznv
authored and
Minh141120
committed
opencl : broadcast for soft_max (#14510)

lhez
authored and
Minh141120
committed
ggml : implement GEGLU_ERF and GEGLU_QUICK ops (#14445)

CISC
authored and
Minh141120
committed
CANN: Replace aclrtMemsetSync with aclnnInplaceZero operator (#14002)

authored and
Minh141120
committed
batch : add n_used count (#14512)

ggerganov
authored and
Minh141120
committed
graph : prepare for 4D mask (#14515)

ggerganov
authored and
Minh141120
committed
batch : add optional for sequential equal split (#14511)

ggerganov
authored and
Minh141120
committed
metal : disable fast math in all quantize kernels (#14528)

ggerganov
authored and
Minh141120
committed
test-backend-ops: add support for specifying output format (#14368)

authored and
Minh141120
committed
eval-callback : check for empty input (#14539)

ggerganov
authored and
Minh141120
committed
opencl: add GELU_ERF (#14476)

CISC
authored and
Minh141120
committed
server : fix assistant prefilling when content is an array (#14360)

CISC
authored and
Minh141120
committed
vulkan: Handle updated FA dim2/3 definition (#14518)

jeffbolznv
authored and
Minh141120
committed
Refactor and optimize various components in the ggml library
Minh141120
committed
ci: add ARM64 support for Windows in build matrix
Minh141120
committed
ci: add pull request trigger for dev branch in workflow
Minh141120
committed
ci: update pull request trigger paths for dev branch
Minh141120
committed

Sync master with upstream release b5833#152jan-service-account wants to merge 378 commits intoarchive-devjanhq/llama.cpp:archive-devfrom update-dev-from-master-2025-07-05-15-07janhq/llama.cpp:update-dev-from-master-2025-07-05-15-07Copy head branch name to clipboard

Commits

Commits on Jun 22, 2025

Commits on Jun 23, 2025

Commits on Jun 24, 2025

Commits on Jun 25, 2025

Commits on Jun 26, 2025

Commits on Jun 27, 2025

Commits on Jun 28, 2025

Commits on Jun 29, 2025

Commits on Jun 30, 2025

Commits on Jul 1, 2025

Commits on Jul 2, 2025

Commits on Jul 3, 2025

Commits on Jul 4, 2025

Commits on Jul 5, 2025

Sync master with upstream release b5833#152
jan-service-account wants to merge 378 commits intoarchive-devfrom
update-dev-from-master-2025-07-05-15-07