Cherry 1220#6

Merged

arthw merged 245 commits intomasterfrom

cherry-1220

Dec 20, 2024

Commits on Dec 20, 2024

Vulkan: Fix device info output format specifiers (#10366)

0cc4m
authored and
arthw
committed
flake.lock: Update (#10346)

authored and
arthw
committed
vulkan: remove use of null initializer (#10372)

jeffbolznv
authored and
arthw
committed
Skip searching root path for cross-compile builds (#10383)

bandoti
authored and
arthw
committed
cuda : only use native when supported by cmake (#10389)

slaren
authored and
arthw
committed
sycl: Revert MUL_MAT_OP support changes (#10385)

Alcpz
authored and
arthw
committed
vulkan: Optimize soft_max (#10301)

jeffbolznv
authored and
arthw
committed
sycl : Add option to set the SYCL architecture for all targets (#10266)

Rbiessy
authored and
arthw
committed
llama : add OLMo November 2024 support (#10394)

2015aroras
authored and
arthw
committed
llama : add check for KV cache shifts (#10401)

ggerganov
authored and
arthw
committed
cuda : fix CUDA_FLAGS not being applied (#10403)

slaren
authored and
arthw
committed
Add required ggml-base and backend libs to cmake pkg (#10407)

bandoti
authored and
arthw
committed
cmake: force MSVC compiler charset to utf-8 (#9989)

MakeDecisionWorth
authored and
arthw
committed
metal : add `GGML_UNARY_OP_ELU` kernel (ggml/1018)

PABannier
authored and
arthw
committed
metal : fox offset integer overflows in im2col (ggml/1015)

pminev
authored and
arthw
committed
sync : ggml

ggerganov
authored and
arthw
committed
add cmake rvv support (#10411)

lhpqaq
authored and
arthw
committed
Fix missing file renames in Makefile due to changes in commit ae8de6d50a (#10413)

avdg
authored and
arthw
committed
update rel to 4040 (#10395)

NeoZhangJianyu
and
arthw
committed
vulkan: further optimize mul_mat_vec using larger loads (#10387)

jeffbolznv
authored and
arthw
committed
vulkan: copy iq4_nl LUT into shared memory (#10409)

jeffbolznv
authored and
arthw
committed
llama : add .clang-format file (#10415)

slaren
authored and
arthw
committed
cmake: add link dependencies to cmake find pkg (#10433)

bandoti
authored and
arthw
committed
vulkan: predicate max operation in soft_max shaders/soft_max (#10437)

jeffbolznv
authored and
arthw
committed
ggml-opt: fix data corruption (ggml/1022)

JohannesGaessler
authored and
arthw
committed
ggml/sched : do not skip views in pre-assignments

slaren
authored and
arthw
committed
sync : ggml

ggerganov
authored and
arthw
committed
llama : handle KV shift for recurrent models (#10402)

ggerganov
authored and
arthw
committed
cuda : optimize argmax (#10441)

authored and
arthw
committed
CANN: Support Ascend310P to accelerate F32 and F16 Model (#10216)

leo-pony
authored and
arthw
committed
GitHub: ask for more info in issue templates (#10426)

JohannesGaessler
authored and
arthw
committed
ci: Update oneAPI runtime dll packaging (#10428)

MakeDecisionWorth
authored and
arthw
committed
ggml : do not use ARM features not included in the build (#10457)

slaren
authored and
arthw
committed
fix gguf-py: Conversion error when multiple licenses are configured (#9807)

authored and
arthw
committed
convert : XLMRoberta Type Vocab Size (#10458)

gabe-l-hart
authored and
arthw
committed
llama : fix op mul check with command-r-plus (#10476)

slaren
authored and
arthw
committed
flake.lock: Update (#10470)

authored and
arthw
committed
speculative : refactor and add a simpler example (#10362)

ggerganov
authored and
arthw
committed
[SYCL] Fix building Win package for oneAPI 2025.0 update (#10483)

NeoZhangJianyu
and
arthw
committed
metal : minor code formatting

ggerganov
authored and
arthw
committed
tests : fix compile warning

ggerganov
authored and
arthw
committed
ggml : add support for dynamic loading of backends (#10469)

authored and
arthw
committed
server : add speculative decoding support (#10455)

ggerganov
authored and
arthw
committed
Add download chat feature to server chat (#10481)

authored and
arthw
committed
Github: update issue templates [no ci] (#10489)

JohannesGaessler
authored and
arthw
committed
llama : accept a list of devices to use to offload a model (#10497)

slaren
authored and
arthw
committed
Rename Olmo1124 to Olmo2 (#10500)

2015aroras
authored and
arthw
committed
metal : enable mat-vec kernels for bs <= 4 (#10491)

ggerganov
authored and
arthw
committed
server : enable cache_prompt by default (#10501)

ggerganov
authored and
arthw
committed
server : add more information about error (#10455)

ggerganov
authored and
arthw
committed
ci : build docker images only once daily (#10503)

slaren
authored and
arthw
committed
Introduce llama-run (#10291)

ericcurtin
authored and
arthw
committed
vulkan: Fix a vulkan-shaders-gen arugment parsing error (#10484)

sparkleholic
authored and
arthw
committed
CANN: RoPE and CANCAT operator optimization (#10488)

authored and
arthw
committed
CANN: Improve the Inferencing Performance for Ascend NPU Device (#10454)

authored and
arthw
committed
speculative : simplify the implementation (#10504)

ggerganov
authored and
arthw
committed
server : fix parallel speculative decoding (#10513)

ggerganov
authored and
arthw
committed
ggml-cpu: cmake add arm64 cpu feature check for macos (#10487)

chaxu01
authored and
arthw
committed
ci : add ubuntu cuda build, build with one arch on windows (#10456)

slaren
authored and
arthw
committed
ci : publish the docker images created during scheduled runs (#10515)

slaren
authored and
arthw
committed
cmake : enable warnings in llama (#10474)

ggerganov
authored and
arthw
committed
restore the condistion to build & update pacakge when merge (#10507)

NeoZhangJianyu
and
arthw
committed
server : replace behave with pytest (#10416)

ngxson
authored and
arthw
committed
vulkan: fix group_norm (#10496)

jeffbolznv
authored and
arthw
committed
mtgpu: Add MUSA_DOCKER_ARCH in Dockerfiles && update cmake and make (#10516)

yeahdongcn
authored and
arthw
committed
Fix HIP flag inconsistency & build docs (#10524)

tristandruyen
authored and
arthw
committed
llama : disable warnings for 3rd party sha1 dependency (#10527)

slaren
authored and
arthw
committed
ci : remove nix workflows (#10526)

slaren
authored and
arthw
committed
Add OLMo 2 model in docs (#10530)

2015aroras
authored and
arthw
committed
ci : fix cuda releases (#10532)

slaren
authored and
arthw
committed
vulkan: optimize Q2_K and Q3_K mul_mat_vec (#10459)

jeffbolznv
authored and
arthw
committed
vulkan: skip integer div/mod in get_offsets for batch_idx==0 (#10506)

jeffbolznv
authored and
arthw
committed
vulkan: further optimize q5_k mul_mat_vec (#10479)

jeffbolznv
authored and
arthw
committed
vulkan: Handle GPUs with less shared memory (#10468)

jeffbolznv
authored and
arthw
committed
vulkan: define all quant data structures in types.comp (#10440)

jeffbolznv
authored and
arthw
committed
Do not include arm_neon.h when compiling CUDA code (ggml/1028)

frankier
authored and
arthw
committed
sync : ggml

ggerganov
authored and
arthw
committed
metal : fix group_norm support condition (#0)

ggerganov
authored and
arthw
committed
ci : faster CUDA toolkit installation method and use ccache (#10537)

slaren
authored and
arthw
committed
Add some minimal optimizations for CDNA (#10498)

IMbackK
authored and
arthw
committed
common : fix duplicated file name with hf_repo and hf_file (#10550)

ngxson
authored and
arthw
committed
CANN: ROPE operator optimization (#10540)

authored and
arthw
committed
CANN: Fix SOC_TYPE compile bug (#10519)

leo-pony
authored and
arthw
committed
CANN: Update cann.md to display correctly in CLion (#10538)

HRXWEB
authored and
arthw
committed
kompute : improve backend to pass test_backend_ops (#10542)

slp
authored and
arthw
committed
ggml-cpu: support IQ4_NL_4_4 by runtime repack (#10541)

FanShupei
authored and
arthw
committed
cmake : fix ARM feature detection (#10543)

ggerganov
authored and
arthw
committed
ggml : fix row condition for i8mm kernels (#10561)

ggerganov
authored and
arthw
committed
ci : fix tag name in cuda and hip releases (#10566)

slaren
authored and
arthw
committed
docs: fix outdated usage of llama-simple (#10565)

rand-fly
authored and
arthw
committed
common: fix warning message when no GPU found (#10564)

JohannesGaessler
authored and
arthw
committed
server : (tests) don't use thread for capturing stdout/stderr, bump openai client library (#10568)

ngxson
authored and
arthw
committed
llama : add missing model types

ggerganov
authored and
arthw
committed
ggml : remove redundant copyright notice + update authors

ggerganov
authored and
arthw
committed
llava: return false instead of exit (#10546)

tinglou
authored and
arthw
committed
vulkan: get the first command buffer submitted sooner (#10499)

jeffbolznv
authored and
arthw
committed
CANN: RoPE operator optimization (#10563)

authored and
arthw
committed
sycl : Reroute permuted mul_mats through oneMKL (#10408)

Alcpz
authored and
arthw
committed
sycl : offload of get_rows set to 0 (#10432)

Alcpz
authored and
arthw
committed
ggml-cpu: fix typo in gemv/gemm iq4_nl_4_4 (#10580)

FanShupei
authored and
arthw
committed
ggml : fix I8MM Q4_1 scaling factor conversion (#10562)

ggerganov
authored and
arthw
committed
cleanup UI link list (#10577)

slaren
authored and
arthw
committed
imatrix : support combine-only (#10492)

robbiemu
authored and
arthw
committed
server : add more test cases (#10569)

ngxson
authored and
arthw
committed
ggml : move AMX to the CPU backend (#10570)

authored and
arthw
committed
vulkan: Dynamic subgroup size support for Q6_K mat_vec (#10536)

netrunnereve
authored and
arthw
committed
readme : refresh (#10587)

ggerganov
authored and
arthw
committed
readme : remove old badge

ggerganov
authored and
arthw
committed
ggml-cpu: replace AArch64 NEON assembly with intrinsics in ggml_gemv_q4_0_4x4_q8_0() (#10567)

angt
authored and
arthw
committed
build: update Makefile comments for C++ version change (#10598)

wangqin0
authored and
arthw
committed
readme : update the usage section with examples (#10596)

ggerganov
authored and
arthw
committed
server : bind to any port when specified (#10590)

alek3y
authored and
arthw
committed
ggml : automatic selection of best CPU backend (#10606)

slaren
authored and
arthw
committed
ci: add error handling for Python venv creation in run.sh (#10608)

wangqin0
authored and
arthw
committed
grammars : add English-only grammar (#10612)

ggerganov
authored and
arthw
committed
Add `mistral-v1`, `mistral-v3`, `mistral-v3-tekken` and `mistral-v7` chat template types (#10572)

jukofyork
authored and
arthw
committed
contrib : refresh (#10593)

ggerganov
authored and
arthw
committed
server: Add "tokens per second" information in the backend (#10548)

authored and
arthw
committed
make : deprecate (#10514)

authored and
arthw
committed
llama : add enum for built-in chat templates (#10623)

ngxson
authored and
arthw
committed
server : fix default draft model parameters (#10586)

ggerganov
authored and
arthw
committed
github : minify link [no ci]

ggerganov
authored and
arthw
committed
github : minify link [no ci] (revert)

ggerganov
authored and
arthw
committed
metal : small-batch mat-mul kernels (#10581)

ggerganov
authored and
arthw
committed
readme : add option, update default value, fix formatting (#10271)

pothitos
authored and
arthw
committed
llama : add missing LLAMA_API for llama_chat_builtin_templates (#10636)

ngxson
authored and
arthw
committed
metal : add `GGML_OP_CONV_TRANSPOSE_1D` kernels (ggml/1026)

PABannier
authored and
arthw
committed
feat: add `GGML_UNARY_OP_ARGMAX` Metal kernel (ggml/1019)

authored and
arthw
committed
CUDA: remove unnecessary warp reduce in FA (ggml/1032)

authored and
arthw
committed
sync : ggml

ggerganov
authored and
arthw
committed
scripts : remove amx sync

ggerganov
authored and
arthw
committed
server : (web ui) Various improvements, now use vite as bundler (#10599)

ngxson
authored and
arthw
committed
vulkan: optimize and reenable split_k (#10637)

jeffbolznv
authored and
arthw
committed
clip : add sycl support (#10574)

authored and
arthw
committed
Add docs for creating a static build (#10268) (#10630)

authored and
arthw
committed
Avoid using __fp16 on ARM with old nvcc (#10616)

frankier
authored and
arthw
committed
fix typo of README.md (#10605)

WrRan
authored and
arthw
committed
SYCL : Move to compile time oneMKL interface backend selection for NVIDIA backend (#10584)

s-Nick
authored and
arthw
committed
vulkan: Implement "fast divide" (mul+shift) for unary ops like copy (#10642)

jeffbolznv
authored and
arthw
committed
llama: Support MiniCPM-1B (with & w/o longrope) (#10559)

JFLFY2255
authored and
arthw
committed
Fix HF repo commit to clone lora test models (#10649)

ltoniazzi
authored and
arthw
committed
ggml-cpu : fix HWCAP2_I8MM value (#10646)

slaren
authored and
arthw
committed
ggml : add predefined list of CPU backend variants to build (#10626)

slaren
authored and
arthw
committed
server : fix speculative decoding with context shift (#10641)

ggerganov
authored and
arthw
committed
Update deprecation-warning.cpp (#10619)

aryantandon01
authored and
arthw
committed
py : update outdated copy-paste instructions [no ci] (#10667)

danbev
authored and
arthw
committed
ggml : add `GGML_PAD_REFLECT_1D` operation (ggml/1034)

PABannier
authored and
arthw
committed
ggml: add `GGML_SET` Metal kernel + i32 CPU kernel (ggml/1037)

PABannier
authored and
arthw
committed
sync : ggml

ggerganov
authored and
arthw
committed
llama : add Minerva 7B model support (#10673)

Riccorl
authored and
arthw
committed
vulkan: Add VK_NV_cooperative_matrix2 support for mul_mat and flash attention (#10206)

jeffbolznv
authored and
arthw
committed
fix(server) : not show alert when DONE is received (#10674)

pminev
authored and
arthw
committed
server : (refactoring) do not rely on JSON internally (#10643)

ngxson
authored and
arthw
committed
common : bring back --no-warmup to server (#10686)

ngxson
authored and
arthw
committed
convert : add custom attention mapping

ggerganov
authored and
arthw
committed
convert : add support for Roberta embeddings (#10695)

Ssukriti
authored and
arthw
committed
metal : Extend how Llama.cpp locates metal resources (#10676)

authored and
arthw
committed
Vulkan: VK_KHR_cooperative_matrix support to speed up prompt processing (#10597)

0cc4m
authored and
arthw
committed
server : fix free of spec context and batch (#10651)

ggerganov
authored and
arthw
committed
ggml : refactor online repacking (#10446)

authored and
arthw
committed
server : various fixes (#10704)

authored and
arthw
committed
ggml : disable iq4_nl interleave size 8 (#10709)

ggerganov
authored and
arthw
committed
server : (refactor) no more json in server_task input (#10691)

ngxson
authored and
arthw
committed
llama : add 128k yarn context for Qwen (#10698)

robbiemu
authored and
arthw
committed
vulkan: compile a test shader in cmake to check for coopmat2 support (#10713)

jeffbolznv
authored and
arthw
committed
llama : use cmake for swift build (#10525)

authored and
arthw
committed
Vulkan: fix NaN in tanh.comp with AMD proprietary driver on Windows (#10723)

stduhpf
authored and
arthw
committed
server : bring back info of final chunk in stream mode (#10722)

ngxson
authored and
arthw
committed
server : fix format_infill (#10724)

ngxson
authored and
arthw
committed
cmake : simplify msvc charsets (#10672)

iboB
authored and
arthw
committed
vulkan: fix compile warnings (#10731)

jeffbolznv
authored and
arthw
committed
Changes to CMakePresets.json to add ninja clang target on windows (#10668)

authored and
arthw
committed
CUDA: fix shared memory access condition for mmv (#10740)

JohannesGaessler
authored and
arthw
committed
vulkan: disable spirv-opt for coopmat shaders (#10763)

jeffbolznv
authored and
arthw
committed
server : add flag to disable the web-ui (#10762) (#10751)

authored and
arthw
committed
CUDA: rename macros to avoid conflicts with WinAPI (#10736)

aendk
authored and
arthw
committed
imatrix : Add imatrix to --no-context-shift (#10766)

bartowski1182
authored and
arthw
committed
vulkan: dynamic subgroup size for the remaining k quants (#10745)

netrunnereve
authored and
arthw
committed
vulkan: request round-to-even for fp16 in im2col/rope_head (#10767)

jeffbolznv
authored and
arthw
committed
ggml: load all backends from a user-provided search path (#10699)

authored and
arthw
committed
docs: fix server documentation formatting (#10776)

CentricStorm
authored and
arthw
committed
bug-fix: snprintf prints NULL in place of the last character (#10419)

kallewoof
authored and
arthw
committed
ci : pin nodejs to 22.11.0 (#10779)

ngxson
authored and
arthw
committed
Update README.md (#10772)

qingy1337
authored and
arthw
committed
server : (UI) add tok/s, get rid of completion.js (#10786)

ngxson
authored and
arthw
committed
gguf-py : bump version to 0.11.0

ggerganov
authored and
arthw
committed
docs: update server streaming mode documentation (#9519)

CentricStorm
authored and
arthw
committed
common : add missing env var for speculative (#10801)

ngxson
authored and
arthw
committed
Vulkan: Add VK_EXT_subgroup_size_control support to ensure full subgroups for coopmats (#10721)

0cc4m
authored and
arthw
committed
Vulkan: Use improved q4_k and q5_k dequant code in dequant shaders (#10798)

0cc4m
authored and
arthw
committed
remove CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS (#10797)

slaren
authored and
arthw
committed
CUDA: faster non-contiguous concat (#10760)

authored and
arthw
committed
contrib : add ngxson as codeowner (#10804)

ngxson
authored and
arthw
committed
common : improve -ctv -ctk CLI arguments (#10806)

ngxson
authored and
arthw
committed
ggml : Fix compilation issues on ARM platform when building without fp16 (#10811)

kkontny
authored and
arthw
committed
SYCL: Reduce most of the compiler warnings (#10748)

authored and
arthw
committed
vulkan: small mul_mat_vec optimizations (#10665)

netrunnereve
authored and
arthw
committed
Fix crash caused by ggml_backend_load_all when launching on Android Activity (#10812)

authored and
arthw
committed
gguf-py : numpy 2 newbyteorder fix (#9772)

jettjaniak
authored and
arthw
committed
fix: graceful shutdown for Docker images (#10815)

co42
authored and
arthw
committed
Opt class for positional argument handling (#10508)

ericcurtin
authored and
arthw
committed
Introducing experimental OpenCL backend with support for Qualcomm Adreno GPUs (#10693)

authored and
arthw
committed
Removes spurious \r in output that causes logging in journalctl to treat lines as binary and therefore hidden by default (#10771)

authored and
arthw
committed
llama : add Qwen2VL support + multimodal RoPE (#10361)

authored and
arthw
committed
nix: allow to override rocm gpu targets (#10794)

kurnevsky
authored and
arthw
committed
server: Fix `has_next_line` in JSON response (#10818)

MichelleTanPY
authored and
arthw
committed
gguf-py : bump to v0.13.0

ggerganov
authored and
arthw
committed
server: (UI) add syntax highlighting and latex math rendering (#10808)

authored and
arthw
committed
scripts : change build path to "build-bench" for compare-commits.sh (#10836)

ggerganov
authored and
arthw
committed
llama : add Deepseek MoE v1 & GigaChat models (#10827)

Inf1delis
authored and
arthw
committed
llava : Allow locally downloaded models for QwenVL (#10833)

authored and
arthw
committed
sampling : refactor + optimize penalties sampler (#10803)

authored and
arthw
committed
unicode : improve naming style (#10838)

ggerganov
authored and
arthw
committed
rwkv6: add wkv6 support for Vulkan backend (#10829)

authored and
arthw
committed
vulkan: bugfixes for small subgroup size systems + llvmpipe test (#10809)

netrunnereve
authored and
arthw
committed
server : (UI) fix missing async generator on safari (#10857)

ngxson
authored and
arthw
committed
readme : update typos (#10863)

ruanych
authored and
arthw
committed
llama : add Falcon3 support (#10864)

mokeddembillel
authored and
arthw
committed
server : fill usage info in embeddings and rerank responses (#10852)

krystiancha
authored and
arthw
committed
ggml : update ggml_backend_cpu_device_supports_op (#10867)

ggerganov
authored and
arthw
committed
ggml : add check for grad_accs (ggml/1046)

danbev
authored and
arthw
committed
ggml : remove return from ggml_gallocr_allocate_node (ggml/1048)

danbev
authored and
arthw
committed
vulkan : fix soft_max.comp division by zero (whisper/2633)

gn64
authored and
arthw
committed
cmake : fix "amd64" processor string (whisper/2638)

ggerganov
authored and
arthw
committed
sync : ggml

ggerganov
authored and
arthw
committed
tests: add tests for GGUF (#10830)

JohannesGaessler
authored and
arthw
committed
Use model->gguf_kv for loading the template instead of using the C API. (#10868)

dranger003
authored and
arthw
committed
Revert "llama : add Falcon3 support (#10864)" (#10876)

slaren
authored and
arthw
committed
docs: Fix HIP (née hipBLAS) in README (#10880)

brianredbeard
authored and
arthw
committed
server : (embeddings) using same format for "input" and "content" (#10872)

ngxson
authored and
arthw
committed
server : add "tokens" output (#10853)

authored and
arthw
committed
server : output embeddings for all tokens when pooling = none (#10861)

authored and
arthw
committed
server: avoid overwriting Authorization header (#10878)

authored and
arthw
committed
tts : add OuteTTS support (#10784)

ggerganov
authored and
arthw
committed
ggml : fix arm build (#10890)

authored and
arthw
committed
llama-run : improve progress bar (#10821)

ericcurtin
authored and
arthw
committed
tests: disable GGUF test for bad value size (#10886)

JohannesGaessler
authored and
arthw
committed
convert : Add support for Microsoft Phi-4 model (#10817)

authored and
arthw
committed
llama : fix Roberta embeddings (#10856)

authored and
arthw
committed
ggml: fix arm build with gcc (#10895)

angt
authored and
arthw
committed
server : fix logprobs, make it OAI-compatible (#10783)

ngxson
authored and
arthw
committed
tts : small QoL for easy model fetch (#10903)

ggerganov
authored and
arthw
committed
llama : minor grammar refactor (#10897)

ggerganov
authored and
arthw
committed
clip : disable GPU support (#10896)

ggerganov
authored and
arthw
committed
correct the device info format
arthw
committed

Cherry 1220#6arthw merged 245 commits intomasterarthw/llama.cpp:masterfrom cherry-1220arthw/llama.cpp:cherry-1220Copy head branch name to clipboard

Commits

Commits on Dec 20, 2024

Cherry 1220#6
arthw merged 245 commits intomasterfrom
cherry-1220