Cherry 1118#5

Merged

arthw merged 581 commits intomasterfrom

cherry-1118

Nov 19, 2024

This pull request is big! We're only showing the most recent 250 commits

Commits on Nov 18, 2024

ggml: unify backend logging mechanism (#9709)

bandoti
authored and
arthw
committed
ggml-backend : add device description to CPU backend (#9720)

slaren
authored and
arthw
committed
metal : fix compute pass descriptor autorelease crash (#9718)

jmousseau
authored and
arthw
committed
ggml: refactor cross entropy loss CPU impl. (ggml/976)

JohannesGaessler
authored and
arthw
committed
ggml/ex: calculate accuracy in graph, adapt MNIST (ggml/980)

JohannesGaessler
authored and
arthw
committed
sync : ggml

ggerganov
authored and
arthw
committed
metal : remove abort (skip) (ggml/0)

ggerganov
authored and
arthw
committed
Fixed RNG seed docs (#9723)

d-kleine
authored and
arthw
committed
ci : fine-grant permission (#9710)

ngxson
authored and
arthw
committed
ggml : fixes after sync (ggml/983)

slaren
authored and
arthw
committed
ggml : fix typo in example usage ggml_gallocr_new (ggml/984)

danbev
authored and
arthw
committed
sync : ggml

ggerganov
authored and
arthw
committed
Add Llama Assistant (#9744)

vietanhdev
authored and
arthw
committed
metal : zero-init buffer contexts (whisper/0)

ggerganov
authored and
arthw
committed
sync : ggml

ggerganov
authored and
arthw
committed
rerank : use [SEP] token instead of [BOS] (#9737)

ggerganov
authored and
arthw
committed
vulkan : retry allocation with fallback flags (whisper/2451)

authored and
arthw
committed
sync : llama.cpp

ggerganov
authored and
arthw
committed
readme : fix typo [no ci]

ggerganov
authored and
arthw
committed
contrib : simplify + minor edits [no ci]

ggerganov
authored and
arthw
committed
metal : single allocation of encode_async block (#9747)

authored and
arthw
committed
ggml : add metal backend registry / device (#9713)

authored and
arthw
committed
flake.lock: Update (#9753)

authored and
arthw
committed
Update building for Android (#9672)

amqdn
authored and
arthw
committed
ggml : add backend registry / device interfaces to BLAS backend (#9752)

slaren
authored and
arthw
committed
scripts : fix spelling typo in messages and comments (#9782)

standby24x7
authored and
arthw
committed
server : better security control for public deployments (#9776)

ngxson
authored and
arthw
committed
ggml : fix BLAS with unsupported types (#9775)

slaren
authored and
arthw
committed
examples : remove llama.vim

ggerganov
authored and
arthw
committed
perplexity : fix integer overflow (#9783)

ggerganov
authored and
arthw
committed
cmake : do not build common library by default when standalone (#9804)

slaren
authored and
arthw
committed
examples : do not use common library in simple example (#9803)

slaren
authored and
arthw
committed
musa: add docker image support (#9685)

yeahdongcn
authored and
arthw
committed
rpc : add backend registry / device interfaces (#9812)

slaren
authored and
arthw
committed
common : use common_ prefix for common library functions (#9805)

authored and
arthw
committed
ggml : move more prints to the ggml log system (#9839)

slaren
authored and
arthw
committed
musa : update doc (#9856)

yeahdongcn
authored and
arthw
committed
llama : improve infill support and special token detection (#9798)

ggerganov
authored and
arthw
committed
server : remove legacy system_prompt feature (#9857)

ggerganov
authored and
arthw
committed
server : remove self-extend features (#9860)

ggerganov
authored and
arthw
committed
server : add option to time limit the generation phase (#9865)

ggerganov
authored and
arthw
committed
flake.lock: Update (#9870)

authored and
arthw
committed
server : reuse cached context chunks (#9866)

ggerganov
authored and
arthw
committed
server : accept extra_context for the infill endpoint (#9874)

ggerganov
authored and
arthw
committed
Vectorize load instructions in dmmv f16 CUDA kernel (#9816)

authored and
arthw
committed
server : handle "logprobs" field with false value (#9871)

authored and
arthw
committed
readme : update bindings list (#9889)

srgtuszy
authored and
arthw
committed
server : update preact (#9895)

ggerganov
authored and
arthw
committed
sampling : add XTC sampler (#9742)

MaggotHATE
authored and
arthw
committed
server : improve infill context reuse (#9894)

ggerganov
authored and
arthw
committed
llama : add infill sampler (#9896)

ggerganov
authored and
arthw
committed
[CANN] Fix cann compilation error (#9891)

leo-pony
authored and
arthw
committed
ggml-alloc : remove buffer_id from leaf_alloc (ggml/987)

danbev
authored and
arthw
committed
sync : ggml

ggerganov
authored and
arthw
committed
server : fix the disappearance of the end of the text (#9867)

z80maniac
authored and
arthw
committed
llama : add tensor name for "result_norm" (#9907)

MollySophia
authored and
arthw
committed
grammar : fix JSON Schema for string regex with top-level alt. (#9903)

jemc
authored and
arthw
committed
llava : fix typo in error message [no ci] (#9884)

danbev
authored and
arthw
committed
llama : suppress conversion from 'size_t' to 'int' (#9046)

danbev
authored and
arthw
committed
fix: use `vm_allocate` to allocate CPU backend buffer on macOS (#9875)

giladgd
authored and
arthw
committed
fix: allocating CPU buffer with size `0` (#9917)

giladgd
authored and
arthw
committed
vulkan : add backend registry / device interfaces (#9721)

slaren
authored and
arthw
committed
readme : update bindings list (#9918)

authored and
arthw
committed
llama : infill sampling handle very long tokens (#9924)

ggerganov
authored and
arthw
committed
llama : change warning to debug log

ggerganov
authored and
arthw
committed
readme : remove --memory-f32 references (#9925)

ggerganov
authored and
arthw
committed
llama : rename batch_all to batch (#8881)

danbev
authored and
arthw
committed
server : add n_indent parameter for line indentation requirement (#9929)

ggerganov
authored and
arthw
committed
add amx kernel for gemm (#8998)

mingfeima
authored and
arthw
committed
[SYCL] Add SYCL Backend registry, device and Event Interfaces (#9705)

OuadiElfarouki
authored and
arthw
committed
rpc : backend refactoring (#9912)

rgerganov
authored and
arthw
committed
llama : remove all_pos_0, all_pos_1, all_seq_id from llama_batch (#9745)

ngxson
authored and
arthw
committed
readme : update infra list (#9942)

icppWorld
authored and
arthw
committed
readme : update bindings list (#9951)

lcarrere
authored and
arthw
committed
fix mul_mat_vec_q and *_vec_q error (#9939)

NeoZhangJianyu
and
arthw
committed
fix error
arthw
committed
llama : default sampling changes + greedy update (#9897)

ggerganov
authored and
arthw
committed
rpc : pack only RPC structs (#9959)

rgerganov
authored and
arthw
committed
ggml : add asserts for type conversion in fattn kernels (#9971)

ggerganov
authored and
arthw
committed
llama.vim : plugin for Neovim (#9787)

ggerganov
authored and
arthw
committed
arg : fix attention non-causal arg value hint (#9985)

danbev
authored and
arthw
committed
readme : update UI list (#9972)

a-ghorbani
authored and
arthw
committed
llama.vim : move info to the right of screen [no ci] (#9787)

ggerganov
authored and
arthw
committed
llama.vim : fix info text display [no ci] (#9787)

ggerganov
authored and
arthw
committed
arg : fix typo in embeddings argument help [no ci] (#9994)

danbev
authored and
arthw
committed
[CANN] Adapt to dynamically loadable backends mechanism (#9970)

leo-pony
authored and
arthw
committed
llama : add chat template for RWKV-World + fix EOT (#9968)

MollySophia
authored and
arthw
committed
lora : warn user if new token is added in the adapter (#9948)

ngxson
authored and
arthw
committed
Rwkv chat template fix (#10001)

authored and
arthw
committed
llama : rename batch to ubatch (#9950)

danbev
authored and
arthw
committed
llama : fix empty batch causing llama_batch_allocr to crash (#9966)

authored and
arthw
committed
flake.lock: Update

github-actions[bot]
authored and
arthw
committed
metal : add POOL2D and fix IM2COL (#9943)

junhee-yoo
authored and
arthw
committed
llama.vim : add classic vim support (#9995)

authored and
arthw
committed
ggml : remove redundant set of contexts used field (ggml/978)

danbev
authored and
arthw
committed
CUDA: fix 1D im2col, add tests (ggml/993)

JohannesGaessler
authored and
arthw
committed
llama.vim : bump generation time limit to 3s [no ci]

ggerganov
authored and
arthw
committed
sync : ggml

ggerganov
authored and
arthw
committed
server : samplers accept the prompt correctly (#10019)

wwoodsTM
authored and
arthw
committed
CUDA: fix MMQ for non-contiguous src0, add tests (#10021)

JohannesGaessler
authored and
arthw
committed
CUDA: fix insufficient buffer clearing for MMQ (#10032)

JohannesGaessler
authored and
arthw
committed
ci : fix cmake flags for SYCL

ggerganov
authored and
arthw
committed
server : refactor slot input data, move tokenizer to HTTP thread (#10023)

ngxson
authored and
arthw
committed
server : check that the prompt fits in the slot's context (#10030)

ggerganov
authored and
arthw
committed
llamafile : extend sgemm.cpp support for Q5_0 models (#10010)

Srihari-mcw
authored and
arthw
committed
llama: string_split fix (#10022)

Xarbirus
authored and
arthw
committed
llama : add DRY sampler (#9702)

authored and
arthw
committed
metal : support permuted matrix multiplicaions (#10033)

ggerganov
authored and
arthw
committed
scripts : fix amx sync [no ci]

ggerganov
authored and
arthw
committed
increase cuda_cpy block size (ggml/996)

authored and
arthw
committed
sync : ggml

ggerganov
authored and
arthw
committed
llama : switch KQ multiplication to F32 precision by default (#10015)

ggerganov
authored and
arthw
committed
server : don't overfill the batch during infill (#10018)

ggerganov
authored and
arthw
committed
musa: workaround for Guilty Lockup in cleaning src0 (#10042)

yeahdongcn
authored and
arthw
committed
flake.lock: Update (#10063)

authored and
arthw
committed
llama : Add IBM granite template (#10013)

authored and
arthw
committed
llama : remove Tail-Free sampling (#10071)

ggerganov
authored and
arthw
committed
ggml: Add POOL2D OP for GPU acceleration to the Vulkan backend in the MobileVLM model. (#9763)

cyzero-kim
authored and
arthw
committed
llama : refactor model loader with backend registry (#10026)

slaren
authored and
arthw
committed
ggml : add Q4_0_8_8 RISC-V GEMV and GEMM kernels (#10029)

xctan
authored and
arthw
committed
convert : more detailed convert lora usage docs (#10065)

richdougherty
authored and
arthw
committed
readme : more lora detail in main example readme (#10064)

richdougherty
authored and
arthw
committed
ggml : fix memory leaks when loading invalid gguf files (#10094)

slaren
authored and
arthw
committed
kompute: add backend registry / device interfaces (#10045)

slp
authored and
arthw
committed
kompute: add mul_mat_q4_k shader (#10097)

slp
authored and
arthw
committed
ggml : check tensor name lengths in gguf files (#10100)

slaren
authored and
arthw
committed
server : include scheme when printing URL (#10106)

bakkot
authored and
arthw
committed
loader: refactor tensor weights storage (#9935)

authored and
arthw
committed
llama : fix buffer checks for mamba and rwk (#10111)

slaren
authored and
arthw
committed
quantize : fix --keep-split (#10114)

slaren
authored and
arthw
committed
llama : improve output buffer type selection (#10098)

slaren
authored and
arthw
committed
build: fix build error in Windows env with OneAPI setup (#10107)

kylo5aby
authored and
arthw
committed
ggml : alloc ggml_contexts on the heap (whisper/2525)

ggerganov
authored and
arthw
committed
sync : ggml

ggerganov
authored and
arthw
committed
ggml : remove ggml_scratch (#10121)

ggerganov
authored and
arthw
committed
server : fix smart selection of available slot (#10120)

sasha0552
authored and
arthw
committed
readme : update hot topics

ggerganov
authored and
arthw
committed
vulkan : improve ggml_vk_create_buffer error handling (#9898)

FanShupei
authored and
arthw
committed
llama : use smart pointers for ggml resources (#10117)

slaren
authored and
arthw
committed
llama : add simple-chat example (#10124)

authored and
arthw
committed
convert-lora : make `--base` optional (#10110)

ngxson
authored and
arthw
committed
simple-chat : only add bos on first prompt (#10129)

slaren
authored and
arthw
committed
llama : adjust default context size + print warnings (#10136)

ggerganov
authored and
arthw
committed
server : fix endpoint checks (#10135)

ggerganov
authored and
arthw
committed
server : fix slot selection by lru (#10126)

sasha0552
authored and
arthw
committed
Add apple arm to presets (#10134)

kohnech
authored and
arthw
committed
flake.lock: Update (#10146)

ggerganov
authored and
arthw
committed
metal : minor fixup in FA kernel (#10143)

ggerganov
authored and
arthw
committed
ggml : move CPU backend to a separate file (#10144)

slaren
authored and
arthw
committed
metal : fix minor string leaks (ggml/1004)

pminev
authored and
arthw
committed
cmake : make it possible linking ggml as external lib (ggml/1003)

ykhrustalev
authored and
arthw
committed
sync : ggml

ggerganov
authored and
arthw
committed
CANN: adjust backend registry refactor. (#10158)

leo-pony
authored and
arthw
committed
metal : move dequantize templates to beginning of MSL source (#0)

ggerganov
authored and
arthw
committed
metal : simplify f16 and f32 dequant kernels (#0)

ggerganov
authored and
arthw
committed
cuda : clear error after changing peer access (#10153)

slaren
authored and
arthw
committed
fix build break on arm64 linux (#10166)

snadampal
authored and
arthw
committed
server : clarify /slots endpoint, add is_processing (#10162)

ngxson
authored and
arthw
committed
ggml : fix q4xx mat mul, increase ggml_aligned_malloc alignment (#10167)

slaren
authored and
arthw
committed
ggml : fix gelu tables initialization (#10172)

slaren
authored and
arthw
committed
Q6_K AVX improvements (#10118)

netrunnereve
authored and
arthw
committed
ggml : fix arch check in bf16_to_fp32 (#10164)

slaren
authored and
arthw
committed
llama : add <|tool_call|> formatting to Granite template (#10177)

gabe-l-hart
authored and
arthw
committed
metal : add quantized FA support (#10149)

ggerganov
authored and
arthw
committed
ggml : adjust is_first_call init value (#10193)

ggerganov
authored and
arthw
committed
metal : fix from ptr buffer name (#10189)

slaren
authored and
arthw
committed
server : remove hack for extra parallel slot (#10187)

ggerganov
authored and
arthw
committed
metal : add BF16 support (#8439)

ggerganov
authored and
arthw
committed
Optimize RWKV6 Operator Naming and Implement Multi-core CPU/ SYCL Acceleration (#10133)

authored and
arthw
committed
fix q4_0_8_8 format for corrupted tokens issue (#10198)

authored and
arthw
committed
DRY: Fixes clone functionality (#10192)

wwoodsTM
authored and
arthw
committed
Remove identical wte/etw logic for jais (#10203)

fmz
authored and
arthw
committed
ggml : add ggml-cpu.h to the public headers (#10204)

slaren
authored and
arthw
committed
scripts : sync update

ggerganov
authored and
arthw
committed
sync : ggml

ggerganov
authored and
arthw
committed
scripts : add amx to sync-ggml.sh [no ci]

ggerganov
authored and
arthw
committed
server : revamp chat UI with vuejs and daisyui (#10175)

ngxson
authored and
arthw
committed
server : minor UI fix (#10207)

ngxson
authored and
arthw
committed
swift : exclude ggml-metal-embed.metal (#10211)

jhen0409
authored and
arthw
committed
metal : optimize FA kernels (#10171)

ggerganov
authored and
arthw
committed
metal : improve clarity (minor) (#10171)

ggerganov
authored and
arthw
committed
metal : opt-in compile flag for BF16 (#10218)

ggerganov
authored and
arthw
committed
scripts : fix pattern and get n_tokens in one go (#10221)

lhpqaq
authored and
arthw
committed
ggml : optimize llamafile cpu matrix multiplication for ppc64le (#10156)

amritahs-ibm
authored and
arthw
committed
ggml: fix zero division in ‘dne’ calculation in CUDA COUNT_EQUAL operator when ‘ne’ is small (#10213)

SongXiaoXi
authored and
arthw
committed
metal : hide debug messages from normal log

ggerganov
authored and
arthw
committed
llama : fix Qwen model type strings

ggerganov
authored and
arthw
committed
metal : fix F32 accumulation in FA vec kernel (#10232)

ggerganov
authored and
arthw
committed
metal : fix build and some more comments (#10229)

ggerganov
authored and
arthw
committed
metal : reorder write loop in mul mat kernel + style (#10231)

ggerganov
authored and
arthw
committed
vulkan: Fix newly added tests for permuted mul_mat and 1D im2col (#10226)

jeffbolznv
authored and
arthw
committed
server : (web UI) Add back sampler settings (#10239)

authored and
arthw
committed
flake.lock: Update (#10243)

authored and
arthw
committed
server : enable KV cache defrag by default (#10233)

ggerganov
authored and
arthw
committed
metal : more precise Q*K in FA vec kernel (#10247)

ggerganov
authored and
arthw
committed
vulkan: Throttle the number of shader compiles during the build step. (#10222)

jeffbolznv
authored and
arthw
committed
vulkan: Optimize contiguous copies (#10254)

jeffbolznv
authored and
arthw
committed
metadata: Detailed Dataset Authorship Metadata (#8875)

mofosyne
authored and
arthw
committed
server : fix incorrect res in validate_model_chat_template (#10272)

jhen0409
authored and
arthw
committed
server : add missing docs (#10269)

z80maniac
authored and
arthw
committed
docs : update bindings list (#10261)

xuegao-tzx
authored and
arthw
committed
sync : ggml

ggerganov
authored and
arthw
committed
llama : propagate the results of `graph_compute` (#9525)

authored and
arthw
committed
vulkan: Use macros to make the mat mul pipeline creation more concise (#10259)

jeffbolznv
authored and
arthw
committed
vulkan: Optimize binary ops (#10270)

jeffbolznv
authored and
arthw
committed
speculative : fix out-of-bounds access (#10289)

ggerganov
authored and
arthw
committed
CUDA: no -sm row for very small matrices (#10185)

JohannesGaessler
authored and
arthw
committed
ggml : build backends as libraries (#10256)

authored and
arthw
committed
backend cpu: add online flow for aarch64 Q4_0 GEMV/GEMM kernels (#9921)

authored and
arthw
committed
sycl: Use syclcompat::dp4a (#10267)

Rbiessy
authored and
arthw
committed
scripts : fix regex in sync [no ci]

ggerganov
authored and
arthw
committed
cann: dockerfile and doc adjustment (#10302)

authored and
arthw
committed
server : (web UI) add copy button for code block, fix api key (#10242)

ngxson
authored and
arthw
committed
sycl: Update Intel docker images to use DPC++ 2025.0 (#10305)

Rbiessy
authored and
arthw
committed
ci: build test musa with cmake (#10298)

yeahdongcn
authored and
arthw
committed
AVX BF16 and single scale quant optimizations (#10212)

netrunnereve
authored and
arthw
committed
sync : ggml

ggerganov
authored and
arthw
committed
ggml : vulkan logs (whisper/2547)

thewh1teagle
authored and
arthw
committed
cmake : fix ppc64 check (whisper/0)

ggerganov
authored and
arthw
committed
ggml : fix some build issues

slaren
authored and
arthw
committed
scripts: update compare-llama-bench.py (#10319)

JohannesGaessler
authored and
arthw
committed
Make updates to fix issues with clang-cl builds while using AVX512 flags (#10314)

Srihari-mcw
authored and
arthw
committed
llama : save number of parameters and the size in llama_model (#10286)

FirstTimeEZ
authored and
arthw
committed
ggml : optimize Q4_0 into Q4_0_X_Y repack (#10324)

eddnjjn
authored and
arthw
committed
vulkan : add cmake preset debug/release (#10306)

FirstTimeEZ
authored and
arthw
committed
vulkan: Optimize some mat-vec mul quant shaders (#10296)

jeffbolznv
authored and
arthw
committed
scripts : fix missing key in compare-llama-bench.py (#10332)

ggerganov
authored and
arthw
committed
server: (web UI) Add samplers sequence customization (#10255)

authored and
arthw
committed
make : auto-determine dependencies (#0)

ggerganov
authored and
arthw
committed
llamafile : fix include path (#0)

ggerganov
authored and
arthw
committed
llama/ex: remove --logdir argument (#10339)

JohannesGaessler
authored and
arthw
committed
docs : vulkan build instructions to use git bash mingw64 (#10303)

FirstTimeEZ
authored and
arthw
committed
scripts : update sync

ggerganov
authored and
arthw
committed
ggml: new optimization interface (ggml/988)

JohannesGaessler
authored and
arthw
committed
ggml : fix compile warnings (#0)

ggerganov
authored and
arthw
committed
tests : remove test-grad0

ggerganov
authored and
arthw
committed
make : add ggml-opt (#0)

ggerganov
authored and
arthw
committed
ggml : adapt AMX to tensor->grad removal (#0)

ggerganov
authored and
arthw
committed
ggml : inttypes.h -> cinttypes (#0)

ggerganov
authored and
arthw
committed
ggml : fix possible buffer use after free in sched reserve (#9930)

slaren
authored and
arthw
committed
CMake: default to -arch=native for CUDA build (#10320)

JohannesGaessler
authored and
arthw
committed
CUDA: remove DMMV, consolidate F16 mult mat vec (#10318)

JohannesGaessler
authored and
arthw
committed
ggml : fix undefined reference to 'getcpu' (#10354)

FirstTimeEZ
authored and
arthw
committed
metal : refactor kernel args into structs (#10238)

ggerganov
authored and
arthw
committed
gitignore : ignore local run scripts [no ci]

ggerganov
authored and
arthw
committed
llama : only use default buffer types for the KV cache (#10358)

slaren
authored and
arthw
committed
CMake: fix typo in comment [no ci] (#10360)

JohannesGaessler
authored and
arthw
committed
CUDA: fix MMV kernel being used for FP16 src1 (#10357)

JohannesGaessler
authored and
arthw
committed
docker: use GGML_NATIVE=OFF (#10368)

JohannesGaessler
authored and
arthw
committed

Commits on Nov 19, 2024

fix for windows building
arthw
committed

Cherry 1118#5arthw merged 581 commits intomasterarthw/llama.cpp:masterfrom cherry-1118arthw/llama.cpp:cherry-1118Copy head branch name to clipboard

Commits

Commits on Nov 18, 2024

Cherry 1118#5
arthw merged 581 commits intomasterfrom
cherry-1118