Arm AArch64: optimized GEMV and GEMM kernels for q4_0_q8_0, and q8_0_q8_0 quantization by Dibakar · Pull Request #5780 · ggml-org/llama.cpp

Arm AArch64: optimized GEMV and GEMM kernels for q4_0_q8_0, and q8_0_q8_0 quantization

Dibakar

committed

Arm AArch64: add optimized GEMV and GEMM asm kernels for q4_0_q8_0 quantization and refactor code to address llama.cpp pr#5780 suggestions

Dibakar

committed

Arm AArch64: add optimized GEMV and GEMM asm kernels for q4_0_q8_0 quantization and refactor code to address llama.cpp pr#5780 suggestions

Dibakar

committed

Arm AArch64: add optimized GEMV and GEMM asm kernels for q4_0_q8_0 quantization and refactor code to address llama.cpp pr#5780 suggestions

Dibakar

committed

Arm AArch64: add optimized GEMV and GEMM asm kernels for q4_0_q8_0 quantization and refactor code to address llama.cpp pr#5780 suggestions

Dibakar

committed

Arm AArch64: add copyright claim only to ggml-aarch64.cpp and ggml-aarch64.h files

Dibakar

committed

Arm AArch64: minor code refactoring for rebase

Dibakar

committed

Arm AArch64: minor code refactoring for resolving a build issue with cmake

Dibakar

committed

Arm AArch64: minor code refactoring to split the Q4_0_AARC64 type into three separate types: Q4_0_4_4, Q4_0_4_8, and Q4_0_8_8

Dibakar

committed

Arm AArch64: minor code change for resolving a build issue with server-windows

Dibakar

committed

retrigger checks

Dibakar

committed

Arm AArch64: minor code changes for rebase

Dibakar

committed

Arm AArch64: minor changes to skip the pr#7433 vec_dot code for arm cpus with SVE VL not equal to 256 bits

Dibakar

committed

Arm AArch64: remove stale LLAMA_QKK_64 from CMakeLists.txt and delete build.zig

Dibakar

committed

Arm AArch64: add reference scalar gemm and gemv, and avoid dynamic memory allocations during quantization for Q4_0_4_4, Q4_0_4_8, and Q4_0_8_8

Dibakar

committed

Arm AArch64: add multithreaded quantization support for the new types: Q4_0_4_4, Q4_0_4_8, and Q4_0_8_8

Dibakar

committed

Arm AArch64: minor code refactoring

Dibakar

committed

Arm AArch64: simplify logic for calling gemm and gemv functions in ggml_compute_forward_mul_mat

Dibakar

committed

Arm AArch64: minimize changes in ggml_compute_forward_mul_mat

Dibakar

committed

Arm AArch64: minor code refactoring, and add reference scalar code to quantize routines for new quant types

Dibakar

committed

Arm AArch64: minor code refactoring

Dibakar

committed

Arm AArch64: minor code refactoring

Dibakar

committed

Arm AArch64: minor code refactoring

Dibakar

committed

rebase on the latest master commit 3fd62a6 and adapt to the new directory structure

Dibakar

committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Arm AArch64: optimized GEMV and GEMM kernels for q4_0_q8_0, and q8_0_q8_0 quantization#5780

Arm AArch64: optimized GEMV and GEMM kernels for q4_0_q8_0, and q8_0_q8_0 quantization#5780
ggerganov merged 28 commits intoggml-org:masterfrom
Dibakar:aarch64-kernel-opts

Commits on Jul 8, 2024

Arm AArch64: optimized GEMV and GEMM kernels for q4_0_q8_0, and q8_0_q8_0 quantization

Arm AArch64: add optimized GEMV and GEMM asm kernels for q4_0_q8_0 quantization and refactor code to address llama.cpp pr#5780 suggestions

Arm AArch64: add optimized GEMV and GEMM asm kernels for q4_0_q8_0 quantization and refactor code to address llama.cpp pr#5780 suggestions

Arm AArch64: add optimized GEMV and GEMM asm kernels for q4_0_q8_0 quantization and refactor code to address llama.cpp pr#5780 suggestions

Arm AArch64: add optimized GEMV and GEMM asm kernels for q4_0_q8_0 quantization and refactor code to address llama.cpp pr#5780 suggestions

Arm AArch64: add copyright claim only to ggml-aarch64.cpp and ggml-aarch64.h files

Arm AArch64: minor code refactoring for rebase

Arm AArch64: minor code refactoring for resolving a build issue with cmake

Arm AArch64: minor code refactoring to split the Q4_0_AARC64 type into three separate types: Q4_0_4_4, Q4_0_4_8, and Q4_0_8_8

Arm AArch64: minor code change for resolving a build issue with server-windows

retrigger checks

Arm AArch64: minor code changes for rebase

Arm AArch64: minor changes to skip the pr#7433 vec_dot code for arm cpus with SVE VL not equal to 256 bits

Arm AArch64: remove stale LLAMA_QKK_64 from CMakeLists.txt and delete build.zig

Arm AArch64: add reference scalar gemm and gemv, and avoid dynamic memory allocations during quantization for Q4_0_4_4, Q4_0_4_8, and Q4_0_8_8

Arm AArch64: add multithreaded quantization support for the new types: Q4_0_4_4, Q4_0_4_8, and Q4_0_8_8

Arm AArch64: minor code refactoring

Arm AArch64: simplify logic for calling gemm and gemv functions in ggml_compute_forward_mul_mat

Arm AArch64: minimize changes in ggml_compute_forward_mul_mat

Arm AArch64: minor code refactoring, and add reference scalar code to quantize routines for new quant types

Arm AArch64: minor code refactoring

Arm AArch64: minor code refactoring

Arm AArch64: minor code refactoring

rebase on the latest master commit 3fd62a6 and adapt to the new directory structure

Commits on Jul 9, 2024

Arm AArch64: remove a redundant comment

Arm AArch64: add pragma in ggml-aarch64.c to turn -Woverlength-strings warning off

Arm AArch64: use aarch64 check to guard 64-bit neon kernels

Arm AArch64: update docs/build.md README to include compile time flags for buiilding the Q4_0_4_4 quant type

Arm AArch64: optimized GEMV and GEMM kernels for q4_0_q8_0, and q8_0_q8_0 quantization#5780ggerganov merged 28 commits intoggml-org:masterggml-org/llama.cpp:masterfrom Dibakar:aarch64-kernel-optsDibakar/llama.cpp:aarch64-kernel-optsCopy head branch name to clipboard

Commits

Commits on Jul 8, 2024

Commits on Jul 9, 2024

Arm AArch64: optimized GEMV and GEMM kernels for q4_0_q8_0, and q8_0_q8_0 quantization#5780
ggerganov merged 28 commits intoggml-org:masterfrom
Dibakar:aarch64-kernel-opts