Skip to content

ggml: new, 0.9.3 #12741

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Sep 29, 2025
Merged

ggml: new, 0.9.3 #12741

merged 3 commits into from
Sep 29, 2025

Conversation

@HmnSn
Copy link
Contributor

@HmnSn HmnSn commented Sep 27, 2025 •
edited by aosc-buildit-bot
Loading

Topic Description

  • whisper.cpp: use system ggml
  • llama.cpp: update to 6615
    • Use system ggml.
    • Drop unnecessary patch.
  • ggml: new, 0.9.3

Package(s) Affected

  • ggml: 0.9.3
  • llama.cpp: 6615
  • whisper.cpp: 1.7.6-1

Security Update?

No

Build Order

#buildit ggml llama.cpp whisper.cpp

Test Build(s) Done

Primary Architectures

  • AMD64 amd64
  • AArch64 arm64
  • LoongArch 64-bit loongarch64
  • LoongArch 64-bit (No SIMD) loongarch64_nosimd

Secondary Architectures

  • Loongson 3 loongson3
  • PowerPC 64-bit (Little Endian) ppc64el
  • RISC-V 64-bit riscv64

@HmnSn HmnSn added the new-package New AOSC OS package label Sep 27, 2025
@HmnSn HmnSn force-pushed the ggml-new branch 2 times, most recently from 53dd269 to a016916 Compare September 27, 2025 23:08
@aosc-buildit-bot

This comment was marked as outdated.

Sign in to view
@HmnSn HmnSn force-pushed the ggml-new branch 7 times, most recently from 2796455 to 02818bd Compare September 28, 2025 16:17
@HmnSn
Copy link
Contributor Author

HmnSn commented Sep 29, 2025 •
edited
Loading

Blocked by LSX backend is broken

-DGGML_LSX=ON -DGGML_LASX=OFF

./test-quantize-fns
Testing f32
Testing f16
  f16 dot product error:              FAILED (282806.906250)
Testing q4_0
 q4_0 dot product error:              FAILED (1.090274)
Testing q4_1
Testing q5_0
Testing q5_1
Testing q8_0
Testing q8_1
Testing q2_K
Testing q3_K
Testing q4_K
Testing q5_K
Testing q6_K
Testing q8_K
Testing iq2_xxs
Testing iq2_xs
Testing iq3_xxs
Testing iq1_s
Testing iq4_nl
Testing iq3_s
Testing iq2_s
Testing iq4_xs
Testing i8
Testing i16
Testing i32
Testing i64
Testing f64
Testing iq1_m
Testing bf16
Testing tq1_0
Testing tq2_0
Testing mxfp4
2 tests failed

-DGGML_LASX=ON

./test-quantize-fns
Testing f32
Testing f16
Testing q4_0
Testing q4_1
Testing q5_0
Testing q5_1
Testing q8_0
Testing q8_1
Testing q2_K
Testing q3_K
Testing q4_K
Testing q5_K
Testing q6_K
Testing q8_K
Testing iq2_xxs
Testing iq2_xs
Testing iq3_xxs
Testing iq1_s
Testing iq4_nl
Testing iq3_s
Testing iq2_s
Testing iq4_xs
Testing i8
Testing i16
Testing i32
Testing i64
Testing f64
Testing iq1_m
Testing bf16
Testing tq1_0
Testing tq2_0
Testing mxfp4

@MingcongBai MingcongBai marked this pull request as ready for review September 29, 2025 02:41
@HmnSn
Copy link
Contributor Author

HmnSn commented Sep 29, 2025

Tested on loongarhc64 with dg1

whisper.cpp
image

llama.cpp
cpu
image
gpu
image

@aosc-buildit-bot
Copy link
Contributor

aosc-buildit-bot commented Sep 29, 2025

Dickens-topic report:

ggml introduced at 0.9.3~pre20250929T025257Z on amd64:

43 added, 0 removed, size +9.38 MB 
--- a
+++ b
@@ -0,0 +1,43 @@
+drwxr-xr-x     ./
+drwxr-xr-x     ./usr/
+drwxr-xr-x     ./usr/include/
+-rw-r--r--     ./usr/include/ggml-alloc.h
+-rw-r--r--     ./usr/include/ggml-backend.h
+-rw-r--r--     ./usr/include/ggml-blas.h
+-rw-r--r--     ./usr/include/ggml-cann.h
+-rw-r--r--     ./usr/include/ggml-cpp.h
+-rw-r--r--     ./usr/include/ggml-cpu.h
+-rw-r--r--     ./usr/include/ggml-cuda.h
+-rw-r--r--     ./usr/include/ggml-metal.h
+-rw-r--r--     ./usr/include/ggml-opt.h
+-rw-r--r--     ./usr/include/ggml-rpc.h
+-rw-r--r--     ./usr/include/ggml-sycl.h
+-rw-r--r--     ./usr/include/ggml-vulkan.h
+-rw-r--r--     ./usr/include/ggml-webgpu.h
+-rw-r--r--     ./usr/include/ggml.h
+-rw-r--r--     ./usr/include/gguf.h
+drwxr-xr-x     ./usr/lib/
+drwxr-xr-x     ./usr/lib/cmake/
+drwxr-xr-x     ./usr/lib/cmake/ggml/
+-rw-r--r--     ./usr/lib/cmake/ggml/ggml-config.cmake
+-rw-r--r--     ./usr/lib/cmake/ggml/ggml-version.cmake
+drwxr-xr-x     ./usr/lib/ggml/
+-rwxr-xr-x     ./usr/lib/ggml/libggml-blas.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-alderlake.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-haswell.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-icelake.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-sandybridge.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-sapphirerapids.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-skylakex.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-sse42.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-x64.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-opencl.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-vulkan.so
+-rwxr-xr-x     ./usr/lib/libggml-base.so
+-rwxr-xr-x     ./usr/lib/libggml.so
+drwxr-xr-x     ./usr/share/
+drwxr-xr-x     ./usr/share/doc/
+drwxr-xr-x     ./usr/share/doc/ggml/
+-rw-r--r--     ./usr/share/doc/ggml/LICENSE
+drwxr-xr-x     ./usr/share/pkgconfig/
+-rw-r--r--     ./usr/share/pkgconfig/ggml.pc
ggml introduced at 0.9.3~pre20250929T025257Z on arm64:
43 added, 0 removed, size +8.91 MB 
--- a
+++ b
@@ -0,0 +1,43 @@
+drwxr-xr-x     ./
+drwxr-xr-x     ./usr/
+drwxr-xr-x     ./usr/include/
+-rw-r--r--     ./usr/include/ggml-alloc.h
+-rw-r--r--     ./usr/include/ggml-backend.h
+-rw-r--r--     ./usr/include/ggml-blas.h
+-rw-r--r--     ./usr/include/ggml-cann.h
+-rw-r--r--     ./usr/include/ggml-cpp.h
+-rw-r--r--     ./usr/include/ggml-cpu.h
+-rw-r--r--     ./usr/include/ggml-cuda.h
+-rw-r--r--     ./usr/include/ggml-metal.h
+-rw-r--r--     ./usr/include/ggml-opt.h
+-rw-r--r--     ./usr/include/ggml-rpc.h
+-rw-r--r--     ./usr/include/ggml-sycl.h
+-rw-r--r--     ./usr/include/ggml-vulkan.h
+-rw-r--r--     ./usr/include/ggml-webgpu.h
+-rw-r--r--     ./usr/include/ggml.h
+-rw-r--r--     ./usr/include/gguf.h
+drwxr-xr-x     ./usr/lib/
+drwxr-xr-x     ./usr/lib/cmake/
+drwxr-xr-x     ./usr/lib/cmake/ggml/
+-rw-r--r--     ./usr/lib/cmake/ggml/ggml-config.cmake
+-rw-r--r--     ./usr/lib/cmake/ggml/ggml-version.cmake
+drwxr-xr-x     ./usr/lib/ggml/
+-rwxr-xr-x     ./usr/lib/ggml/libggml-blas.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-armv8.0_1.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-armv8.2_1.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-armv8.2_2.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-armv8.2_3.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-armv8.6_1.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-armv8.6_2.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-armv9.2_1.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-armv9.2_2.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-opencl.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-vulkan.so
+-rwxr-xr-x     ./usr/lib/libggml-base.so
+-rwxr-xr-x     ./usr/lib/libggml.so
+drwxr-xr-x     ./usr/share/
+drwxr-xr-x     ./usr/share/doc/
+drwxr-xr-x     ./usr/share/doc/ggml/
+-rw-r--r--     ./usr/share/doc/ggml/LICENSE
+drwxr-xr-x     ./usr/share/pkgconfig/
+-rw-r--r--     ./usr/share/pkgconfig/ggml.pc
ggml introduced at 0.9.3~pre20250929T025257Z on loongarch64, loongson3, riscv64:
36 added, 0 removed, size +25.5 MB 
--- a
+++ b
@@ -0,0 +1,36 @@
+drwxr-xr-x     ./
+drwxr-xr-x     ./usr/
+drwxr-xr-x     ./usr/include/
+-rw-r--r--     ./usr/include/ggml-alloc.h
+-rw-r--r--     ./usr/include/ggml-backend.h
+-rw-r--r--     ./usr/include/ggml-blas.h
+-rw-r--r--     ./usr/include/ggml-cann.h
+-rw-r--r--     ./usr/include/ggml-cpp.h
+-rw-r--r--     ./usr/include/ggml-cpu.h
+-rw-r--r--     ./usr/include/ggml-cuda.h
+-rw-r--r--     ./usr/include/ggml-metal.h
+-rw-r--r--     ./usr/include/ggml-opt.h
+-rw-r--r--     ./usr/include/ggml-rpc.h
+-rw-r--r--     ./usr/include/ggml-sycl.h
+-rw-r--r--     ./usr/include/ggml-vulkan.h
+-rw-r--r--     ./usr/include/ggml-webgpu.h
+-rw-r--r--     ./usr/include/ggml.h
+-rw-r--r--     ./usr/include/gguf.h
+drwxr-xr-x     ./usr/lib/
+drwxr-xr-x     ./usr/lib/cmake/
+drwxr-xr-x     ./usr/lib/cmake/ggml/
+-rw-r--r--     ./usr/lib/cmake/ggml/ggml-config.cmake
+-rw-r--r--     ./usr/lib/cmake/ggml/ggml-version.cmake
+drwxr-xr-x     ./usr/lib/ggml/
+-rwxr-xr-x     ./usr/lib/ggml/libggml-blas.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-opencl.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-vulkan.so
+-rwxr-xr-x     ./usr/lib/libggml-base.so
+-rwxr-xr-x     ./usr/lib/libggml.so
+drwxr-xr-x     ./usr/share/
+drwxr-xr-x     ./usr/share/doc/
+drwxr-xr-x     ./usr/share/doc/ggml/
+-rw-r--r--     ./usr/share/doc/ggml/LICENSE
+drwxr-xr-x     ./usr/share/pkgconfig/
+-rw-r--r--     ./usr/share/pkgconfig/ggml.pc
ggml introduced at 0.9.3~pre20250929T025257Z on ppc64el:
43 added, 0 removed, size +9.54 MB 
--- a
+++ b
@@ -0,0 +1,43 @@
+drwxr-xr-x     ./
+drwxr-xr-x     ./usr/
+drwxr-xr-x     ./usr/include/
+-rw-r--r--     ./usr/include/ggml-alloc.h
+-rw-r--r--     ./usr/include/ggml-backend.h
+-rw-r--r--     ./usr/include/ggml-blas.h
+-rw-r--r--     ./usr/include/ggml-cann.h
+-rw-r--r--     ./usr/include/ggml-cpp.h
+-rw-r--r--     ./usr/include/ggml-cpu.h
+-rw-r--r--     ./usr/include/ggml-cuda.h
+-rw-r--r--     ./usr/include/ggml-metal.h
+-rw-r--r--     ./usr/include/ggml-opt.h
+-rw-r--r--     ./usr/include/ggml-rpc.h
+-rw-r--r--     ./usr/include/ggml-sycl.h
+-rw-r--r--     ./usr/include/ggml-vulkan.h
+-rw-r--r--     ./usr/include/ggml-webgpu.h
+-rw-r--r--     ./usr/include/ggml.h
+-rw-r--r--     ./usr/include/gguf.h
+drwxr-xr-x     ./usr/lib/
+drwxr-xr-x     ./usr/lib/cmake/
+drwxr-xr-x     ./usr/lib/cmake/ggml/
+-rw-r--r--     ./usr/lib/cmake/ggml/ggml-config.cmake
+-rw-r--r--     ./usr/lib/cmake/ggml/ggml-version.cmake
+drwxr-xr-x     ./usr/lib/ggml/
+-rwxr-xr-x     ./usr/lib/ggml/libggml-blas.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-power0.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-power10.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-power11.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-power7_1.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-power7_2.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-power8_1.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-power8_2.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-cpu-power9.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-opencl.so
+-rwxr-xr-x     ./usr/lib/ggml/libggml-vulkan.so
+-rwxr-xr-x     ./usr/lib/libggml-base.so
+-rwxr-xr-x     ./usr/lib/libggml.so
+drwxr-xr-x     ./usr/share/
+drwxr-xr-x     ./usr/share/doc/
+drwxr-xr-x     ./usr/share/doc/ggml/
+-rw-r--r--     ./usr/share/doc/ggml/LICENSE
+drwxr-xr-x     ./usr/share/pkgconfig/
+-rw-r--r--     ./usr/share/pkgconfig/ggml.pc
llama.cpp upgraded from 6265 to 6615~pre20250929T025257Z on amd64, arm64, loongarch64, loongson3, ppc64el, riscv64:
7 added, 4 removed, size -1.13 GB (-93.7%) 
--- a
+++ b
@@ -5,18 +5,14 @@
 -rwxr-xr-x     ./usr/bin/llama-batched-bench
 -rwxr-xr-x     ./usr/bin/llama-bench
 -rwxr-xr-x     ./usr/bin/llama-cli
--rwxr-xr-x     ./usr/bin/llama-convert-llama2c-to-ggml
--rwxr-xr-x     ./usr/bin/llama-cvector-generator
 -rwxr-xr-x     ./usr/bin/llama-diffusion-cli
 -rwxr-xr-x     ./usr/bin/llama-embedding
 -rwxr-xr-x     ./usr/bin/llama-eval-callback
--rwxr-xr-x     ./usr/bin/llama-export-lora
 -rwxr-xr-x     ./usr/bin/llama-finetune
 -rwxr-xr-x     ./usr/bin/llama-gen-docs
 -rwxr-xr-x     ./usr/bin/llama-gguf
 -rwxr-xr-x     ./usr/bin/llama-gguf-hash
 -rwxr-xr-x     ./usr/bin/llama-gguf-split
--rwxr-xr-x     ./usr/bin/llama-gritlm
 -rwxr-xr-x     ./usr/bin/llama-imatrix
 -rwxr-xr-x     ./usr/bin/llama-logits
 -rwxr-xr-x     ./usr/bin/llama-lookahead
@@ -39,11 +35,18 @@
 -rwxr-xr-x     ./usr/bin/llama-speculative-simple
 -rwxr-xr-x     ./usr/bin/llama-tokenize
 -rwxr-xr-x     ./usr/bin/llama-tts
+drwxr-xr-x     ./usr/include/
+-rw-r--r--     ./usr/include/llama-cpp.h
+-rw-r--r--     ./usr/include/llama.h
+-rw-r--r--     ./usr/include/mtmd-helper.h
+-rw-r--r--     ./usr/include/mtmd.h
 drwxr-xr-x     ./usr/lib/
 drwxr-xr-x     ./usr/lib/cmake/
 drwxr-xr-x     ./usr/lib/cmake/llama/
 -rw-r--r--     ./usr/lib/cmake/llama/llama-config.cmake
 -rw-r--r--     ./usr/lib/cmake/llama/llama-version.cmake
+-rwxr-xr-x     ./usr/lib/libllama.so
+-rwxr-xr-x     ./usr/lib/libmtmd.so
 drwxr-xr-x     ./usr/lib/pkgconfig/
 -rw-r--r--     ./usr/lib/pkgconfig/llama.pc
 drwxr-xr-x     ./usr/share/
whisper.cpp upgraded from 1.7.6 to 1.7.6-1~pre20250929T025257Z on amd64, arm64, loongarch64, loongson3, ppc64el, riscv64:
0 added, 22 removed, size -25.7 MB (-86.6%) 
--- a
+++ b
@@ -6,34 +6,12 @@
 -rwxr-xr-x     ./usr/bin/whisper-cli
 -rwxr-xr-x     ./usr/bin/whisper-server
 drwxr-xr-x     ./usr/include/
--rw-r--r--     ./usr/include/ggml-alloc.h
--rw-r--r--     ./usr/include/ggml-backend.h
--rw-r--r--     ./usr/include/ggml-blas.h
--rw-r--r--     ./usr/include/ggml-cann.h
--rw-r--r--     ./usr/include/ggml-cpp.h
--rw-r--r--     ./usr/include/ggml-cpu.h
--rw-r--r--     ./usr/include/ggml-cuda.h
--rw-r--r--     ./usr/include/ggml-kompute.h
--rw-r--r--     ./usr/include/ggml-metal.h
--rw-r--r--     ./usr/include/ggml-opt.h
--rw-r--r--     ./usr/include/ggml-rpc.h
--rw-r--r--     ./usr/include/ggml-sycl.h
--rw-r--r--     ./usr/include/ggml-vulkan.h
--rw-r--r--     ./usr/include/ggml.h
--rw-r--r--     ./usr/include/gguf.h
 -rw-r--r--     ./usr/include/whisper.h
 drwxr-xr-x     ./usr/lib/
 drwxr-xr-x     ./usr/lib/cmake/
-drwxr-xr-x     ./usr/lib/cmake/ggml/
--rw-r--r--     ./usr/lib/cmake/ggml/ggml-config.cmake
--rw-r--r--     ./usr/lib/cmake/ggml/ggml-version.cmake
 drwxr-xr-x     ./usr/lib/cmake/whisper/
 -rw-r--r--     ./usr/lib/cmake/whisper/whisper-config.cmake
 -rw-r--r--     ./usr/lib/cmake/whisper/whisper-version.cmake
--rwxr-xr-x     ./usr/lib/libggml-base.so
--rwxr-xr-x     ./usr/lib/libggml-cpu.so
--rwxr-xr-x     ./usr/lib/libggml-vulkan.so
--rwxr-xr-x     ./usr/lib/libggml.so
 -rwxr-xr-x     ./usr/lib/libwhisper.so.1.7.6
 drwxr-xr-x     ./usr/lib/pkgconfig/
 -rw-r--r--     ./usr/lib/pkgconfig/whisper.pc

@MingcongBai
Copy link
Member

MingcongBai commented Sep 29, 2025

Blocked by LSX backend is broken

-DGGML_LSX=ON -DGGML_LASX=OFF

./test-quantize-fns
Testing f32
Testing f16
  f16 dot product error:              FAILED (282806.906250)
Testing q4_0
 q4_0 dot product error:              FAILED (1.090274)
Testing q4_1
Testing q5_0
Testing q5_1
Testing q8_0
Testing q8_1
Testing q2_K
Testing q3_K
Testing q4_K
Testing q5_K
Testing q6_K
Testing q8_K
Testing iq2_xxs
Testing iq2_xs
Testing iq3_xxs
Testing iq1_s
Testing iq4_nl
Testing iq3_s
Testing iq2_s
Testing iq4_xs
Testing i8
Testing i16
Testing i32
Testing i64
Testing f64
Testing iq1_m
Testing bf16
Testing tq1_0
Testing tq2_0
Testing mxfp4
2 tests failed

-DGGML_LASX=ON

./test-quantize-fns
Testing f32
Testing f16
Testing q4_0
Testing q4_1
Testing q5_0
Testing q5_1
Testing q8_0
Testing q8_1
Testing q2_K
Testing q3_K
Testing q4_K
Testing q5_K
Testing q6_K
Testing q8_K
Testing iq2_xxs
Testing iq2_xs
Testing iq3_xxs
Testing iq1_s
Testing iq4_nl
Testing iq3_s
Testing iq2_s
Testing iq4_xs
Testing i8
Testing i16
Testing i32
Testing i64
Testing f64
Testing iq1_m
Testing bf16
Testing tq1_0
Testing tq2_0
Testing mxfp4

Since we have been using an LASX baseline and that LSX is broken, let's just keep it there until we find a fix for ggml-org/llama.cpp#15854.

@MingcongBai MingcongBai merged commit bc3645b into stable Sep 29, 2025
7 checks passed
@MingcongBai MingcongBai deleted the ggml-new branch September 29, 2025 06:21
@MingcongBai MingcongBai mentioned this pull request Oct 10, 2025
Closed
6 tasks
@MQ-mengqing
Copy link

MQ-mengqing commented Nov 3, 2025

ggml-org/llama.cpp#16958

@MingcongBai
Copy link
Member

MingcongBai commented Nov 3, 2025

ggml-org/llama.cpp#16958

感谢！

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@MingcongBai MingcongBai MingcongBai approved these changes

Assignees

No one assigned

Labels

new-package New AOSC OS package

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@HmnSn @aosc-buildit-bot @MingcongBai @MQ-mengqing