Skip to content

Conversation

@chaxu01
Copy link
Collaborator

@chaxu01 chaxu01 commented Sep 25, 2025

This patch also updates the kleidiai lib to v1.14.0.

@github-actions github-actions bot added the ggml changes relating to the ggml tensor library for machine learning label Sep 25, 2025
@chaxu01
Copy link
Collaborator Author

chaxu01 commented Sep 29, 2025

This PR addresses FP16 work-size and thread synchronization in the KleidiAI backend and also updates the KleidiAI library to v1.14.0. The changes only affect the KleidiAI path, which is gated behind GGML_CPU_KLEIDIAI and not enabled in the default CI builds. The two CI failures shown here are unrelated, as neither job includes KleidiAI in the build. The arm64 SVE job failed due to a rerank score outside the expected range, and the x64 AMX job failed with a ppl value above threshold along with a missing log file. I believe these are independent of this patch.

Copy link
Member

@ggerganov ggerganov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be useful to have a CI workflow that exercises the KleidiAI builds. Please consider adding one.

A starting point could be the new SVE workflow:

ggml-ci-arm64-cpu-high-perf-sve:
runs-on: ubuntu-22.04-arm
steps:
- name: Clone
id: checkout
uses: actions/checkout@v4
- name: ccache
uses: ggml-org/[email protected]
with:
key: ggml-ci-arm64-cpu-high-perf-sve
evict-old-files: 1d
- name: Dependencies
id: depends
run: |
sudo apt-get update
sudo apt-get install build-essential libcurl4-openssl-dev
- name: Test
id: ggml-ci
run: |
LLAMA_ARG_THREADS=$(nproc) GG_BUILD_NO_BF16=1 GG_BUILD_EXTRA_TESTS_0=1 bash ./ci/run.sh ./tmp/results ./tmp/mnt

@chaxu01
Copy link
Collaborator Author

chaxu01 commented Sep 30, 2025

Thanks, @ggerganov — good suggestion. We’re definitely interested in adding CI coverage for KleidiAI builds. We’ll explore how best to integrate KleidiAI into the CI without adding too much overhead, and follow up with a separate PR to cover this.

@ggerganov ggerganov merged commit f1eb1cb into ggml-org:master Sep 30, 2025
64 of 66 checks passed
@chaxu01 chaxu01 deleted the feature/kleidiai-fp16-worksize branch October 7, 2025 14:36
yael-works pushed a commit to yael-works/llama.cpp that referenced this pull request Oct 15, 2025
pwilkin pushed a commit to pwilkin/llama.cpp that referenced this pull request Oct 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants