Skip to content

Conversation

@yeahdongcn
Copy link
Collaborator

Make sure to read the contributing guidelines before submitting a PR

This PR is a follow-up based on comments from @JohannesGaessler :

I think it would be better to have a more specific name. (The table for llama-bench is just named "test", that could also be improved upon.)

1

Testing Done

root@ff759de5dd90:/ws# ./scripts/compare-commits.sh e1c2f3ea0 737f78c34 llama-bench -m /models/DeepSeek-R1-Distill-Qwen-7B-Q4_K_M.gguf
 -ngl 999
+ commit1=e1c2f3ea0
+ commit2=737f78c34
+ tool=llama-bench
+ additional_args='-m /models/DeepSeek-R1-Distill-Qwen-7B-Q4_K_M.gguf -ngl 999'
+ '[' llama-bench '!=' llama-bench ']'
+ ./scripts/compare-llama-bench.py --check
+ '[' llama-bench = llama-bench ']'
+ db_file=llama-bench.sqlite
+ target=llama-bench
+ run_args='-o sql -oe md -m /models/DeepSeek-R1-Distill-Qwen-7B-Q4_K_M.gguf -ngl 999'
+ rm -f llama-bench.sqlite
+ '[' -n '' ']'
+ dir=build-bench
+ git checkout e1c2f3ea0
Previous HEAD position was 737f78c34 llama-bench: rename DB table name from test to llama_bench
HEAD is now at e1c2f3ea0 1
+ run
+ rm -fr build-bench
+ cmake -B build-bench -S .
++ nproc
+ cmake --build build-bench -t llama-bench -j 12
+ build-bench/bin/llama-bench -o sql -oe md -m /models/DeepSeek-R1-Distill-Qwen-7B-Q4_K_M.gguf -ngl 999
+ sqlite3 llama-bench.sqlite
| model                          |       size |     params | backend    | threads |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | --------------: | -------------------: |
| qwen2 7B Q4_K - Medium         |   4.36 GiB |     7.62 B | CPU        |       6 |           pp512 |         41.91 ± 1.66 |
| qwen2 7B Q4_K - Medium         |   4.36 GiB |     7.62 B | CPU        |       6 |           tg128 |         10.37 ± 0.20 |

build: e1c2f3ea0 (6055)
+ git checkout 737f78c34
Previous HEAD position was e1c2f3ea0 1
HEAD is now at 737f78c34 llama-bench: rename DB table name from test to llama_bench
+ run
+ rm -fr build-bench
+ cmake -B build-bench -S .
++ nproc
+ cmake --build build-bench -t llama-bench -j 12
+ build-bench/bin/llama-bench -o sql -oe md -m /models/DeepSeek-R1-Distill-Qwen-7B-Q4_K_M.gguf -ngl 999
+ sqlite3 llama-bench.sqlite
| model                          |       size |     params | backend    | threads |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | --------------: | -------------------: |
| qwen2 7B Q4_K - Medium         |   4.36 GiB |     7.62 B | CPU        |       6 |           pp512 |         41.45 ± 1.78 |
| qwen2 7B Q4_K - Medium         |   4.36 GiB |     7.62 B | CPU        |       6 |           tg128 |         10.29 ± 0.15 |

build: 737f78c34 (6054)
+ ./scripts/compare-llama-bench.py -b e1c2f3ea0 -c 737f78c34 --tool llama-bench -i llama-bench.sqlite
| CPU                                 | Model           | Test   |   t/s xd/llama_bench |   t/s 737f78c34 |   Speedup |
|:------------------------------------|:----------------|:-------|---------------------:|----------------:|----------:|
| 12th Gen Intel(R) Core(TM) i5-12400 | qwen2 7B Q4_K_M | pp512  |                41.91 |           41.45 |      0.99 |
| 12th Gen Intel(R) Core(TM) i5-12400 | qwen2 7B Q4_K_M | tg128  |                10.37 |           10.29 |      0.99 |
root@ff759de5dd90:/ws#  

@github-actions github-actions bot added script Script related examples python python script changes labels Aug 1, 2025
@yeahdongcn yeahdongcn merged commit 3025b62 into ggml-org:master Aug 2, 2025
49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

examples python python script changes script Script related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants