llama-bench : skip repeated values in consecutive lines #10140

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Closed

slaren wants to merge 1 commit into master from sl/llama-bench-headers

Member

slaren commented Nov 2, 2024

The goal is to improve readability of the markdown output by skipping repeated column values. Not sure if it is actually better.

For example:

./llama-bench -fa 0,1 -p 128,512 -m models/7B/ggml-model-f16.gguf -m models/7B/ggml-model-Q4_0.gguf

model	size	params	backend	ngl	fa	test	t/s
llama 7B F16	12.55 GiB	6.74 B	CUDA	99	0	pp128	4258.54 ± 11.35
						pp512	5613.05 ± 15.36
						tg128	60.14 ± 0.04
					1	pp128	4376.25 ± 7.00
						pp512	6078.50 ± 11.93
						tg128	61.07 ± 0.35
llama 7B Q4_0	3.56 GiB				0	pp128	4925.08 ± 13.67
						pp512	5894.38 ± 5.63
						tg128	161.10 ± 0.30
					1	pp128	5076.93 ± 6.05
						pp512	6422.28 ± 5.30
						tg128	167.63 ± 0.55


          llama-bench : skip repeated values in consecutive lines

4fc8673

github-actions bot added the examples label

slaren closed this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels