llama : print size and type of overridden tensors

[slaren]

Small QoL improvement. Example output:

tensor blk.10.exp_probs_b.bias (0 MiB f32) buffer type overridden to CPU
tensor blk.10.ffn_gate_exps.weight (700 MiB iq1_s) buffer type overridden to CPU
tensor blk.10.ffn_down_exps.weight (700 MiB iq1_s) buffer type overridden to CPU
tensor blk.10.ffn_up_exps.weight (700 MiB iq1_s) buffer type overridden to CPU
tensor blk.10.ffn_gate_shexp.weight (9 MiB q5_K) buffer type overridden to CPU
tensor blk.10.ffn_down_shexp.weight (11 MiB q6_K) buffer type overridden to CPU
tensor blk.10.ffn_up_shexp.weight (9 MiB q5_K) buffer type overridden to CPU

[Copilot]

Pull Request Overview

This PR enhances the debugging output for overridden tensor buffer types by including the tensor’s memory size (in MiB) and data type in the log message. It also corrects the spelling of "overridden" in the log output.

• Enhanced debug logging for better visibility into tensor properties.
• Fixed minor spelling error in the log messages.

Comments suppressed due to low confidence (1)

src/llama-model.cpp:1655

• The updated debug log message improves clarity by including additional tensor details. Consider verifying that the integer division used for calculating the tensor size meets your precision requirements, or switch to floating point arithmetic if a more precise value is desired.

LLAMA_LOG_DEBUG("tensor %s (%zu MiB %s) buffer type overridden to %s\n", tensor_name.c_str(), ggml_nbytes(t_meta) / 1024 / 1024, ggml_type_name(t_meta->type), ggml_backend_buft_name(buft));

[ddh0]

yayy thank you!