make memset range dynamic

[pockers21]

When the model processes multiple data batches during inference, if the first data entry has a length of 1000, the output buffer length will be calculated as (1000 × vocab_size / embd_size). After allocating the output buffer space, the system performs a full-range memset operation. However, when subsequent inference data (e.g., the second batch) has a shorter length than 1000, the memset operation still clears the entire 1000-length space, resulting in significant performance overhead.
This PR introduces dynamic length adjustment to update memset parameters based on actual data length, thereby minimizing unnecessary performance waste.

[ngxson]

I think this will cause memory leak rather than actually saving memory

And worst, if cur_size is bigger than the actually capacity of the buffer, you're now having buffer overflow

[pockers21]

I think this will cause memory leak rather than actually saving memory

And worst, if cur_size is bigger than the actually capacity of the buffer, you're now having buffer overflow

1, I will carefully review for potential memory leaks.
2, if the buffer is unallocated or the new size exceeds the previous size, it should allocate a new buffer of the new size. With this code:
if (!buf_output || prev_size < new_size) { buf_output.reset(ggml_backend_buft_alloc_buffer(buft, new_size)); }
it shouldn't cause out-of-bounds issues, correct?

[slaren]

This is not a good change to ggml-backend. I don't know why the output buffer needs to be cleared, just remove the call to ggml_backend_buffer_clear.

[pockers21]

Hi @ngxson and @slaren ,

Thank you both for taking the time to review this PR and providing valuable feedback!

@slaren , your comment about the necessity of the ggml_backend_buffer_clear call itself is a great point. I've taken another look at the code in llama-context.cpp where this buffer is used.

It appears that the memory regions pointed to by logits and embd are indeed intended to be fully populated by the results of the subsequent graph computation (ggml_graph_compute). If that's the case, clearing them beforehand seems redundant and introduces unnecessary overhead, just as my initial PR description aimed to reduce.

Therefore, I tend to agree with your suggestion: removing the ggml_backend_buffer_clear(buf_output.get(), ...). This would simplify the code and directly address the performance concern without needing changes in ggml-backend or introducing the potential risks @ngxson rightly pointed out regarding dynamic size management.

My plan is to update this PR to simply remove that line. Does this approach sound reasonable to you, or are there any specific reasons why this buffer needs to be cleared that I might be missing?

Thanks again for your guidance!

[pockers21]

This PR's history became messy after rebasing onto the latest master/main. I've created a new, clean pull request at #13152 to replace this one. Closing this PR in favor of the new one. Thanks!