server: fix OpenAI Streaming API compatibility for usage statistics in chat streams

[TeoZosa]

Fixes OpenAI Streaming API spec compatibility for chat streams which include usage statistics (the default in llama-server).

Closes:

• Misc. bug: server: Usage statistics in chat streams added to slightly different chunk from OpenAI Streaming API #15443

[TeoZosa]

The only (non-test) PR change: adding an extra chunk with an empty choices array and setting usage stats there.

[TeoZosa]

Signposting that this change looks to be backwards-compatible with the bench script which checks if a chunk contains a usage field, independent of the choices content:

• llama.cpp/tools/server/bench/script.js

  Lines 128 to 136 in 1d36b36

  	if (chunk.usage) {
  	prompt_tokens = chunk.usage.prompt_tokens
  	llamacpp_prompt_tokens.add(prompt_tokens)
  	llamacpp_prompt_tokens_total_counter.add(prompt_tokens)
  	completions_tokens = chunk.usage.completion_tokens
  	llamacpp_completion_tokens.add(completions_tokens)
  	llamacpp_completion_tokens_total_counter.add(completions_tokens)
  	}

[h9j6k]

Hello,

I am getting can't access property "delta", ie.choices[0] is undefined in a browser pop-up error msg when using llama-server as before.

./llama-server -m ~/llm/models/google_gemma-3n-E4B-it-Q4_0.gguf -ot per_layer_token_embd.weight=CPU --host HOSTNAME --port PORT -c 8192 -b 8192 -e -ngl 99 -t 8 -n -1 --no-mmap -fa --jinja

Could it be related to this commit? Thanks.

[TeoZosa]

Hello,

I am getting can't access property "delta", ie.choices[0] is undefined in a browser pop-up error msg when using llama-server as before.

./llama-server -m ~/llm/models/google_gemma-3n-E4B-it-Q4_0.gguf -ot per_layer_token_embd.weight=CPU --host HOSTNAME --port PORT -c 8192 -b 8192 -e -ngl 99 -t 8 -n -1 --no-mmap -fa --jinja

Could it be related to this commit? Thanks.

Most likely! My fault for not catching what other compatibility was affected outside of tests and calling the model directly. It is most likely due to this line:

• llama.cpp/tools/server/webui/src/utils/app.context.tsx

  Line 258 in 5682a37

  const addedContent = chunk.choices[0].delta.content;

I can make a PR later today (assuming no one else gets to it first).