Agent: Token usage metrics

[manstis]

Hi,

Keeping track of LLM Token usage is important.

The InferenceApi's InferenceProvider.completion and InferenceProvider.chat_completion both support MetricResponseMixin.

• https://github.com/meta-llama/llama-stack/blob/main/llama_stack/providers/inline/inference/meta_reference/inference.py#L192

llama_stack/distribution/routers/inference.py also seems to support creating Token metrics, returning them in response chunks and exporting them to configured telemetry.

• https://github.com/meta-llama/llama-stack/blob/main/llama_stack/distribution/routers/inference.py#L146-L157
• https://github.com/meta-llama/llama-stack/blob/main/llama_stack/distribution/routers/inference.py#L159-L171

All good so far.

However if we use an Agent to support use of Tools (RAG, MCP etc) the Token metrics from the Agent's use of the InferenceApi are dropped, lost, not propagated out in the response chunks.

• https://github.com/meta-llama/llama-stack/blob/main/llama_stack/providers/inline/agents/meta_reference/agent_instance.py#L511

Is it reasonable to enhance llama-stack to return Token metrics when using an Agent?

Is there anything on a road-map to support it at a future date?

Perhaps I'm mistaken; so feel free to correct my understanding.