feat(llm): add token usage tracking and statistics for LLM providers

[einsiedler0408]

Describe this PR

feat(llm): add token usage tracking and statistics for LLM providers

This PR introduces comprehensive token usage tracking capabilities across all LLM provider clients to enable better monitoring and optimization of token consumption.

Key Features Added:

1. Cumulative Usage Tracking: Added token usage fields to LLMProviderClientBase to track:

   • Total input tokens (including cached and uncached)
   • Total output tokens (including reasoning and response tokens)
   • Session-level accumulation for both main and sub-agents

2. Automatic Usage Collection: Enhanced the base method to automatically extract and accumulate usage statistics from LLM responses using a standardized interface

3. Provider-Specific Support: Updated all LLM provider clients to properly support usage extraction:

   • claude_anthropic_client.py
   • claude_newapi_client.py
   • claude_openrouter_client.py
   • deepseek_newapi_client.py
   • gpt_openai_client.py
   • gpt_openai_response_client.py
   • qwen_sglang_client.py

4. Orchestrator Integration: Enhanced the orchestrator to:

   • Reset usage stats at the start of new agent sessions
   • Log cumulative usage statistics when completing agent tasks
   • Track usage independently for main agents and sub-agents

5. Utility Methods: Added helper methods for:

   • Formatted usage log generation with provider/model information
   • Usage stats reset functionality for new sessions
   • Graceful error handling for usage extraction failures

[Copilot]

Pull Request Overview

This PR adds comprehensive token usage tracking and statistics across all LLM provider clients to enable better monitoring and optimization of token consumption.

Key changes:

• Added cumulative usage tracking fields to the base LLM client class
• Implemented automatic usage extraction and accumulation for all responses
• Updated provider clients to use unified configuration access patterns and support usage extraction

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/llm/provider_client_base.py	Added token usage tracking fields, accumulation logic, and utility methods
src/core/orchestrator.py	Integrated usage tracking with session management and logging
src/llm/providers/claude_anthropic_client.py	Updated config access and added Anthropic-specific usage extraction
src/llm/providers/gpt_openai_response_client.py	Updated config access and added OpenAI Response API usage extraction
src/llm/providers/gpt_openai_client.py	Updated config access pattern
src/llm/providers/qwen_sglang_client.py	Updated config access pattern
src/llm/providers/deepseek_newapi_client.py	Updated config access pattern
src/llm/providers/claude_newapi_client.py	Updated config access pattern
src/llm/providers/claude_openrouter_client.py	Added usage tracking request parameter

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

This commented-out code should be removed rather than left as a comment. The functionality has been replaced by the new usage tracking system in the base class.

Suggested change

# self._update_token_usage(response.get("usage", None))