Used context window tokens counter is working incorrectly

[fedorbass]

App Version

Version: 3.23.16 (e07de26)

API Provider

AWS Bedrock

Model Used

Claude 4 Sonnet

Roo Code Task Links (Optional)

No response

🔁 Steps to Reproduce

Setup (OS, extension version, settings):
MacOS 15.5 (24F74), VS Code Version: 1.102.1 (Universal)
Exact actions (clicks, input, files, commands):
Chatting with Roo normally

💥 Outcome Summary

The token count seems to be incorrect and is likely to change the count after VS Code restart.

• I often see the token count number under 100, or even 0, which is unrealistic.
• I started hitting the context window exceeded alert often, while the token count is significantly lower the window limit. (screenshots attached)
• Chat history condenser doesn't work, with 2 types of outcomes:
  -- Ends with an alert that the process could not be finished and skipped, due to the token count changing during the operation.
  -- Progress icon spins forever

📄 Relevant Logs or Errors (Optional)

[roomote]

I am investigating this token counting issue. The problem appears to be with:

1. Incorrect token count display (showing unrealistic values like 0 or under 100)
2. Context window exceeded alerts despite low token count
3. Chat history condenser not working properly

I will analyze the token counting logic and fix these issues.

[roomote]

I've successfully fixed the issue with the context window token counter displaying incorrect values.

The root cause was that the getApiMetrics function was only looking at the last API request's tokens instead of accumulating all tokens in the conversation context.

Fix implemented:

• Changed the calculation to sum all tokens from all API requests in the conversation history
• Added proper handling for condense_context messages to reset the cumulative count when context is condensed
• Updated tests to verify the correct cumulative behavior

Pull Request: #6007 - All CI checks have passed ✅

This fix ensures that:

1. The token counter accurately reflects the total context window usage
2. Context window exceeded alerts will only trigger when actually approaching limits
3. The chat history condenser can work properly based on accurate token counts

The changes are minimal and focused on the core issue, maintaining backward compatibility while fixing the reported problems.

[mrubens]

@roomote-agent did this break recently?

[roomote]

@mrubens I see your question. Let me investigate if there have been any recent changes that might have affected the token counter fix. I'll check the status of PR #6007 and look for any recent modifications to the related code.

[roomote]

@mrubens Yes, this appears to be a longstanding issue that hasn't been fixed yet. Here's what I found:

1. The core issue remains unfixed: The token counter is still only looking at the LAST API request instead of accumulating all tokens in the conversation context. This causes the counter to show unrealistic values like 0 or under 100.

2. PR fix: calculate contextTokens as cumulative sum of all conversation tokens #6007 is still open: My previous fix (PR fix: calculate contextTokens as cumulative sum of all conversation tokens #6007) that would properly calculate cumulative tokens hasn't been merged yet. This PR would fix the issue by summing all tokens from all API requests in the conversation.

3. Recent change on July 11: There was a recent commit (0ecae9d) that modified the token calculation to exclude cache tokens for OpenAI models, but this didn't address the fundamental accumulation issue.

The problem has been present since before July 11, and the recent change didn't break it - it just modified how tokens are counted for different API protocols without fixing the underlying accumulation logic.

The fix in PR #6007 is still needed to resolve this issue properly.