VS Code LM API unable to get the correct context size

[NaccOll]

App Version

3.24.0

API Provider

VS Code Language Model API

Model Used

copilot - gpt-4.1

Roo Code Task Links (Optional)

No response

🔁 Steps to Reproduce

Select "VS Code LM API", then choose "copilot - gpt-4.1". First, send a message saying "hi", at which point the context size will increase to just over 8k. Then, drag and drop some files into the dialog box to send them, and you'll notice that the context changes very little.

💥 Outcome Summary

expected to correctly calculate the context size, but apart from the system prompt being accurately accounted for, the conversation content has very little impact on the context size.

📄 Relevant Logs or Errors (Optional)

The updated VS Code's countTokens takes LanguageModelChatMessage2 rather than LanguageModelChatMessage.

[roomote]

Hi @NaccOll! I'm here to help out the maintainers and am going to see if I can fix this issue.

From your report, it looks like the VS Code Language Model API is not correctly calculating the context size when using the "copilot - gpt-4.1" model. The key insight from your logs is that VS Code's countTokens now expects LanguageModelChatMessage2 rather than LanguageModelChatMessage.

I'll investigate the codebase to find where the token counting is implemented and update it to use the correct interface. Thanks for the detailed bug report with screenshots!

[roomote]

Hi @NaccOll! I've successfully implemented a fix for this issue.

The problem was that VS Code's Language Model API has been updated to use LanguageModelChatMessage2 instead of LanguageModelChatMessage for the countTokens method, which was causing incorrect context size calculations.

I've created PR #6291 that addresses this by:

• Adding a fallback mechanism in the token counting logic to handle both old and new API versions
• Extracting text content from messages when the new API format is detected
• Ensuring backward compatibility so it works with both API versions

All tests are passing and CI checks have completed successfully. The fix ensures that Roo Code continues to work correctly with VS Code's Language Model API regardless of which version is being used.

You can view the PR here: #6291

Thank you for reporting this issue with such detailed information!

[NaccOll]

@roomote-agent
Still use the internalCountTokens method, except when the text type is LanguageModelChatMessage and tokenCount equals 4, convert the LanguageModelChatMessage to a string and recalculate via this.client.countTokens.

[roomote]

Hi @NaccOll! I understand your feedback. You want me to:

1. Continue using the internalCountTokens method
2. Only when the text type is LanguageModelChatMessage AND tokenCount equals 4, convert the LanguageModelChatMessage to a string and recalculate using this.client.countTokens

I'll investigate the current implementation and make these specific changes. Thank you for the clarification!

[roomote]

Hi @NaccOll! I understand your specific requirements:

1. Continue using the internalCountTokens method as the primary approach
2. Only when the text type is LanguageModelChatMessage AND the tokenCount equals 4, then:
   • Convert the LanguageModelChatMessage to a string
   • Recalculate using this.client.countTokens

I'll implement this targeted fix that maintains the existing logic while adding this specific condition. Let me explore the codebase to understand the current implementation and make these precise changes.