Expose Context Window Size for GitHub Copilot Models

[PhilippOesch]

Hello everyone,
this has quietly but surely become my go-to plugin for LLM pair programming. Thanks to the maintainer for the amazing job.

Recently, I’ve done some research and read about other people’s experiences with LLMs. A recurring theme, which I can also confirm from my own experience, is that there seems to be a correlation between how much of the LLM’s context window is used and the quality of its results.

Fortunately, the API of the adapter I use (Copilot) already provides this information.
I looked into the current implementation and realized that the Copilot adapter already reads other token limits when retrieving available models from the API, so adding the context window limits in practice would only require a single line addition in lua/codecompanion/adapters/http/copilot/get_models.lua

to do the trick.

--- a/lua/codecompanion/adapters/http/copilot/get_models.lua
+++ b/lua/codecompanion/adapters/http/copilot/get_models.lua
@@ -143,6 +143,7 @@ local function fetch_async(adapter, provided_token)
               if model.capabilities.limits then
                 limits.max_output_tokens = model.capabilities.limits.max_output_tokens
                 limits.max_prompt_tokens = model.capabilities.limits.max_prompt_tokens
+                limits.max_context_window_tokens = model.capabilities.limits.max_context_window_tokens
               end
             end

This would already be enough to, for example, display how much of the context window is currently being used:

I would, of course, be happy to provide this one liner PR.

It’s a small and non-intrusive change that would make me, and maybe some others, very happy.

[olimorris]

Hey @PhilippOesch. Yep great idea. This was on my radar after I made max_tokens dynamic on the Copilot adapter. Please do PR this. I would just ask that we investigate updating chat:_update_metadata in the chat class to contain this.