Using GitHub Copilot's LLM

[Hzbeta]

I’d like to know if it’s possible to use GitHub Copilot’s LLM directly in the Roo Cline extension. This would allow leveraging the GitHub Copilot subscription system instead of managing a separate API key, making it more cost-efficient and convenient.

For instance, as shown in the screenshot, GitLens supports similar integration, enabling features like commit interpretation without requiring an API key.

If we could use GitHub Copilot's LLM, subscribing to GitHub Copilot Pro with monthly payments would allow almost unlimited access to Claude Sonnet 3.5, similar to subscribing to ChatGPT Plus for nearly unlimited GPT-4 usage. This would be far more convenient and cost-effective compared to paying per usage through API calls.

I’d like to know if this is possible, or if extensions like Roo Cline, which heavily rely on LLMs, cannot call GitHub Copilot? Thank you for your hard work and the wonderful extension you’ve brought us!

[bkutasi]

Very good request and I'm all for it. However, I would like to point out that subscribing to ChatGPT Plus won't get you near unlimited requests, especially during peak hours(US work time). In cases like that, you might even be limited to 50-100 messages/day. There might be the same rate-limiting guardrails implemented in copilot.

[mrubens]

Yes, someone actually opened a PR for this that I was very interested in, but when I tried it out I got rate limited very quickly.

[Hzbeta]

Thank you for your response. I had only checked issues and discussions for related content but didn’t find anything, so I created a new discussion. I’m glad to see that this topic has already been discussed recently. Although rate limiting exists, I still look forward to the relevant PR being merged. For others interested in this topic, please follow the progress of the following PRs and repositories:

• feat: add vscode-lm provider integration #276
• feat: implements the VSCode Language Model API as a provider. (Copilot) cline/cline#970
• https://github.com/julesmons/recline

[vincentsong]

@Hzbeta do you mean this kind of feature which was done by https://github.com/kturung/cogent ? I liked it since I have a copilot pro subscription.

[mrubens]

@Hzbeta I just released an experimental version of this in 3.1.2. Want to try it out and let me know if it works for you? Thanks!

[vincentsong]

@mrubens I chose VS Code LM API with my copilot - claude sonnet, and got the error below when tried it

Could not resolve authentication method. Expected either apiKey or authToken to be set. Or for one of the "X-Api-Key" or "Authorization" headers to be explicitly omitted

[mrubens]

Ugh. Does restarting vscode help at all?

[vincentsong]

I even tried to restart my macbook, still same. Do you have any setup instruction?

[mrubens]

Sorry! It was a bug on my end. Just released 3.1.3 with the fix.

[vincentsong]

it's working now. Thank you @mrubens.

[gkruger1]

Works very well for me. I hit Rate Limit Exceeded at 4.9m; 30.9k tokens.

[zblabs]

Today I started using the GitHub Copilot subscription ($10/mo) and blew through over 10M tokens. AWESOME. This build is incredible. And had crazy good progress on my app. Beats using CoPilot or Amazon Q alone. With all the attention given to node based development I think that is actually a distraction compared to having code assistants like Roo Cline.

[mdlmarkham]

Github Copilot is working in VSCode... but it's not showing up in the settings. Is there something else that I need to configure to enable API access?

And when I try to chat, it returns this:

[mdlmarkham]

My issue was that I needed to upgrade VSCode to the latest release - that fixed it.

[ebynnerlys]

English Version:
While using Roo Code 3.3.2, I encountered the "rate limit exceeded" error with the Sonnet 3.5 model via GitHub Copilot. Could someone clarify what the maximum request limit is within a specific time frame? Additionally, if I exceed this limit, how long should I wait before the rate limit resets? I am located in Spain and have a paid subscription to GitHub Copilot. My system is running Windows 11 23H2 (Build 22631.4751), and I am using Visual Studio Code version 1.96.4.

Spanish Version:
Al utilizar Roo Code 3.3.2, me aparece el mensaje de "rate limit exceeded" con el modelo Sonnet 3.5 a través de GitHub Copilot. ¿Podría alguien aclarar cuál es el límite máximo de solicitudes permitidas en un período de tiempo determinado? Además, si supero este límite, ¿cuánto tiempo debo esperar para que se restablezca? Estoy en España y tengo una suscripción de pago a GitHub Copilot. Mi sistema está ejecutando Windows 11 23H2 (Compilación 22631.4751) y estoy utilizando Visual Studio Code versión 1.96.4.

[MuriloFP]

English Version: While using Roo Code 3.3.2, I encountered the "rate limit exceeded" error with the Sonnet 3.5 model via GitHub Copilot. Could someone clarify what the maximum request limit is within a specific time frame? Additionally, if I exceed this limit, how long should I wait before the rate limit resets? I am located in Spain and have a paid subscription to GitHub Copilot. My system is running Windows 11 23H2 (Build 22631.4751), and I am using Visual Studio Code version 1.96.4.

Spanish Version: Al utilizar Roo Code 3.3.2, me aparece el mensaje de "rate limit exceeded" con el modelo Sonnet 3.5 a través de GitHub Copilot. ¿Podría alguien aclarar cuál es el límite máximo de solicitudes permitidas en un período de tiempo determinado? Además, si supero este límite, ¿cuánto tiempo debo esperar para que se restablezca? Estoy en España y tengo una suscripción de pago a GitHub Copilot. Mi sistema está ejecutando Windows 11 23H2 (Compilación 22631.4751) y estoy utilizando Visual Studio Code versión 1.96.4.

This is Copilot's rate limit. It's not specified what are the rate limits exactly, but it's related to token usage in a short time. If you open Copilot's chat window, and try to send a message, it'll tell how long you need to wait before using it again. Maybe we could add this rate limit timer to RooCode's UI, but I'm not sure how complicated this would be.

[mrubens]

If anyone has ideas for how to get this information I'm all ears!

[coredevel]

I've just tried using sonnet with Roo Code via GH but got the following:

Request Failed: 400 {"error":{"message":"The requested model is not supported.","param":"model","code":"model_not_supported","type":"invalid_request_error"}}

[ebynnerlys]

English Version

Hey! I found a way to optimize context usage and avoid hitting rate limits. Here’s what I’m doing:

Before starting any development, I ask Roo Code to handle this task:

“Your task is to identify large files and split them into smaller, modular parts. Goal: Simplify maintenance and reduce context overload.”  

This way, when Roo Code reads the directory tree, it knows exactly which files to include in the context and which to exclude. This helps me:

• Avoid GitHub Copilot’s rate limits.
• Stay within DeepSeek R1’s 128k token limit.
• Prevent Claude Sonnet 3.5 from truncating responses (since it seems to have a ~500-line output limit).

Also, I noticed on Anthropic’s website that Claude 3.5 Sonnet has these standard limits:

• Requests per Minute: 5
• Input Tokens per Minute: 20,000
• Output Tokens per Minute: 4,000

They also mention that custom limits are available if you contact their sales team.

This approach has been working well for me so far. Let me know if you try it or have other tips!

---

Versión en Español

¡Hola! Encontré una forma de optimizar el uso del contexto y evitar los límites de tasa. Esto es lo que hago:

Antes de empezar a desarrollar, le pido a Roo Code que haga esta tarea:

“Tu tarea es identificar archivos grandes y dividirlos en partes más pequeñas y modulares. Objetivo: Simplificar el mantenimiento y reducir la sobrecarga de contexto.”  

Así, cuando Roo Code lee el árbol de directorios, sabe exactamente qué archivos incluir en el contexto y cuáles no. Esto me ayuda a:

• Evitar los límites de velocidad de GitHub Copilot.
• Mantenerme dentro del límite de 128k tokens de DeepSeek R1.
• Evitar que Claude Sonnet 3.5 corte las respuestas (parece que tiene un límite de ~500 líneas por respuesta).

Además, vi en la web de Anthropic que Claude 3.5 Sonnet tiene estos límites estándar:

• Solicitudes por minuto: 5
• Tokens de entrada por minuto: 20,000
• Tokens de salida por minuto: 4,000

También mencionan que hay límites personalizados si contactas a su equipo de ventas.

Esta estrategia me ha funcionado bien hasta ahora. ¡Dime si la pruebas o si tienes otros consejos!

---

Espero que te sirva. 😊

[mdlmarkham]

That is fantastic - is there any way to incorporate this into the standard behavior of the tool? Perhaps using .clinerules?

[ebynnerlys]

Code Quality Rules

1. Test Coverage:

   • Before attempting completion, always make sure that any code changes have test coverage.
   • Ensure all tests pass before submitting changes.

2. Lint Rules:

   • Never disable any lint rules without explicit user approval.
   • If a lint rule needs to be disabled, ask the user first and explain why.
   • Prefer fixing the underlying issue over disabling the lint rule.
   • Document any approved lint rule disabling with a comment explaining the reason.

3. Context Optimization:

   • Before switching to Code Mode, identify and split large files into smaller, modular parts to optimize context usage.
   • Goal: Simplify maintenance and reduce context overload.
   • Trigger: Automatically apply this rule in Architect Mode before transitioning to Code Mode.
   • Action: Preprocess files to ensure only relevant and manageable parts are included in the context.

Adding a New Setting

To add a new setting that persists its state, follow the steps in cline_docs/settings.md.

[ebynnerlys]

El archivo está en en el directorio C:\Users\tu-usuario.vscode\extensions\rooveterinaryinc.roo-cline-3.3.4.clinerules

[miteshashar]

As expected and as many people have encountered, my experience is no different. Two precise features may help:

1. Letting the users also look at the rate limit response headers for each API call.
2. Instead of or in addition to the well-intended minimum time gap slider, we could have some logic that reads the rate limit headers to:
   a. Throttle API requests
   b. Show when to next try using the API.

[marijnbent]

Especially with the low limit of Gemini that would be amazing