Add option to select multiple model for modes

[berkaycagli]

Hey guys, first of all, thanks for the great open-source project. It would have been great if we could add multiple models from multiple providers to the modes. (e.g. I want my Code mode to use Claude Code CLI, when the limit is reached, fallback to Gemini CLI, when the limit is reached fallback to API call as an example).

This would help optimizing the cost a lot in my opinion especially when a free version of a model is available from openrouter.

[hassoncs]

Hey @berkaycagli- Thank you for opening this discussion! We recently added a new Virtual Quota Fallback Provider which you should be able to configure exactly how you described!

I know I teased this a while back, but now with @hassoncs's help: Virtual Quota Fallback Provider

This new virtual provider lets you set cost- or request-based quotas for a list of profiles. It will automatically falls back to the next profile's provider when any limit is reached!

So what do do here is configure your providers as usual (including the ones like AIstudio or GeminiCLI that have rate limits). Then, spin up the Virtual provider in a new profile and select your other profiles, with appropriate rate limits that much what the vendor states (like 5 RPM/100 RPD for Gemini Pro on AI Studio). The plugin keeps track of usage in your vscode profile (shared across all projects), and switches profiles whenever the current profile cannot proceed with the request due to rate limits. Everything is tracked on a sliding minute/hour/day scale.

You can set up to 32 providers in a single config :).

What I've done in the screen shot above is use my free AI studio and Gemini CLI stuff first, and then fall back to Kilocode when those run out. So I'm always maximizing my free keys, but keeping the same models, same context, same thread, same configs. Any setting specifically set on the profile (like thinking tokens, etc) is respected.

Some things that dont work great yet:

You dont get notification of changing providers - I have a PR up for that
You dont get to see current usage - I'm writing a PR for that now
The little picker at the bottom below the Mode isn't accurate. Researching that.
The pricing reporting - I'm not confident its accurate.

You can configure this provider in the Provider Settings- let us know what you think and if you have any feedback!

[kenken1985]

Can you add max token criteria? It would be nice if it uses up shorter context model first whenever it can.

[berkaycagli]

Wow, cool!! Thanks a lot!