Re-add free models to API provider, but help people to use it when rate limited

[Juice10]

All free models were removed ( https://github.com/Kilo-Org/kilocode-backend/pull/1743 ) from the Kilo Code API Provider because people getting rate limited constantly made it pretty much unusable.

This is bad because now the free models aren't discoverable anymore.

We should re-add the free models so they are discoverable. And guide people towards adding their own Open Router API key to the Kilo Code provider (we'd also need to accept OR API keys in our backend) for the free models so they don't get rate limited.

This way free models are still discoverable and people can also be successful with them.

[markijbema]

what would be the incentive for people be to go this route instead of using their own key directly from the extension?

[Juice10]

Less overhead of having to switch between and create many profiles. They can just use the model picker in the bottom left with the kilo code provider as their default.

[chrarnoldus]

We could fall back to OpenRouter in the extension, no need to go through our backend.

[chrarnoldus]

adding their own Open Router API key to the Kilo Code provider (we'd also need to accept OR API keys in our backend)

I doubt this is a feature we want to build. We can show a warning in the extension when people try to use a free model instead.

[Juice10]

The goal here is not to want to build a feature. The goal is how can we get users productive and happy with Kilo Code. Does making free models undiscoverable make people happy and productive? No. Does showing a warning make people happy and productive? No. Does giving helping the users to use the free models make users happy and productive? Yes. Does it have to be in the way I proposed above? No, as long as whatever we do does make users happy

[chrarnoldus]

Considering users have to fund their OpenRouter account in order to get the free requests, there's a limit to how happy we can make them. I think a "warning" providing guidance is a good first step, we don't have to jump to the most complex thing immediately.

[chrarnoldus]

If we really want this, we could provide our own selection of "free" models.

[Juice10]

What does this mean?

[chrarnoldus]

Give people with a funded Kilo Code account limited free access (basically what OpenRouter is doing).

[devlux76]

When I hit limits I just switch to the free models on open router.
I never even noticed you had free models on offer.

[Juice10]

Some of Open Router's free models are not subject to the same type of per user rate limit that was the argument for removing all of the free models. This is another reason for restoring the free models in the Kilo Code API Provider.

Source: https://kilo-code.slack.com/archives/C08P0HYC9S4/p1753921167645229

[chrarnoldus]

That model is not affected by the change.