-
Notifications
You must be signed in to change notification settings - Fork 2.6k
feat: add Featherless AI as a provider #7238
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
- Add Featherless to provider names list - Create Featherless model types with popular models - Implement FeatherlessHandler using OpenAI-compatible API - Add Featherless schema and configuration - Wire up Featherless in API handler Closes #7237
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewing my own code is like debugging in a mirror - everything looks backward but the bugs are still mine.
| contextWindow: 131072, | ||
| supportsImages: false, | ||
| supportsPromptCache: false, | ||
| inputPrice: 0.1, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The pricing values here (0.1, 0.4, 2.0, etc.) appear to be placeholder values. Could we verify the actual pricing from Featherless AI's documentation? These rounded values might not reflect the actual per-token costs.
| // Featherless AI models - https://api.featherless.ai/v1/models | ||
| export type FeatherlessModelId = keyof typeof featherlessModels | ||
|
|
||
| export const featherlessDefaultModelId: FeatherlessModelId = "meta-llama/Meta-Llama-3.1-8B-Instruct" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have we confirmed that all these model IDs are actually available through Featherless AI's API? It would be good to verify these against their API documentation or model list endpoint at https://api.featherless.ai/v1/models
| @@ -0,0 +1,81 @@ | |||
| import type { ModelInfo } from "../model.js" | |||
|
|
|||
| // Featherless AI models - https://api.featherless.ai/v1/models | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider adding the actual API documentation URL here for reference, not just the models endpoint. This would help future maintainers understand the API structure.
|
|
||
| export class FeatherlessHandler extends BaseOpenAiCompatibleProvider<FeatherlessModelId> { | ||
| constructor(options: ApiHandlerOptions) { | ||
| super({ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While the base class checks for apiKey, could we add a more specific error message for missing Featherless API key? Something like:
| super({ | |
| if (!options.featherlessApiKey) { | |
| throw new Error("Featherless API key is required") | |
| } | |
| super({ | |
| ...options, | |
| providerName: "Featherless", | |
| baseURL: "https://api.featherless.ai/v1", | |
| apiKey: options.featherlessApiKey, | |
| defaultProviderModelId: featherlessDefaultModelId, | |
| providerModels: featherlessModels, | |
| defaultTemperature: 0.7, | |
| }) |
Summary
This PR adds Featherless AI as a new provider to Roo Code, allowing users to leverage Featherless AI's API for model inference.
Changes
packages/types/src/provider-settings.tspackages/types/src/providers/featherless.tsFeatherlessHandlerusing the OpenAI-compatible API patternImplementation Details
Featherless AI provides an OpenAI-compatible API, so the implementation extends the existing
BaseOpenAiCompatibleProviderclass. This ensures consistency with other similar providers and reduces code duplication.Models Included
Testing
Related Issue
Closes #7237
Acceptance Criteria
When users:
They should be able to use it just like any other provider in Roo Code.
Important
Adds Featherless AI as a provider with model support and API integration using
BaseOpenAiCompatibleProvider.provider-settings.tsandindex.ts.FeatherlessHandlerinfeatherless.tsusingBaseOpenAiCompatibleProvider.featherless.tswith details likemaxTokens,contextWindow, and pricing.buildApiHandler()inindex.tsto handle API requests.providers/index.tsto exportFeatherlessHandler.This description was created by
for 5cf78a4. You can customize this summary. It will automatically update as commits are pushed.