Skip to content

Conversation

@roomote
Copy link
Contributor

@roomote roomote bot commented Aug 19, 2025

Summary

This PR adds Featherless AI as a new provider to Roo Code, allowing users to leverage Featherless AI's API for model inference.

Changes

  • Added Featherless to the provider names list in packages/types/src/provider-settings.ts
  • Created Featherless model types with popular models in packages/types/src/providers/featherless.ts
  • Implemented FeatherlessHandler using the OpenAI-compatible API pattern
  • Added Featherless schema and configuration
  • Wired up Featherless in the API handler

Implementation Details

Featherless AI provides an OpenAI-compatible API, so the implementation extends the existing BaseOpenAiCompatibleProvider class. This ensures consistency with other similar providers and reduces code duplication.

Models Included

  • Meta Llama 3.1 (8B, 70B, 405B)
  • Qwen 2.5 72B
  • Mistral 7B and Mixtral 8x7B
  • DeepSeek R1 Distill
  • Moonshot Kimi K2

Testing

  • ✅ TypeScript compilation successful
  • ✅ All existing tests pass
  • ✅ Linting passes

Related Issue

Closes #7237

Acceptance Criteria

When users:

  1. Choose Featherless AI as a provider
  2. Enter their API key
  3. Select a model from Featherless AI

They should be able to use it just like any other provider in Roo Code.


Important

Adds Featherless AI as a provider with model support and API integration using BaseOpenAiCompatibleProvider.

  • Behavior:
    • Adds Featherless AI as a provider in provider-settings.ts and index.ts.
    • Implements FeatherlessHandler in featherless.ts using BaseOpenAiCompatibleProvider.
    • Supports models like Meta Llama 3.1, Qwen 2.5, Mistral 7B, DeepSeek R1, and Moonshot Kimi K2.
  • Models:
    • Defines Featherless model types in featherless.ts with details like maxTokens, contextWindow, and pricing.
  • API Integration:
    • Integrates Featherless in buildApiHandler() in index.ts to handle API requests.
  • Misc:
    • Updates providers/index.ts to export FeatherlessHandler.

This description was created by Ellipsis for 5cf78a4. You can customize this summary. It will automatically update as commits are pushed.

- Add Featherless to provider names list
- Create Featherless model types with popular models
- Implement FeatherlessHandler using OpenAI-compatible API
- Add Featherless schema and configuration
- Wire up Featherless in API handler

Closes #7237
@roomote roomote bot requested review from cte, jr and mrubens as code owners August 19, 2025 23:07
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. enhancement New feature or request labels Aug 19, 2025
@roomote roomote bot mentioned this pull request Aug 19, 2025
4 tasks
@daniel-lxs daniel-lxs closed this Aug 19, 2025
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap Aug 19, 2025
@github-project-automation github-project-automation bot moved this from Triage to Done in Roo Code Roadmap Aug 19, 2025
Copy link
Contributor Author

@roomote roomote bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewing my own code is like debugging in a mirror - everything looks backward but the bugs are still mine.

contextWindow: 131072,
supportsImages: false,
supportsPromptCache: false,
inputPrice: 0.1,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pricing values here (0.1, 0.4, 2.0, etc.) appear to be placeholder values. Could we verify the actual pricing from Featherless AI's documentation? These rounded values might not reflect the actual per-token costs.

// Featherless AI models - https://api.featherless.ai/v1/models
export type FeatherlessModelId = keyof typeof featherlessModels

export const featherlessDefaultModelId: FeatherlessModelId = "meta-llama/Meta-Llama-3.1-8B-Instruct"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have we confirmed that all these model IDs are actually available through Featherless AI's API? It would be good to verify these against their API documentation or model list endpoint at https://api.featherless.ai/v1/models

@@ -0,0 +1,81 @@
import type { ModelInfo } from "../model.js"

// Featherless AI models - https://api.featherless.ai/v1/models
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding the actual API documentation URL here for reference, not just the models endpoint. This would help future maintainers understand the API structure.


export class FeatherlessHandler extends BaseOpenAiCompatibleProvider<FeatherlessModelId> {
constructor(options: ApiHandlerOptions) {
super({
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While the base class checks for apiKey, could we add a more specific error message for missing Featherless API key? Something like:

Suggested change
super({
if (!options.featherlessApiKey) {
throw new Error("Featherless API key is required")
}
super({
...options,
providerName: "Featherless",
baseURL: "https://api.featherless.ai/v1",
apiKey: options.featherlessApiKey,
defaultProviderModelId: featherlessDefaultModelId,
providerModels: featherlessModels,
defaultTemperature: 0.7,
})

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:L This PR changes 100-499 lines, ignoring generated files.

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Add Featherless AI as a provider

3 participants