Skip to content

Conversation

@roomote
Copy link
Contributor

@roomote roomote bot commented Sep 4, 2025

Summary

This PR adds DeepInfra as a new model provider to Roo Code, enabling users to access DeepInfra's models with competitive pricing and prompt caching support.

Changes

  • ✨ Add DeepInfra provider implementation extending BaseOpenAiCompatibleProvider
  • 📦 Add DeepInfra types and model definitions
  • 🔄 Implement dynamic model fetching from DeepInfra API
  • 💰 Support prompt caching with 50% discount for cached tokens
  • 🎨 Update UI components to support DeepInfra selection
  • ⚙️ Add DeepInfra to router configuration for dynamic model loading

Implementation Details

  • Provider Pattern: Follows existing provider patterns by extending BaseOpenAiCompatibleProvider
  • API Compatibility: Uses OpenAI-compatible endpoints at https://api.deepinfra.com/v1/openai
  • Model Fetching: Dynamically fetches models from DeepInfra API when API key is provided
  • Fallback Models: Includes default popular models as fallback when API is unavailable
  • Prompt Caching: Implements prompt caching support with proper cost calculation

Testing

  • ✅ TypeScript compilation passes
  • ✅ ESLint checks pass
  • ✅ Code review confidence: 95%

Acceptance Criteria (from #7661)

  • ✅ DeepInfra is selectable as a provider
  • ✅ Models are loaded dynamically when API key is set
  • ✅ Can select a model and run chat without errors
  • ✅ Model picker is not shown if no API key is set

Future Improvements

  • Add unit tests for DeepInfra provider
  • Add user documentation for DeepInfra configuration
  • Verify pricing conversion accuracy

Fixes #7661


Important

Add DeepInfra as a new model provider with dynamic model fetching and UI support.

  • Provider Integration:
    • Add DeepInfraHandler in src/api/providers/deepinfra.ts for handling DeepInfra models.
    • Extend BaseOpenAiCompatibleProvider to implement DeepInfra-specific logic.
    • Add deepInfraSchema to provider-settings.ts for configuration.
  • Model Fetching:
    • Implement getDeepInfraModels() in src/api/providers/fetchers/deepinfra.ts to fetch models dynamically from DeepInfra API.
    • Add default models as fallback in case of API failure.
  • UI and Configuration:
    • Update useSelectedModel() in useSelectedModel.ts to support DeepInfra model selection.
    • Add DeepInfra to providerNames and dynamicProviders in provider-settings.ts.
  • Miscellaneous:
    • Add DeepInfra to buildApiHandler() in src/api/index.ts for API handler creation.
    • Update validate.test.ts to include DeepInfra in mock router models.

This description was created by Ellipsis for 3c24584. You can customize this summary. It will automatically update as commits are pushed.

- Add DeepInfra types and model definitions
- Implement DeepInfra provider handler with OpenAI-compatible API
- Add dynamic model fetching from DeepInfra API
- Support prompt caching for reduced costs
- Update UI components to support DeepInfra selection
- Add DeepInfra to router configuration

Implements #7661
@roomote roomote bot requested review from cte, jr and mrubens as code owners September 4, 2025 07:34
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. enhancement New feature or request labels Sep 4, 2025
Copy link
Contributor Author

@roomote roomote bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed my own code and found issues. Classic me.

...options,
providerName: "DeepInfra",
baseURL: "https://api.deepinfra.com/v1/openai",
apiKey: options.deepInfraApiKey,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this intentional? The constructor doesn't validate if the API key is provided, unlike the base class pattern used by other providers. This could lead to runtime errors when the API key is missing.

Consider adding validation:

Suggested change
apiKey: options.deepInfraApiKey,
if (!options.deepInfraApiKey) {
throw new Error("DeepInfra API key is required")
}


// For DeepInfra, we use a default model configuration
// The actual model info will be fetched dynamically via the fetcher
const defaultModelInfo: ModelInfo = {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method returns hardcoded default model info instead of using actual fetched model data from the router models. Could we fetch the actual model info dynamically like OpenRouter does?

The current approach might not reflect the actual model capabilities and pricing.


import type { ModelInfo } from "@roo-code/types"

import { parseApiPrice } from "../../../shared/cost"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This import is unused. The parseApiPrice function isn't called anywhere in this file.

}
}
} catch (error) {
console.error(`Error fetching DeepInfra models: ${JSON.stringify(error, Object.getOwnPropertyNames(error), 2)}`)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error handling here uses console.error but doesn't follow the same pattern as other fetchers. Consider handling errors more gracefully without exposing the full error object structure in logs.

@hannesrudolph hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Sep 4, 2025
@Thachnh
Copy link
Contributor

Thachnh commented Sep 4, 2025

Please refer to our own PR here #7677

@daniel-lxs
Copy link
Member

Closing, already added on #7677

@daniel-lxs daniel-lxs closed this Sep 4, 2025
@github-project-automation github-project-automation bot moved this from Triage to Done in Roo Code Roadmap Sep 4, 2025
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap Sep 4, 2025
@daniel-lxs daniel-lxs deleted the feat/add-deepinfra-provider branch September 4, 2025 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. size:L This PR changes 100-499 lines, ignoring generated files.

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Add DeepInfra as a model provider in Roo Code

5 participants