-
Notifications
You must be signed in to change notification settings - Fork 199
Open
Labels
P3new integrationDiscuss the creation of a new integration in CoreDiscuss the creation of a new integration in Core
Description
Is your feature request related to a problem? Please describe.
We need more robust, region-diverse, and cost-effective LLM options in Haystack pipelines. Relying on a small set of providers can cause issues during rate limiting, regional outages, or price-sensitive workloads. Adding CometAPI as an OpenAI-compatible provider would improve availability, portability, and resilience without disrupting existing workflows.
Describe the solution you'd like
Add CometAPI as a first-class LLM provider in Haystack, or officially support configuring the existing OpenAI components via base_url to point to CometAPI. Desired capabilities:
- Chat and text generation via OpenAI-compatible endpoints
- Model listing and selection via CometAPI model APIs
- API Key configuration and credential management
- Streaming and non-streaming support (leveraging Haystack abstractions if available)
- Minimal example + docs showing how to use CometAPI in a Haystack pipeline
Describe alternatives you've considered
- Use current OpenAI adapters with a custom
base_url(if already supported). This works but lacks discoverability and may have subtle compatibility gaps. - Go through third-party proxies/aggregators. This adds an extra dependency and less control.
- Stay with current providers only. This limits resilience, vendor flexibility, and cost optimization.
Additional context
CometAPI Resources:
- Website: https://www.cometapi.com/
- API Documentation: https://api.cometapi.com/doc
- Base URL: https://api.cometapi.com/v1/
- Model List: https://api.cometapi.com/v1/models
- Get API Key: https://api.cometapi.com/console/token
- Pricing: https://api.cometapi.com/pricing
Compatibility:
- CometAPI exposes OpenAI-compatible endpoints, enabling low-friction integration by reusing Haystack's existing OpenAI components where possible.
Implementation Offer:
- We’re happy to implement this integration and open a PR following your contribution guidelines (including examples and docs). We can also provide a temporary test API key and a small demo notebook/script if helpful. Please advise on the preferred integration point (new provider vs. extending OpenAI components) and acceptance criteria.
Metadata
Metadata
Assignees
Labels
P3new integrationDiscuss the creation of a new integration in CoreDiscuss the creation of a new integration in Core
Type
Projects
Status
In Progress