Skip to content

[Feature Request] Add Support for Custom API Endpoints #55

@mwnu

Description

@mwnu

Is your feature request related to a problem? Please describe.

Currently, baibot is an excellent AI bot for Matrix, but its predefined API endpoints are primarily compatible with a limited set of major LLM providers like OpenAI. As the AI landscape rapidly diversifies, many new and powerful models are emerging with unique API specifications that differ from the OpenAI standard.

This limitation restricts the bot's ability to integrate with a wider range of AI services, including different model modalities (like specialized vision models) and non-chat functionalities (like search APIs).

Describe the solution you'd like

I propose the implementation of a "Custom API Endpoint" feature. This would allow users to define their own HTTP request and response mappings directly in the bot's configuration.

A user should be able to configure:

  1. Endpoint URL: The target API endpoint for the custom service.
  2. Request Mapping: How to structure the outgoing request payload. This could involve using template variables (e.g., {{.UserID}}, {{.Content}}) to pass data from the Matrix event to the target API.
  3. Response Mapping: How to parse the incoming JSON response from the custom API and format it into a message to be sent back to the Matrix room. For instance, specifying which JSON field contains the desired text or image URL.

Describe alternatives you've considered

The only alternative is to manually add support for each new model or API standard by modifying the bot's source code. This is not scalable and places a significant maintenance burden on the project developers. A generic, user-configurable solution would be far more flexible and future-proof.

Additional context

This feature would unlock several powerful new capabilities:

  • Integrate with a wider variety of AI models: Users could easily add support for models that don't follow the OpenAI API spec, such as Google's Gemini Vision models, Alibaba's qwen-vl, image llms, or other specialized multimodal models. This is related to the request in issue #53.
  • Connect to new types of AI functions: It would enable baibot to interact with tools like search APIs (e.g., Serper, Google Search API), enabling functionalities similar to those discussed in issue #42.
  • Serve as a generic Webhook endpoint: baibot could be used to forward messages from Matrix to any external service that accepts webhooks, effectively acting as a simple bridge.

By implementing custom API endpoint support, baibot would evolve from a dedicated LLM bot into a highly extensible and versatile automation tool for the Matrix ecosystem.

Thank you for considering this request!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions