LibreChat API for external access to LLMs available in the LibreChat instance

[moritzfl]

What features would you like to see added?

An API to talk with the models available on LibreChat on a per user-account basis

More details

It seems like the request-schema that is used for the web-frontend to communicate with the backend is already fairly easy to understand. However, we still need to manage a user session and through OAuth before LibreChat this gets complicated rather quickly.

It would therefore be nice to have an official LibreChat-API that allows the linking of LibreChat with other software that wants access to a LLM (possibly with access through a LibreChat-API-Token, in the very best case compatible to OpenAIs API).

In a company setting, LibreChat could be the web-frontend that is offered to all employees.
Sometimes, the integration in another UI is desirable however. For coding, we want access within our IDE, for our technical writers it would be nice to have integration in their tooling too etc.

This would also have the following advantages:

• we could provide API access to all desired models without disclosing the LLM API keys (e.g. Azure or OpenAI API Keys) to employees.
• we can manage, switch out and easily try out multiple AI models
• we can easily integrate our AI-Plugins in a way so that they are accessible from the other UIs as well
• we have centralized user management that now also works for AI and can restrict token usage etc.

As an example, you could look at CodeGPT which is what we currently use for our developers:
https://github.com/carlrobertoh/CodeGPT
carlrobertoh/ProxyAI#426

Which components are impacted by your request?

General

Pictures

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Hi thanks for checking out the project. This has been brought up a few times to me but as a full stack developer, this is concerning as it introduces a new potential attack vector. Also concerning: I really started seeing this request following counter measures taken in my work against users bypassing the frontend for backend access. While I'm sure your use case and others' are legitimate, it can often not be the case.

I would almost rather provide an "alternate" api that only exposes a few routes, mainly the chat stuff, and exclusively deals with API "keys" or w/e authentication would work best for this. Also, personally, this is not very high on my list of requests.

I'll keep this issue open so other users can "vote" on it.

[stevenbedrick]

I would like to "second" this idea; we are evaluating LibreChat as a way to allow groups of users to access various LLM services and it would be great if LibreChat could also act as an API endpoint.

[el-analista]

Me as well. Its a great way to centralize full ai usage

[sellino]

Hi,
I would like to see that. I am currently learning how to use it and my goal is to communicate with the documents stored in Librechat via a web call. Ultimately, Siri should access the API via a shortcut. As far as I know, this access is not possible at the moment. Feel free to correct me. Sry for my English, Google Translate;)

[michnovka]

This would be great at least for chat API, given that agents now can call tools and MCP servers. So by exposing a chat/completions API we could use any model with extended capabilities!

[kpatdev]

Yes, I agree. An API to talk to LibreChat LLMs would be nice. I'd love to make (at least try lol) a Raycast extension that can interact with LibreChat backend.

[danny-avila]

Due to popular demand here, I’m adding it to the roadmap for this year!

[semiautomatix]

Cheers, the really obvious use is as an api for voice assistants, which are obviously a very different type of front end.

[tedivm]

I love to hear this!

[adamscross04]

https://www.librechat.ai/blog/2025-02-20_2025_roadmap

I don't see a reference to this issue (2517) or "external" referenced on the roadmap. Was it added?

Or is it encompassed into "OpenAI-compatible API for LibreChat Agents"?

[ronak21691]

+1 on this.
Apart from integrating it in internal platforms. This will also be highly used in setting up evals in platforms like comet/Opik, Langsmith where they require Agent endpoint for evaluation and Observability

[ronak21691]

has any one done work around on this meanwhile?
We are setting up evals using https://github.com/comet-ml/opik and would need Agent endpoint

Any pointers would be really helpful