Cerebras

[Dual-0]

Cerebras Systems designs and manufactures cutting-edge AI hardware, specifically high-performance chips significantly faster and more efficient than many competitors. They've recently been in the news due to a previously undisclosed 2017 attempt by OpenAI to acquire them, highlighting the significant value and advanced technology they offer. This acquisition, though ultimately unsuccessful, underscores Cerebras' position as a key player in the AI hardware landscape.

I've tested the integration of Cerebras powerful AI computing platform into LibreChat successfully. The API can be integrated easily cause its based on OpenAI API.

API Reference

https://inference-docs.cerebras.ai/api-reference

cURL

curl --location 'https://api.cerebras.ai/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer ${CEREBRAS_API_KEY}" \
--data '{
  "model": "llama3.1-8b",
  "stream": false,
  "messages": [{"content": "Hello!", "role": "user"}],
  "temperature": 0,
  "max_completion_tokens": -1,
  "seed": 0,
  "top_p": 1
}'

Response:

{
  "id": "chatcmpl-292e278f-514e-4186-9010-91ce6a14168b",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "Hello! How can I assist you today?",
        "role": "assistant"
      }
    }
  ],
  "created": 1723733419,
  "model": "llama3.1-8b",
  "system_fingerprint": "fp_70185065a4",
  "object": "chat.completion",
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 10,
    "total_tokens": 22
  },
  "time_info": {
    "queue_time": 0.000073161,
    "prompt_time": 0.0010744798888888889,
    "completion_time": 0.005658071111111111,
    "total_time": 0.022224903106689453,
    "created": 1723733419
  }
}

librechat.yaml

# Definition of custom endpoints
endpoints:
    - name: "Cerebras"
      apiKey: "${CEREBRAS_API_KEY}"
      baseURL: "https://api.cerebras.ai/v1/"
      models:
        default: ["llama3.1-8b", "llama-3.3-70b"]
        fetch: true
      titleConvo: true
      titleModel: "llama3.1-8b"
      modelDisplayLabel: "Cerebras"

Try it, its blazin fast! even faster than groq

would be nice to add them to the known endpoints and docs

[Fusion-Parth]

Yes, it works like a charm! I implemented these configurations just a few days ago, and the integration of Cerebras' AI computing platform into LibreChat has been seamless. The API integration is smooth due to its similarity with OpenAI's API.