Skip to content

Commit c048e88

Browse files
daisyfaithaumakodster28
daisyfaithauma
and
kodster28
authored
[AIG]Chat-completions-endpoint (#22261)
* initial rest API guide * Initial documentation for chat completions * Revert "initial rest API guide" This reverts commit 2a091c4. * renamed * partial-test * added partial * minor edit * Partial fixes * Add Universal endpoint * updated content type --------- Co-authored-by: kodster28 <[email protected]>
1 parent 7766813 commit c048e88

File tree

15 files changed

+291
-4
lines changed

15 files changed

+291
-4
lines changed

‎src/content/docs/ai-gateway/chat-completion.mdx‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
---
2+
title: OpenAI Compatibility
3+
pcx_content_type: reference
4+
sidebar:
5+
order: 4
6+
---
7+
8+
Cloudflare's AI Gateway offers an OpenAI-compatible `/chat/completions` endpoint, enabling integration with multiple AI providers using a single URL. This feature simplifies the integration process, allowing for seamless switching between different models without significant code modifications.
9+
10+
## Endpoint URL
11+
12+
```txt
13+
https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions
14+
```
15+
16+
Replace `{account_id}` and `{gateway_id}` with your Cloudflare account and gateway IDs.
17+
18+
## Parameters
19+
20+
Switch providers by changing the `model` and `apiKey` parameters.
21+
22+
Specify the model using `{provider}/{model}` format. For example:
23+
24+
- `openai/gpt-4o-mini`
25+
- `google-ai-studio/gemini-2.0-flash`
26+
- `anthropic/claude-3-haiku`
27+
28+
## Examples
29+
30+
### OpenAI SDK
31+
32+
```js
33+
import OpenAI from "openai";
34+
const client = new OpenAI({
35+
apiKey: "YOUR_PROVIDER_API_KEY", // Provider API key
36+
baseURL:
37+
"https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions",
38+
});
39+
40+
const response = await client.chat.completions.create({
41+
model: "google-ai-studio/gemini-2.0-flash",
42+
messages: [{ role: "user", content: "What is Cloudflare?" }],
43+
});
44+
45+
console.log(response.choices[0].message.content);
46+
```
47+
48+
### cURL
49+
50+
```bash
51+
curl -X POST https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions \
52+
--header 'Authorization: Bearer {openai_token}' \
53+
--header 'Content-Type: application/json' \
54+
--data '{
55+
"model": "google-ai-studio/gemini-2.0-flash",
56+
"messages": [
57+
{
58+
"role": "user",
59+
"content": "What is Cloudflare?"
60+
}
61+
]
62+
}'
63+
```
64+
65+
### Universal provider
66+
67+
You can also use this pattern with a [Universal Endpoint](/ai-gateway/universal/).
68+
69+
```ts title="index.ts"
70+
export interface Env {
71+
AI: Ai;
72+
}
73+
74+
export default {
75+
async fetch(request: Request, env: Env) {
76+
return env.AI.gateway("default").run({
77+
provider: "compat",
78+
endpoint: "chat/completions",
79+
headers: {
80+
authorization: "Bearer ",
81+
},
82+
query: {
83+
model: "google-ai-studio/gemini-2.0-flash",
84+
messages: [
85+
{
86+
role: "user",
87+
content: "What is Cloudflare?",
88+
},
89+
],
90+
},
91+
});
92+
},
93+
};
94+
```
95+
96+
## Supported Providers
97+
98+
The OpenAI-compatible endpoint supports models from the following providers:
99+
100+
- [Anthropic](/ai-gateway/providers/anthropic/)
101+
- [OpenAI](/ai-gateway/providers/openai/)
102+
- [Groq](/ai-gateway/providers/groq/)
103+
- [Mistral](/ai-gateway/providers/mistral/)
104+
- [Cohere](/ai-gateway/providers/cohere/)
105+
- [Perplexity](/ai-gateway/providers/perplexity/)
106+
- [Workers AI](/ai-gateway/providers/workersai/)
107+
- [Google-AI-Studio](/ai-gateway/providers/google-ai-studio/)
108+
- [Grok](/ai-gateway/providers/grok/)
109+
- [DeepSeek](/ai-gateway/providers/deepseek/)
110+
- [Cerebras](/ai-gateway/providers/cerebras/)

‎src/content/docs/ai-gateway/configuration/index.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ title: Configuration
44
sidebar:
55
group:
66
hideIndex: true
7-
order: 4
7+
order: 5
88
---
99

1010
import { DirectoryListing } from "~/components";

‎src/content/docs/ai-gateway/providers/anthropic.mdx‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@ title: Anthropic
33
pcx_content_type: get-started
44
---
55

6+
import { Render } from "~/components";
7+
68
[Anthropic](https://www.anthropic.com/) helps build reliable, interpretable, and steerable AI systems.
79

810
## Endpoint
@@ -65,3 +67,17 @@ const message = await anthropic.messages.create({
6567
max_tokens: maxTokens,
6668
});
6769
```
70+
71+
<Render
72+
file="chat-completions-providers"
73+
product="ai-gateway"
74+
params={{
75+
name: "Anthropic",
76+
jsonexample: `
77+
{
78+
"model": "anthropic/{model}"
79+
}`
80+
81+
}}
82+
83+
/>

‎src/content/docs/ai-gateway/providers/cerebras.mdx‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ sidebar:
66
text: Beta
77
---
88

9+
import { Render } from "~/components";
10+
911
[Cerebras](https://inference-docs.cerebras.ai/) offers developers a low-latency solution for AI model inference.
1012

1113
## Endpoint
@@ -41,3 +43,17 @@ curl https://gateway.ai.cloudflare.com/v1/ACCOUNT_TAG/GATEWAY/cerebras/chat/comp
4143
]
4244
}'
4345
```
46+
47+
<Render
48+
file="chat-completions-providers"
49+
product="ai-gateway"
50+
params={{
51+
name: "Cerebras",
52+
jsonexample: `
53+
{
54+
"model": "cerebras/{model}"
55+
}`
56+
57+
}}
58+
59+
/>

‎src/content/docs/ai-gateway/providers/cohere.mdx‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@ title: Cohere
33
pcx_content_type: get-started
44
---
55

6+
import { Render } from "~/components";
7+
68
[Cohere](https://cohere.com/) build AI models designed to solve real-world business challenges.
79

810
## Endpoint
@@ -72,3 +74,17 @@ chat = co.chat(
7274
print(chat)
7375

7476
```
77+
78+
<Render
79+
file="chat-completions-providers"
80+
product="ai-gateway"
81+
params={{
82+
name: "Cohere",
83+
jsonexample: `
84+
{
85+
"model": "cohere/{model}"
86+
}`
87+
88+
}}
89+
90+
/>

‎src/content/docs/ai-gateway/providers/deepseek.mdx‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ sidebar:
66
text: Beta
77
---
88

9+
import { Render } from "~/components";
10+
911
[DeepSeek](https://www.deepseek.com/) helps you build quickly with DeepSeek's advanced AI models.
1012

1113
## Endpoint
@@ -80,3 +82,17 @@ try {
8082
return new Response(e);
8183
}
8284
```
85+
86+
<Render
87+
file="chat-completions-providers"
88+
product="ai-gateway"
89+
params={{
90+
name: "DeepSeek",
91+
jsonexample: `
92+
{
93+
"model": "deepseek/{model}"
94+
}`
95+
96+
}}
97+
98+
/>

‎src/content/docs/ai-gateway/providers/google-ai-studio.mdx‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@ title: Google AI Studio
33
pcx_content_type: get-started
44
---
55

6+
import { Render } from "~/components";
7+
68
[Google AI Studio](https://ai.google.dev/aistudio) helps you build quickly with Google Gemini models.
79

810
## Endpoint
@@ -69,3 +71,17 @@ const model = genAI.getGenerativeModel(
6971

7072
await model.generateContent(["What is Cloudflare?"]);
7173
```
74+
75+
<Render
76+
file="chat-completions-providers"
77+
product="ai-gateway"
78+
params={{
79+
name: "Google AI Studio",
80+
jsonexample: `
81+
{
82+
"model": "google-ai-studio/{model}"
83+
}`
84+
85+
}}
86+
87+
/>

‎src/content/docs/ai-gateway/providers/grok.mdx‎

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,9 @@ title: Grok
33
pcx_content_type: get-started
44
---
55

6-
[Grok](https://docs.x.ai/docs#getting-started) is a general purpose model that can be used for a variety of tasks, including generating and understanding text, code, and function calling.
6+
import { Render } from "~/components";
7+
8+
[Grok](https://docs.x.ai/docs#getting-started) is s a general purpose model that can be used for a variety of tasks, including generating and understanding text, code, and function calling.
79

810
## Endpoint
911

@@ -156,3 +158,17 @@ message = client.messages.create(
156158

157159
print(message.content)
158160
```
161+
162+
<Render
163+
file="chat-completions-providers"
164+
product="ai-gateway"
165+
params={{
166+
name: "Grok",
167+
jsonexample: `
168+
{
169+
"model": "grok/{model}"
170+
}`
171+
172+
}}
173+
174+
/>

‎src/content/docs/ai-gateway/providers/groq.mdx‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@ title: Groq
33
pcx_content_type: get-started
44
---
55

6+
import { Render } from "~/components";
7+
68
[Groq](https://groq.com/) delivers high-speed processing and low-latency performance.
79

810
## Endpoint
@@ -68,3 +70,17 @@ const chatCompletion = await groq.chat.completions.create({
6870
model,
6971
});
7072
```
73+
74+
<Render
75+
file="chat-completions-providers"
76+
product="ai-gateway"
77+
params={{
78+
name: "Groq",
79+
jsonexample: `
80+
{
81+
"model": "groq/{model}"
82+
}`
83+
84+
}}
85+
86+
/>

‎src/content/docs/ai-gateway/providers/index.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ title: Model providers
44
sidebar:
55
group:
66
hideIndex: true
7-
order: 3
7+
order: 4
88
---
99

1010
Here is a quick list of the providers we support:

0 commit comments

Comments
 (0)