Skip to content

Commit b72240d

Browse files
daisyfaithaumadaisyfaithauma
committed
Websockets Docs
1 parent b6868eb commit b72240d

File tree

3 files changed

+162
-13
lines changed

3 files changed

+162
-13
lines changed

‎src/content/docs/ai-gateway/configuration/websockets-api/index.mdx‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
---
2+
title: WebSockets API
3+
pcx_content_type: configuration
4+
sidebar:
5+
group:
6+
badge: Beta
7+
---
8+
9+
The AI Gateway WebSockets API provides a persistent connection for AI interactions, eliminating repeated handshakes and reducing latency. This API is divided into two categories:
10+
11+
1. **Non-Realtime APIs** - Supports standard WebSocket communication for AI providers, including those that do not natively support WebSockets.
12+
2. **Realtime APIs** - Designed for AI providers that offer low-latency, multimodal interactions over WebSockets.
13+
14+
## **Key differences**
15+
16+
| Feature | Non-Realtime APIs | Realtime APIs |
17+
| :---------------------- | :----------------------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------- |
18+
| **Purpose** | Supports WebSocket-based AI interactions with providers that do not natively support WebSockets. | Enables real-time, multimodal AI interactions for providers that offer dedicated WebSocket endpoints. |
19+
| **Use Case** | Text-based queries and responses, such as LLM requests. | Streaming responses for voice, video, and live interactions. |
20+
| **AI Provider Support** | All AI providers in AI Gateway. | Limited to providers offering real-time WebSocket APIs. |
21+
| **Streaming Support** | AI Gateway handles streaming via WebSockets. | Providers natively support real-time data streaming. |
22+
23+
For details on implementation, see the next section:
24+
25+
- [Non-Realtime WebSockets API](/ai-gateway/configuration/websockets-api/non-realtime-api.mdx)
26+
- [Realtime WebSockets API](/ai-gateway/configuration/websockets-api/realtime-api.mdx)

‎src/content/docs/ai-gateway/configuration/websockets-api.mdx‎ renamed to ‎src/content/docs/ai-gateway/configuration/websockets-api/non-realtime-api.mdx‎

Lines changed: 3 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,11 @@
11
---
2-
title: WebSockets API
32
pcx_content_type: configuration
3+
title: Non-realtime WebSockets API
44
sidebar:
5-
badge:
6-
text: Beta
5+
order: 2
76
---
87

9-
The AI Gateway WebSockets API provides a single persistent connection, enabling continuous communication. By using WebSockets, you can establish a single connection for multiple AI requests, eliminating the need for repeated handshakes and TLS negotiations, which enhances performance and reduces latency. This API supports all AI providers connected to AI Gateway, including those that do not natively support WebSockets.
10-
11-
## When to use WebSockets?
12-
13-
WebSockets are long-lived TCP connections that enable bi-directional, real-time communication between client and server. Unlike HTTP connections, which require repeated handshakes for each request, WebSockets maintain the connection, supporting continuous data exchange with reduced overhead. WebSockets are ideal for applications needing low-latency, real-time data, such as voice assistants.
14-
15-
## Key benefits
16-
17-
- **Reduced Overhead**: Avoid overhead of repeated handshakes and TLS negotiations by maintaining a single, persistent connection.
18-
- **Provider Compatibility**: Works with all AI providers in AI Gateway. Even if your chosen provider does not support WebSockets, we handle it for you, managing the requests to your preferred AI provider.
8+
The Non-realtime WebSockets API allows you to establish persistent connections for AI requests without requiring repeated handshakes. This approach is ideal for applications that do not require real-time interactions but still benefit from reduced latency and continuous communication.
199

2010
## Set up WebSockets API
2111

‎src/content/docs/ai-gateway/configuration/websockets-api/realtime-api.mdx‎

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
---
2+
pcx_content_type: configuration
3+
title: Realtime WebSockets API
4+
sidebar:
5+
order: 3
6+
---
7+
8+
Some AI providers support real-time, low-latency interactions over WebSockets. AI Gateway allows seamless integration with these APIs, supporting multimodal interactions such as text, audio, and video.
9+
10+
## Supported Providers
11+
12+
- [OpenAI](https://platform.openai.com/docs/guides/realtime-websocket)
13+
- [Google AI Studio](https://ai.google.dev/gemini-api/docs/multimodal-live)
14+
- [Cartesia](https://docs.cartesia.ai/api-reference/tts/tts)
15+
- [ElevenLabs](https://elevenlabs.io/docs/conversational-ai/api-reference/conversational-ai/websocket)
16+
17+
## Authentication
18+
19+
For real-time WebSockets, authentication can be done using:
20+
21+
- Headers (for non-browser environments)
22+
- `sec-websocket-protocol` (for browsers)
23+
24+
## Examples
25+
26+
### OpenAI
27+
28+
```javascript
29+
import WebSocket from "ws";
30+
31+
const url =
32+
"wss://gateway.ai.cloudflare.com/v1/<account_id>/<gateway>/openai?model=gpt-4o-realtime-preview-2024-12-17";
33+
const ws = new WebSocket(url, {
34+
headers: {
35+
"cf-aig-authorization": process.env.CLOUDFLARE_API_KEY,
36+
Authorization: "Bearer " + process.env.OPENAI_API_KEY,
37+
"OpenAI-Beta": "realtime=v1",
38+
},
39+
});
40+
41+
ws.on("open", () => console.log("Connected to server."));
42+
ws.on("message", (message) => console.log(JSON.parse(message.toString())));
43+
44+
ws.send(
45+
JSON.stringify({
46+
type: "response.create",
47+
response: { modalities: ["text"], instructions: "Tell me a joke" },
48+
}),
49+
);
50+
```
51+
52+
### Google AI Studio
53+
54+
```javascript
55+
const ws = new WebSocket(
56+
"wss://gateway.ai.cloudflare.com/v1/<account_id>/<gateway>/google?api_key=<google_api_key>",
57+
["cf-aig-authorization.<cloudflare_token>"],
58+
);
59+
60+
ws.on("open", () => console.log("Connected to server."));
61+
ws.on("message", (message) => console.log(message.data));
62+
63+
ws.send(
64+
JSON.stringify({
65+
setup: {
66+
model: "models/gemini-2.0-flash-exp",
67+
generationConfig: { responseModalities: ["TEXT"] },
68+
},
69+
}),
70+
);
71+
```
72+
73+
### Cartesia
74+
75+
```javascript
76+
const ws = new WebSocket(
77+
"wss://gateway.ai.cloudflare.com/v1/<account_id>/<gateway>/cartesia?cartesia_version=2024-06-10&api_key=<cartesia_api_key>",
78+
["cf-aig-authorization.<cloudflare_token>"],
79+
);
80+
81+
ws.on("open", function open() {
82+
console.log("Connected to server.");
83+
});
84+
85+
ws.on("message", function incoming(message) {
86+
console.log(message.data);
87+
});
88+
89+
ws.send(
90+
JSON.stringify({
91+
model_id: "sonic",
92+
transcript: "Hello, world! I'm generating audio on ",
93+
voice: { mode: "id", id: "a0e99841-438c-4a64-b679-ae501e7d6091" },
94+
language: "en",
95+
context_id: "happy-monkeys-fly",
96+
output_format: {
97+
container: "raw",
98+
encoding: "pcm_s16le",
99+
sample_rate: 8000,
100+
},
101+
add_timestamps: true,
102+
continue: true,
103+
}),
104+
);
105+
```
106+
107+
### ElevenLabs
108+
109+
```javascript
110+
const ws = new WebSocket(
111+
"wss://gateway.ai.cloudflare.com/v1/<account_id>/<gateway>/elevenlabs?agent_id=<elevenlabs_agent_id>",
112+
[
113+
"xi-api-key.<elevenlabs_api_key>",
114+
"cf-aig-authorization.<cloudflare_token>",
115+
],
116+
);
117+
118+
ws.on("open", function open() {
119+
console.log("Connected to server.");
120+
});
121+
122+
ws.on("message", function incoming(message) {
123+
console.log(message.data);
124+
});
125+
126+
ws.send(
127+
JSON.stringify({
128+
text: "This is a sample text ",
129+
voice_settings: { stability: 0.8, similarity_boost: 0.8 },
130+
generation_config: { chunk_length_schedule: [120, 160, 250, 290] },
131+
}),
132+
);
133+
```

0 commit comments

Comments
 (0)