Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
title: WebSockets API
pcx_content_type: configuration
sidebar:
group:
badge: Beta
---

The AI Gateway WebSockets API provides a persistent connection for AI interactions, eliminating repeated handshakes and reducing latency. This API is divided into two categories:

1. **Non-Realtime APIs** - Supports standard WebSocket communication for AI providers, including those that do not natively support WebSockets.
2. **Realtime APIs** - Designed for AI providers that offer low-latency, multimodal interactions over WebSockets.

## When to use WebSockets?

WebSockets are long-lived TCP connections that enable bi-directional, real-time and non realtime communication between client and server. Unlike HTTP connections, which require repeated handshakes for each request, WebSockets maintain the connection, supporting continuous data exchange with reduced overhead. WebSockets are ideal for applications needing low-latency, real-time data, such as voice assistants.

## Key benefits

- **Reduced Overhead**: Avoid overhead of repeated handshakes and TLS negotiations by maintaining a single, persistent connection.
- **Provider Compatibility**: Works with all AI providers in AI Gateway. Even if your chosen provider does not support WebSockets, we handle it for you, managing the requests to your preferred AI provider.

## **Key differences**

| Feature | Realtime APIs | Non-Realtime APIs |
| :---------------------- | :------------------------------------------------------------------------------------------------------------------------------------ | :----------------------------------------------------------------------------------------------- |
| **Purpose** | Enables real-time, multimodal AI interactions for providers that offer dedicated WebSocket endpoints. | Supports WebSocket-based AI interactions with providers that do not natively support WebSockets. |
| **Use Case** | Streaming responses for voice, video, and live interactions. | Text-based queries and responses, such as LLM requests. |
| **AI Provider Support** | [Limited to providers offering real-time WebSocket APIs.](/ai-gateway/configuration/websockets-api/realtime-api/#supported-providers) | [All AI providers in AI Gateway.](/ai-gateway/providers) |
| **Streaming Support** | Providers natively support real-time data streaming. | AI Gateway handles streaming via WebSockets. |

For details on implementation, refer to the next sections:

- [Realtime WebSockets API](/ai-gateway/configuration/websockets-api/realtime-api/)
- [Non-Realtime WebSockets API](/ai-gateway/configuration/websockets-api/non-realtime-api/)
4 changes: 4 additions & 0 deletions src/content/release-notes/ai-gateway.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ productLink: "/ai-gateway/"
productArea: Developer platform
productAreaLink: /workers/platform/changelog/platform/
entries:
- publish_date: "2025-03-18"
title: WebSockets
description: |-
Added [WebSockets API](/ai-gateway/configuration/websockets-api/) provide a persistent connection for AI interactions, eliminating repeated handshakes and reducing latency.
- publish_date: "2025-02-26"
title: Guardrails
description: |-
Expand Down
Loading