From 13e255646c800556885bcc2994bf46c08aaf2732 Mon Sep 17 00:00:00 2001 From: Kathy <153706637+kathayl@users.noreply.github.com> Date: Thu, 20 Mar 2025 13:15:04 -0700 Subject: [PATCH 1/5] Create 2025-03-20-websockets.mdx --- .../ai-gateway/2025-03-20-websockets.mdx | 37 +++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100644 src/content/changelog/ai-gateway/2025-03-20-websockets.mdx diff --git a/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx b/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx new file mode 100644 index 00000000000000..a7a2a83d792051 --- /dev/null +++ b/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx @@ -0,0 +1,37 @@ +--- +title: AI Gateway launches Realtime WebSockets API +description: AI Gateway now supports end-to-end, client-to-provider WebSockets +date: 2025-03-20T14:00:00Z +--- +We are excited to announce that [AI Gateway](/ai-gateway/) now supports real-time AI interactions with the new [Realtime WebSockets API](/configuration/websockets-api/realtime-api/). + +This new capability allows developers to establish persistent, low-latency connections between their applications and AI models, reducing latency and improving user experiences for applications that require dynamic, real-time interactions. + +The Realtime WebSockets API works with the [OpenAI Realtime API](https://platform.openai.com/docs/guides/realtime#connect-with-websockets), [Google Gemini Live API](https://ai.google.dev/gemini-api/docs/multimodal-live), and supports real-time text and speech interactions with models from [Cartesia](https://docs.cartesia.ai/api-reference/tts/tts), and [ElevenLabs](https://elevenlabs.io/docs/conversational-ai/api-reference/conversational-ai/websocket). + +Here's how you can connect AI Gateway to [OpenAI's Realtime API](https://platform.openai.com/docs/guides/realtime#connect-with-websockets) using WebSockets: +```bash title="OpenAI Realtime API example" +import WebSocket from "ws"; + +const url = + "wss://gateway.ai.cloudflare.com/v1///openai?model=gpt-4o-realtime-preview-2024-12-17"; +const ws = new WebSocket(url, { + headers: { + "cf-aig-authorization": process.env.CLOUDFLARE_API_KEY, + Authorization: "Bearer " + process.env.OPENAI_API_KEY, + "OpenAI-Beta": "realtime=v1", + }, +}); + +ws.on("open", () => console.log("Connected to server.")); +ws.on("message", (message) => console.log(JSON.parse(message.toString()))); + +ws.send( + JSON.stringify({ + type: "response.create", + response: { modalities: ["text"], instructions: "Tell me a joke" }, + }), +); +``` + +Get started by checking out the [Realtime WebSockets API](/configuration/websockets-api/realtime-api/) documentation. From 4b6de0a7560a82578888c9fbbdeb47ec6ae447f4 Mon Sep 17 00:00:00 2001 From: Kathy <153706637+kathayl@users.noreply.github.com> Date: Thu, 20 Mar 2025 13:31:44 -0700 Subject: [PATCH 2/5] Update 2025-03-20-websockets.mdx wording changes --- src/content/changelog/ai-gateway/2025-03-20-websockets.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx b/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx index a7a2a83d792051..ef12a5fc4739d2 100644 --- a/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx +++ b/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx @@ -5,7 +5,7 @@ date: 2025-03-20T14:00:00Z --- We are excited to announce that [AI Gateway](/ai-gateway/) now supports real-time AI interactions with the new [Realtime WebSockets API](/configuration/websockets-api/realtime-api/). -This new capability allows developers to establish persistent, low-latency connections between their applications and AI models, reducing latency and improving user experiences for applications that require dynamic, real-time interactions. +This new capability allows developers to establish persistent, low-latency connections between their applications and AI models, enabling natural, real-time conversational AI experiences, including speech-to-speech interactions. The Realtime WebSockets API works with the [OpenAI Realtime API](https://platform.openai.com/docs/guides/realtime#connect-with-websockets), [Google Gemini Live API](https://ai.google.dev/gemini-api/docs/multimodal-live), and supports real-time text and speech interactions with models from [Cartesia](https://docs.cartesia.ai/api-reference/tts/tts), and [ElevenLabs](https://elevenlabs.io/docs/conversational-ai/api-reference/conversational-ai/websocket). From 3e483383e76c39305398c18285f1d55a79653424 Mon Sep 17 00:00:00 2001 From: Kody Jackson Date: Thu, 20 Mar 2025 15:56:16 -0500 Subject: [PATCH 3/5] Apply suggestions from code review --- src/content/changelog/ai-gateway/2025-03-20-websockets.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx b/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx index ef12a5fc4739d2..e839cc05d16c93 100644 --- a/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx +++ b/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx @@ -3,7 +3,7 @@ title: AI Gateway launches Realtime WebSockets API description: AI Gateway now supports end-to-end, client-to-provider WebSockets date: 2025-03-20T14:00:00Z --- -We are excited to announce that [AI Gateway](/ai-gateway/) now supports real-time AI interactions with the new [Realtime WebSockets API](/configuration/websockets-api/realtime-api/). +We are excited to announce that [AI Gateway](/ai-gateway/) now supports real-time AI interactions with the new [Realtime WebSockets API](/ai-gateway/configuration/websockets-api/realtime-api/). This new capability allows developers to establish persistent, low-latency connections between their applications and AI models, enabling natural, real-time conversational AI experiences, including speech-to-speech interactions. @@ -34,4 +34,4 @@ ws.send( ); ``` -Get started by checking out the [Realtime WebSockets API](/configuration/websockets-api/realtime-api/) documentation. +Get started by checking out the [Realtime WebSockets API](/ai-gateway/configuration/websockets-api/realtime-api/) documentation. From 388383148c1579717c5c445447349613503ca49e Mon Sep 17 00:00:00 2001 From: Kathy <153706637+kathayl@users.noreply.github.com> Date: Thu, 20 Mar 2025 15:24:47 -0700 Subject: [PATCH 4/5] Update src/content/changelog/ai-gateway/2025-03-20-websockets.mdx Co-authored-by: Justin Hutchings --- src/content/changelog/ai-gateway/2025-03-20-websockets.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx b/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx index e839cc05d16c93..5896ffebb30518 100644 --- a/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx +++ b/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx @@ -10,7 +10,7 @@ This new capability allows developers to establish persistent, low-latency conne The Realtime WebSockets API works with the [OpenAI Realtime API](https://platform.openai.com/docs/guides/realtime#connect-with-websockets), [Google Gemini Live API](https://ai.google.dev/gemini-api/docs/multimodal-live), and supports real-time text and speech interactions with models from [Cartesia](https://docs.cartesia.ai/api-reference/tts/tts), and [ElevenLabs](https://elevenlabs.io/docs/conversational-ai/api-reference/conversational-ai/websocket). Here's how you can connect AI Gateway to [OpenAI's Realtime API](https://platform.openai.com/docs/guides/realtime#connect-with-websockets) using WebSockets: -```bash title="OpenAI Realtime API example" +```javascript title="OpenAI Realtime API example" import WebSocket from "ws"; const url = From 2fa071bdf45dc793a333d4fed92427bcf5af33ee Mon Sep 17 00:00:00 2001 From: Kathy <153706637+kathayl@users.noreply.github.com> Date: Fri, 21 Mar 2025 09:18:57 -0700 Subject: [PATCH 5/5] Update 2025-03-20-websockets.mdx update date --- src/content/changelog/ai-gateway/2025-03-20-websockets.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx b/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx index 5896ffebb30518..5c9a63ac90fc26 100644 --- a/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx +++ b/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx @@ -1,7 +1,7 @@ --- title: AI Gateway launches Realtime WebSockets API description: AI Gateway now supports end-to-end, client-to-provider WebSockets -date: 2025-03-20T14:00:00Z +date: 2025-03-21T09:00:00Z --- We are excited to announce that [AI Gateway](/ai-gateway/) now supports real-time AI interactions with the new [Realtime WebSockets API](/ai-gateway/configuration/websockets-api/realtime-api/).