You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
|**Images**| Images included in user messages, both as links or as base64-encoded data. The detail parameter must be set the same across requests. |`gpt-4o`<br/>`gpt-4o-mini` <br> `o1` (version 2024-12-17) |
80
-
|**Tool use**| Both the messages array and tool definitions. |`gpt-4o`<br/>`gpt-4o-mini`<br/>`gpt-4o-realtime-preview` (version 2024-12-17)<br/>`gpt-4o-mini-realtime-preview` (version 2024-12-17)<br> `o1` (version 2024-12-17) <br> `o3-mini` (version 2025-01-31) |
80
+
|**Tool use**| Both the messages array and tool definitions. |`gpt-4o`<br/>`gpt-4o-mini`<br/>`gpt-4o-realtime-preview` (version 2024-12-17)<br/>`gpt-4o-mini-realtime-preview` (version 2024-12-17)<br>`gpt-realtime` (version 2025-08-28)<br>`o1` (version 2024-12-17) <br> `o3-mini` (version 2025-01-31) |
81
81
|**Structured outputs**| Structured output schema is appended as a prefix to the system message. |`gpt-4o`<br/>`gpt-4o-mini` <br> `o1` (version 2024-12-17) <br> `o3-mini` (version 2025-01-31) |
82
82
83
83
To improve the likelihood of cache hits occurring, you should structure your requests such that repetitive content occurs at the beginning of the messages array.
Azure OpenAI GPT-4o Realtime API for speech and audio is part of the GPT-4o model family that supports low-latency, "speech in, speech out" conversational interactions.
20
19
@@ -33,6 +32,7 @@ Use the [Realtime API via WebSockets](./realtime-audio-websockets.md) if you nee
33
32
The GPT 4o real-time models are available for global deployments in [East US 2 and Sweden Central regions](../concepts/models.md#global-standard-model-availability).
34
33
-`gpt-4o-mini-realtime-preview` (2024-12-17)
35
34
-`gpt-4o-realtime-preview` (2024-12-17)
35
+
-`gpt-realtime` (version 2025-08-28)
36
36
37
37
You should use API version `2025-04-01-preview` in the URL for the Realtime API. The API version is included in the sessions URL.
38
38
@@ -44,7 +44,7 @@ Before you can use GPT-4o real-time audio, you need:
44
44
45
45
- An Azure subscription - <ahref="https://azure.microsoft.com/free/cognitive-services"target="_blank">Create one for free</a>.
46
46
- An Azure OpenAI resource created in a [supported region](#supported-models). For more information, see [Create a resource and deploy a model with Azure OpenAI](create-resource.md).
47
-
- You need a deployment of the `gpt-4o-realtime-preview` or `gpt-4o-mini-realtime-preview` model in a supported region as described in the [supported models](#supported-models) section in this article. You can deploy the model from the [Azure AI Foundry model catalog](../../../ai-foundry/how-to/model-catalog-overview.md) or from your project in Azure AI Foundry portal.
47
+
- You need a deployment of the `gpt-4o-realtime-preview`, `gpt-4o-mini-realtime-preview`, or `gpt-realtime` model in a supported region as described in the [supported models](#supported-models) section in this article. You can deploy the model from the [Azure AI Foundry model catalog](../../../ai-foundry/how-to/model-catalog-overview.md) or from your project in Azure AI Foundry portal.
Azure OpenAI GPT-4o Realtime API for speech and audio is part of the GPT-4o model family that supports low-latency, "speech in, speech out" conversational interactions.
20
19
@@ -30,6 +29,7 @@ Follow the instructions in this article to get started with the Realtime API via
30
29
The GPT-4o real-time models are available for global deployments in [East US 2 and Sweden Central regions](../concepts/models.md#global-standard-model-availability).
31
30
-`gpt-4o-mini-realtime-preview` (2024-12-17)
32
31
-`gpt-4o-realtime-preview` (2024-12-17)
32
+
-`gpt-realtime` (version 2025-08-28)
33
33
34
34
You should use API version `2025-04-01-preview` in the URL for the Realtime API.
35
35
@@ -41,7 +41,7 @@ Before you can use GPT-4o real-time audio, you need:
41
41
42
42
- An Azure subscription - <ahref="https://azure.microsoft.com/free/cognitive-services"target="_blank">Create one for free</a>.
43
43
- An Azure OpenAI resource created in a [supported region](#supported-models). For more information, see [Create a resource and deploy a model with Azure OpenAI](create-resource.md).
44
-
- You need a deployment of the `gpt-4o-realtime-preview` or `gpt-4o-mini-realtime-preview` model in a supported region as described in the [supported models](#supported-models) section. You can deploy the model from the [Azure AI Foundry portal model catalog](../../../ai-foundry/how-to/model-catalog-overview.md) or from your project in Azure AI Foundry portal.
44
+
- You need a deployment of the `gpt-4o-realtime-preview`, `gpt-4o-mini-realtime-preview`, or `gpt-realtime` model in a supported region as described in the [supported models](#supported-models) section. You can deploy the model from the [Azure AI Foundry portal model catalog](../../../ai-foundry/how-to/model-catalog-overview.md) or from your project in Azure AI Foundry portal.
45
45
46
46
## Connection and authentication
47
47
@@ -55,13 +55,23 @@ You can construct a full request URI by concatenating:
55
55
- Your Azure OpenAI resource endpoint hostname, for example, `my-aoai-resource.openai.azure.com`
56
56
- The `openai/realtime` API path.
57
57
- An `api-version` query string parameter for a supported API version such as `2024-12-17`
58
-
- A `deployment` query string parameter with the name of your `gpt-4o-realtime-preview` or `gpt-4o-mini-realtime-preview` model deployment.
58
+
- A `deployment` query string parameter with the name of your `gpt-4o-realtime-preview`, `gpt-4o-mini-realtime-preview`, or `gpt-realtime` model deployment.
59
59
60
60
The following example is a well-constructed `/realtime` request URI:
-**Microsoft Entra** (recommended): Use token-based authentication with the `/realtime` API for an Azure OpenAI resource with managed identity enabled. Apply a retrieved authentication token using a `Bearer` token with the `Authorization` header.
# How to use the GPT-4o Realtime API for speech and audio
18
16
19
17
Azure OpenAI GPT-4o Realtime API for speech and audio is part of the GPT-4o model family that supports low-latency, "speech in, speech out" conversational interactions. The GPT-4o Realtime API is designed to handle real-time, low-latency conversational interactions. Realtime API is a great fit for use cases involving live interactions between a user and a model, such as customer support agents, voice assistants, and real-time translators.
20
18
@@ -29,6 +27,7 @@ You can use the Realtime API via WebRTC or WebSocket to send audio input to the
29
27
The GPT 4o real-time models are available for global deployments in [East US 2 and Sweden Central regions](../concepts/models.md#global-standard-model-availability).
30
28
-`gpt-4o-mini-realtime-preview` (2024-12-17)
31
29
-`gpt-4o-realtime-preview` (2024-12-17)
30
+
-`gpt-realtime` (version 2025-08-28)
32
31
33
32
You should use API version `2025-04-01-preview` in the URL for the Realtime API.
34
33
@@ -40,10 +39,10 @@ Before you can use GPT-4o real-time audio, you need:
40
39
41
40
- An Azure subscription - <ahref="https://azure.microsoft.com/free/cognitive-services"target="_blank">Create one for free</a>.
42
41
- An Azure OpenAI resource created in a [supported region](#supported-models). For more information, see [Create a resource and deploy a model with Azure OpenAI](create-resource.md).
43
-
- You need a deployment of the `gpt-4o-realtime-preview` or `gpt-4o-mini-realtime-preview` model in a supported region as described in the [supported models](#supported-models) section. You can deploy the model from the [Azure AI Foundry portal model catalog](../../../ai-foundry/how-to/model-catalog-overview.md) or from your project in Azure AI Foundry portal.
42
+
- You need a deployment of the `gpt-4o-realtime-preview`, `gpt-4o-mini-realtime-preview`, or `gpt-realtime` model in a supported region as described in the [supported models](#supported-models) section. You can deploy the model from the [Azure AI Foundry portal model catalog](../../../ai-foundry/how-to/model-catalog-overview.md) or from your project in Azure AI Foundry portal.
44
43
45
44
Here are some of the ways you can get started with the GPT-4o Realtime API for speech and audio:
46
-
- For steps to deploy and use the `gpt-4o-realtime-preview` or `gpt-4o-mini-realtime-preview` model, see [the real-time audio quickstart](../realtime-audio-quickstart.md).
45
+
- For steps to deploy and use the `gpt-4o-realtime-preview`, `gpt-4o-mini-realtime-preview`, or `gpt-realtime` model, see [the real-time audio quickstart](../realtime-audio-quickstart.md).
47
46
- Try the [WebRTC via HTML and JavaScript example](./realtime-audio-webrtc.md#webrtc-example-via-html-and-javascript) to get started with the Realtime API via WebRTC.
48
47
-[The Azure-Samples/aisearch-openai-rag-audio repo](https://github.com/Azure-Samples/aisearch-openai-rag-audio) contains an example of how to implement RAG support in applications that use voice as their user interface, powered by the GPT-4o realtime API for audio.
Copy file name to clipboardExpand all lines: articles/ai-foundry/openai/includes/realtime-deploy-model.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,12 +7,12 @@ ms.topic: include
7
7
ms.date: 1/21/2025
8
8
---
9
9
10
-
To deploy the `gpt-4o-realtime` model in the Azure AI Foundry portal:
10
+
To deploy the `gpt-realtime` model in the Azure AI Foundry portal:
11
11
1. Go to the [Azure AI Foundry portal](https://ai.azure.com/?cid=learnDocs) and create or select your project.
12
12
1. Select **Models + endpoints** from under **My assets** in the left pane.
13
13
1. Select **+ Deploy model** > **Deploy base model** to open the deployment window.
14
-
1. Search for and select the `gpt-4o-realtime` model and then select **Confirm**.
14
+
1. Search for and select the `gpt-realtime` model and then select **Confirm**.
15
15
1. Review the deployment details and select **Deploy**.
16
16
1. Follow the wizard to finish deploying the model.
17
17
18
-
Now that you have a deployment of the `gpt-4o-realtime` model, you can interact with it in the Azure AI Foundry portal **Audio** playground or Realtime API.
18
+
Now that you have a deployment of the `gpt-realtime` model, you can interact with it in the Azure AI Foundry portal **Audio** playground or Realtime API.
Copy file name to clipboardExpand all lines: articles/ai-foundry/openai/includes/realtime-javascript.md
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -12,7 +12,7 @@ ms.date: 3/20/2025
12
12
- An Azure subscription - <ahref="https://azure.microsoft.com/free/cognitive-services"target="_blank">Create one for free</a>
13
13
- <ahref="https://nodejs.org/"target="_blank">Node.js LTS or ESM support.</a>
14
14
- An Azure OpenAI resource created in one of the supported regions. For more information about region availability, see the [models and versions documentation](../concepts/models.md#global-standard-model-availability).
15
-
- Then, you need to deploy a `gpt-4o-realtime` model with your Azure OpenAI resource. For more information, see [Create a resource and deploy a model with Azure OpenAI](../how-to/create-resource.md).
15
+
- Then, you need to deploy a `gpt-realtime` model with your Azure OpenAI resource. For more information, see [Create a resource and deploy a model with Azure OpenAI](../how-to/create-resource.md).
16
16
17
17
### Microsoft Entra ID prerequisites
18
18
@@ -72,7 +72,7 @@ For the recommended keyless authentication with Microsoft Entra ID, you need to:
72
72
// You will need to set these environment variables or edit the following values
Copy file name to clipboardExpand all lines: articles/ai-foundry/openai/includes/realtime-portal.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,16 +13,16 @@ ms.date: 3/20/2025
13
13
14
14
## Use the GPT-4o real-time audio
15
15
16
-
To chat with your deployed `gpt-4o-realtime` model in the [Azure AI Foundry](https://ai.azure.com/?cid=learnDocs)**Real-time audio** playground, follow these steps:
16
+
To chat with your deployed `gpt-realtime` model in the [Azure AI Foundry](https://ai.azure.com/?cid=learnDocs)**Real-time audio** playground, follow these steps:
17
17
18
-
1. Go to the [Azure AI Foundry portal](https://ai.azure.com/?cid=learnDocs) and select your project that has your deployed `gpt-4o-realtime` model.
18
+
1. Go to the [Azure AI Foundry portal](https://ai.azure.com/?cid=learnDocs) and select your project that has your deployed `gpt-realtime` model.
19
19
1. Select **Playgrounds** from the left pane.
20
20
1. Select **Audio playground** > **Try the Audio playground**.
21
21
22
22
> [!NOTE]
23
-
> The **Chat playground** doesn't support the `gpt-4o-realtime` model. Use the **Audio playground** as described in this section.
23
+
> The **Chat playground** doesn't support the `gpt-realtime` model. Use the **Audio playground** as described in this section.
24
24
25
-
1. Select your deployed `gpt-4o-realtime` model from the **Deployment** dropdown.
25
+
1. Select your deployed `gpt-realtime` model from the **Deployment** dropdown.
26
26
27
27
<!--:::image type="content" source="../media/how-to/real-time/real-time-playground.png" alt-text="Screenshot of the audio playground with the deployed model selected." lightbox="../media/how-to/real-time/real-time-playground.png":::-->
Copy file name to clipboardExpand all lines: articles/ai-foundry/openai/includes/realtime-python.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -12,7 +12,7 @@ ms.date: 3/20/2025
12
12
- An Azure subscription. <ahref="https://azure.microsoft.com/free/ai-services"target="_blank">Create one for free</a>.
13
13
- <ahref="https://www.python.org/"target="_blank">Python 3.8 or later version</a>. We recommend using Python 3.10 or later, but having at least Python 3.8 is required. If you don't have a suitable version of Python installed, you can follow the instructions in the [VS Code Python Tutorial](https://code.visualstudio.com/docs/python/python-tutorial#_install-a-python-interpreter) for the easiest way of installing Python on your operating system.
14
14
- An Azure OpenAI resource created in one of the supported regions. For more information about region availability, see the [models and versions documentation](../concepts/models.md#global-standard-model-availability).
15
-
- Then, you need to deploy a `gpt-4o-realtime` model with your Azure OpenAI resource. For more information, see [Create a resource and deploy a model with Azure OpenAI](../how-to/create-resource.md).
15
+
- Then, you need to deploy a `gpt-realtime` model with your Azure OpenAI resource. For more information, see [Create a resource and deploy a model with Azure OpenAI](../how-to/create-resource.md).
16
16
17
17
## Microsoft Entra ID prerequisites
18
18
@@ -112,7 +112,7 @@ For the recommended keyless authentication with Microsoft Entra ID, you need to:
112
112
api_version="2025-08-28",
113
113
)
114
114
async with client.beta.realtime.connect(
115
-
model="gpt-4o-realtime", # name of your deployment
0 commit comments