You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Support for containers is currently available with Document Intelligence version `2022-08-31 (GA)` for all models and `2023-07-31 (GA)` for Read, Layout, ID Document, Receipt, and Invoice models:
23
-
24
-
*[REST API `2022-08-31 (GA)`](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-v3.0%20(2022-08-31)&preserve-view=true&tabs=HTTP)
25
-
*[REST API `2023-07-31 (GA)`](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-v3.1%20(2023-07-31)&tabs=HTTP&preserve-view=true)
26
-
*[Client libraries targeting `REST API 2022-08-31 (GA)`](../sdk-overview-v3-0.md)
27
-
*[Client libraries targeting `REST API 2023-07-31 (GA)`](../sdk-overview-v3-1.md)
28
-
29
-
✔️ See [**Install and run Document Intelligence containers**](?view=doc-intel-3.1.0&preserve-view=true) for supported container documentation.
Azure AI Document Intelligence is an Azure AI service that lets you build automated data processing software using machine-learning technology. Document Intelligence enables you to identify and extract text, key/value pairs, selection marks, table data, and more from your documents. The results are delivered as structured data that ../includes the relationships in the original file.
38
24
39
-
In this article you learn how to download, install, and run Document Intelligence containers. Containers enable you to run the Document Intelligence service in your own environment. Containers are great for specific security and data governance requirements.
25
+
In this article you can learn how to download, install, and run Document Intelligence containers. Containers enable you to run the Document Intelligence service in your own environment. Containers are great for specific security and data governance requirements.
40
26
41
27
***Read**, **Layout**, **ID Document**, **Receipt**, and **Invoice** models are supported by Document Intelligence v3.1 containers.
42
28
43
29
***Read**, **Layout**, **General Document**, **Business Card**, and **Custom** models are supported by Document Intelligence v3.0 containers.
44
30
31
+
## Version support
32
+
33
+
Support for containers is currently available with Document Intelligence version `v3.0: 2022-08-31 (GA)` for all models and `v3.1 2023-07-31 (GA)` for Read, Layout, ID Document, Receipt, and Invoice models:
34
+
35
+
*[REST API `v3.0: 2022-08-31 (GA)`](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-v3.0%20(2022-08-31)&preserve-view=true&tabs=HTTP)
36
+
*[REST API `v3.1: 2023-07-31 (GA)`](/rest/api/aiservices/document-models/analyze-document?view=rest-aiservices-v3.1%20(2023-07-31)&tabs=HTTP&preserve-view=true)
37
+
*[Client libraries targeting `REST API v3.0: 2022-08-31 (GA)`](../sdk-overview-v3-0.md)
38
+
*[Client libraries targeting `REST API v3.1: 2023-07-31 (GA)`](../sdk-overview-v3-1.md)
39
+
45
40
## Prerequisites
46
41
47
42
To get started, you need an active [**Azure account**](https://azure.microsoft.com/free/cognitive-services/). If you don't have one, you can [**create a free account**](https://azure.microsoft.com/free/).
<sup>1</sup> For Azure OpenAI models, only customers who have been approved for modified content filtering have full content filtering control and can turn off content filters. Apply for modified content filters via this form: [Azure OpenAI Limited Access Review: Modified Content Filters](https://ncv.microsoft.com/uEfCgnITdR) For Azure Government customers, please apply for modified content filters via this form: [Azure Government - Request Modified Content Filtering for Azure OpenAI Service](https://aka.ms/AOAIGovModifyContentFilter).
88
+
89
+
Configurable content filters for inputs (prompts) and outputs (completions) are available for the following Azure OpenAI models:
Configurable content filters are currently not available for
98
+
99
+
`o1-preview` and `o1-mini`
100
+
101
+
<sup>*</sup>Only available for GPT-4 Turbo Vision GA, does not apply to GPT-4 Turbo Vision preview
102
+
103
+
Content filtering configurations are created within a Resource in Azure AI Studio, and can be associated with Deployments. [Learn more about configurability here](../how-to/content-filters.md).
104
+
105
+
Customers are responsible for ensuring that applications integrating Azure OpenAI comply with the [Code of Conduct](/legal/cognitive-services/openai/code-of-conduct?context=%2Fazure%2Fai-services%2Fopenai%2Fcontext%2Fcontext).
87
106
88
107
## Scenario details
89
108
@@ -1012,4 +1031,4 @@ As part of your application design, consider the following best practices to del
1012
1031
- Apply for modified content filters via [this form](https://ncv.microsoft.com/uEfCgnITdR).
1013
1032
- Azure OpenAI content filtering is powered by [Azure AI Content Safety](https://azure.microsoft.com/products/cognitive-services/ai-content-safety).
1014
1033
- Learn more about understanding and mitigating risks associated with your application: [Overview of Responsible AI practices for Azure OpenAI models](/legal/cognitive-services/openai/overview?context=/azure/ai-services/openai/context/context).
1015
-
- Learn more about how data is processed in connection with content filtering and abuse monitoring: [Data, privacy, and security for Azure OpenAI Service](/legal/cognitive-services/openai/data-privacy?context=/azure/ai-services/openai/context/context#preventing-abuse-and-harmful-content-generation).
1034
+
- Learn more about how data is processed in connection with content filtering and abuse monitoring: [Data, privacy, and security for Azure OpenAI Service](/legal/cognitive-services/openai/data-privacy?context=/azure/ai-services/openai/context/context#preventing-abuse-and-harmful-content-generation).
@@ -33,16 +33,17 @@ The Azure OpenAI `o1-preview` and `o1-mini` models are specifically designed to
33
33
34
34
### Availability
35
35
36
-
The `o1-preview` and `o1-mini` models are available in the East US2 region for limited access through the [AI Studio](https://ai.azure.com) early access playground. Data processing for the `o1` models may occur in a different region than where they are available for use.
37
-
38
-
To try the `o1-preview` and `o1-mini` models in the early access playground, **registration is required, and access will be granted based on Microsoft’s eligibility criteria**.
36
+
The `o1-preview` and `o1-mini` models are now available for API access and model deployment. **Registration is required, and access will be granted based on Microsoft's eligibility criteria**.
39
37
40
38
Request access: [limited access model application](https://aka.ms/oai/modelaccess)
41
39
42
-
Once access has been granted, you will need to:
40
+
Once access has been granted, you will need to create a deployment for each model.
41
+
42
+
### API support
43
+
44
+
Support for the **o1 series** models was added in API version `2024-09-01-preview`.
43
45
44
-
1. Navigate to https://ai.azure.com/resources and select a resource in the `eastus2` region. If you do not have an Azure OpenAI resource in this region you will need to [create one](https://portal.azure.com/#create/Microsoft.CognitiveServicesOpenAI).
45
-
2. Once the `eastus2` Azure OpenAI resource is selected, in the upper left-hand panel under **Playgrounds** select **Early access playground (preview)**.
46
+
The `max_tokens` parameter has been deprecated and replaced with the new `max_completion_tokens` parameter. **o1 series** models will only work with the `max_completions_tokens` parameter.
46
47
47
48
## GPT-4o audio
48
49
@@ -98,6 +99,8 @@ See [model versions](../concepts/model-versions.md) to learn about how Azure Ope
98
99
99
100
| Model ID | Description | Max Request (tokens) | Training Data (up to) |
100
101
| --- | :--- |:--- |:---: |
102
+
|`o1-preview` (2024-09-12) | The most capable model in the o1 series, offering enhanced reasoning abilities.| Input: 128,000 <br> Output: 32,768 | Oct 2023 |
103
+
|`o1-mini` (2024-09-12) | A faster and more cost-efficient option in the o1 series, ideal for coding tasks requiring speed and lower resource consumption.| Input: 128,000 <br> Output: 65,536 | Oct 2023 |
101
104
|`gpt-4o` (2024-08-06) <br> **GPT-4o (Omni)**|**Latest large GA model** <br> - Structured outputs<br> - Text, image processing <br> - JSON Mode <br> - parallel function calling <br> - Enhanced accuracy and responsiveness <br> - Parity with English text and coding tasks compared to GPT-4 Turbo with Vision <br> - Superior performance in non-English languages and in vision tasks |Input: 128,000 <br> Output: 16,384 | Oct 2023 |
102
105
|`gpt-4o-mini` (2024-07-18) <br> **GPT-4o mini**|**Latest small GA model** <br> - Fast, inexpensive, capable model ideal for replacing GPT-3.5 Turbo series models. <br> - Text, image processing <br>- JSON Mode <br> - parallel function calling | Input: 128,000 <br> Output: 16,384 | Oct 2023 |
103
106
|`gpt-4o` (2024-05-13) <br> **GPT-4o (Omni)**| Text, image processing <br> - JSON Mode <br> - parallel function calling <br> - Enhanced accuracy and responsiveness <br> - Parity with English text and coding tasks compared to GPT-4 Turbo with Vision <br> - Superior performance in non-English languages and in vision tasks |Input: 128,000 <br> Output: 4,096| Oct 2023 |
@@ -421,4 +424,4 @@ For the latest information on model retirements, refer to the [model retirement
421
424
-[Model retirement and deprecation](./model-retirements.md)
422
425
-[Learn more about working with Azure OpenAI models](../how-to/working-with-models.md)
423
426
-[Learn more about Azure OpenAI](../overview.md)
424
-
-[Learn more about fine-tuning Azure OpenAI models](../how-to/fine-tuning.md)
427
+
-[Learn more about fine-tuning Azure OpenAI models](../how-to/fine-tuning.md)
Copy file name to clipboardExpand all lines: articles/ai-services/openai/how-to/audio-real-time.md
+22-13Lines changed: 22 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
---
2
-
title: 'How to use GPT-4o real-time audio with Azure OpenAI Service'
2
+
title: 'How to use GPT-4o Realtime API for speech and audio with Azure OpenAI Service'
3
3
titleSuffix: Azure OpenAI
4
-
description: Learn how to use GPT-4o real-time audio with Azure OpenAI Service.
4
+
description: Learn how to use GPT-4o Realtime API for speech and audio with Azure OpenAI Service.
5
5
manager: nitinme
6
6
ms.service: azure-ai-openai
7
7
ms.topic: how-to
@@ -12,11 +12,11 @@ ms.custom: references_regions
12
12
recommendations: false
13
13
---
14
14
15
-
# GPT-4o real-time audio
15
+
# GPT-4o Realtime API for speech and audio
16
16
17
-
Azure OpenAI GPT-4o audio is part of the GPT-4o model family that supports low-latency, "speech in, speech out" conversational interactions. The GPT-4o audio `realtime` API is designed to handle real-time, low-latency conversational interactions, making it a great fit for use cases involving live interactions between a user and a model, such as customer support agents, voice assistants, and real-time translators.
17
+
Azure OpenAI GPT-4o Realtime API for speech and audio is part of the GPT-4o model family that supports low-latency, "speech in, speech out" conversational interactions. The GPT-4o audio `realtime` API is designed to handle real-time, low-latency conversational interactions, making it a great fit for use cases involving live interactions between a user and a model, such as customer support agents, voice assistants, and real-time translators.
18
18
19
-
Most users of this API need to deliver and receive audio from an end-user in real time, including applications that use WebRTC or a telephony system. The real-time API isn't designed to connect directly to end user devices and relies on client integrations to terminate end user audio streams.
19
+
Most users of this API need to deliver and receive audio from an end-user in real time, including applications that use WebRTC or a telephony system. The Realtime API isn't designed to connect directly to end user devices and relies on client integrations to terminate end user audio streams.
20
20
21
21
## Supported models
22
22
@@ -29,7 +29,7 @@ The `gpt-4o-realtime-preview` model is available for global deployments in [East
29
29
30
30
## API support
31
31
32
-
Support for real-time audio was first added in API version `2024-10-01-preview`.
32
+
Support for the Realtime API was first added in API version `2024-10-01-preview`.
33
33
34
34
> [!NOTE]
35
35
> For more information about the API and architecture, see the [Azure OpenAI GPT-4o real-time audio repository on GitHub](https://github.com/azure-samples/aoai-realtime-audio-sdk).
@@ -56,15 +56,18 @@ You can deploy the model from the Azure OpenAI model catalog or from your projec
56
56
57
57
Now that you have a deployment of the `gpt-4o-realtime-preview` model, you can use the playground to interact with the model in real time. Select **Early access playground** from the list of playgrounds in the left pane.
58
58
59
-
## Use the GPT-4o real-time audio API
59
+
## Use the GPT-4o Realtime API
60
60
61
61
> [!TIP]
62
62
> A playground for GPT-4o real-time audio is coming soon to [Azure AI Studio](https://ai.azure.com). You can already use the API directly in your application.
63
63
64
-
Right now, the fastest way to get started with GPT-4o real-time audio is to download the sample code from the [Azure OpenAI GPT-4o real-time audio repository on GitHub](https://github.com/azure-samples/aoai-realtime-audio-sdk).
64
+
Right now, the fastest way to get started with the GPT-4o Realtime API is to download the sample code from the [Azure OpenAI GPT-4o real-time audio repository on GitHub](https://github.com/azure-samples/aoai-realtime-audio-sdk).
65
+
66
+
The JavaScript web sample demonstrates how to use the GPT-4o Realtime API to interact with the model in real time. The sample code includes a simple web interface that captures audio from the user's microphone and sends it to the model for processing. The model responds with text and audio, which the sample code renders in the web interface.
67
+
68
+
You can run the sample code locally on your machine by following these steps. Refer to the [repository on GitHub](https://github.com/azure-samples/aoai-realtime-audio-sdk) for the most up-to-date instructions.
69
+
1. If you don't have Node.js installed, download and install the [LTS version of Node.js](https://nodejs.org/).
65
70
66
-
The JavaScript web sample demonstrates how to use the GPT-4o real-time audio API to interact with the model in real time. The sample code includes a simple web interface that captures audio from the user's microphone and sends it to the model for processing. The model responds with text and audio, which the sample code renders in the web interface.
67
-
68
71
1. Clone the repository to your local machine:
69
72
70
73
```bash
@@ -74,12 +77,18 @@ The JavaScript web sample demonstrates how to use the GPT-4o real-time audio API
74
77
1. Go to the `javascript/samples/web` folder in your preferred code editor.
75
78
76
79
```bash
77
-
cd .\javascript\samples\web\
80
+
cd ./javascript/samples
78
81
```
79
82
80
-
1. If you don't have Node.js installed, download and install the [LTS version of Node.js](https://nodejs.org/).
83
+
1. Run `download-pkg.ps1` or `download-pkg.sh` to download the required packages.
84
+
85
+
1. Go to the `web` folder from the `./javascript/samples` folder.
86
+
87
+
```bash
88
+
cd ./web
89
+
```
81
90
82
-
1. Run `npm install` to download a few dependency packages. For more information, see the `package.json` file in the same `web` folder.
91
+
1. Run `npm install` to install package dependencies.
83
92
84
93
1. Run `npm run dev` to start the web server, navigating any firewall permissions prompts as needed.
85
94
1. Go to any of the provided URIs from the console output (such as `http://localhost:5173/`) in a browser.
Copy file name to clipboardExpand all lines: articles/ai-services/openai/overview.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,13 +14,13 @@ recommendations: false
14
14
15
15
# What is Azure OpenAI Service?
16
16
17
-
Azure OpenAI Service provides REST API access to OpenAI's powerful language models including GPT-4o, GPT-4o mini, GPT-4 Turbo with Vision, GPT-4, GPT-3.5-Turbo, and Embeddings model series. These models can be easily adapted to your specific task including but not limited to content generation, summarization, image understanding, semantic search, and natural language to code translation. Users can access the service through REST APIs, Python SDK, or our web-based interface in the Azure OpenAI Studio.
17
+
Azure OpenAI Service provides REST API access to OpenAI's powerful language models including o1-preview, o1-mini, GPT-4o, GPT-4o mini, GPT-4 Turbo with Vision, GPT-4, GPT-3.5-Turbo, and Embeddings model series. These models can be easily adapted to your specific task including but not limited to content generation, summarization, image understanding, semantic search, and natural language to code translation. Users can access the service through REST APIs, Python SDK, or our web-based interface in the Azure OpenAI Studio.
18
18
19
19
### Features overview
20
20
21
21
| Feature | Azure OpenAI |
22
22
| --- | --- |
23
-
| Models available |**GPT-4o & GPT-4o mini**<br> **GPT-4 series (including GPT-4 Turbo with Vision)** <br>**GPT-3.5-Turbo series**<br> Embeddings series <br> Learn more in our [Models](./concepts/models.md) page.|
23
+
| Models available |**o1-preview** & **o1-mini** - (Limited Access - [Request Access](https://aka.ms/oai/modelaccess))<br>**GPT-4o & GPT-4o mini**<br> **GPT-4 series (including GPT-4 Turbo with Vision)** <br>**GPT-3.5-Turbo series**<br> Embeddings series <br> Learn more in our [Models](./concepts/models.md) page.|
| Price |[Available here](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/) <br> For details on GPT-4 Turbo with Vision, see the [special pricing information](../openai/concepts/gpt-with-vision.md#special-pricing-information).|
26
26
| Virtual network support & private link support | Yes, unless using [Azure OpenAI on your data](./concepts/use-your-data.md). |
@@ -125,4 +125,4 @@ Learn more about each model on our [models concept page](./concepts/models.md).
125
125
126
126
## Next steps
127
127
128
-
Learn more about the [underlying models that power Azure OpenAI](./concepts/models.md).
128
+
Learn more about the [underlying models that power Azure OpenAI](./concepts/models.md).
0 commit comments