Skip to content

Commit e0d86f1

Browse files
PatrickFarleyPatrickFarley
committed
Merge branch 'main' of https://github.com/MicrosoftDocs/azure-ai-docs-pr into consaf-updates
2 parents d43d570 + 0e48306 commit e0d86f1

File tree

2,359 files changed

+56582
-18280
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

2,359 files changed

+56582
-18280
lines changed

‎.openpublishing.publish.config.json

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,18 @@
104104
"branch": "master",
105105
"branch_mapping": {}
106106
},
107+
{
108+
"path_to_root": "azureai-samples-main",
109+
"url": "https://github.com/Azure-Samples/azureai-samples",
110+
"branch": "main",
111+
"branch_mapping": {}
112+
},
113+
{
114+
"path_to_root": "azureai-samples-csharp",
115+
"url": "https://github.com/Azure-Samples/azureai-samples",
116+
"branch": "dantaylo/csharp",
117+
"branch_mapping": {}
118+
},
107119
{
108120
"path_to_root": "azureml-examples-main",
109121
"url": "https://github.com/azure/azureml-examples",
@@ -164,6 +176,12 @@
164176
"branch": "main",
165177
"branch_mapping": {}
166178
},
179+
{
180+
"path_to_root": "azureai-model-inference-bicep",
181+
"url": "https://github.com/Azure-Samples/azureai-model-inference-bicep",
182+
"branch": "main",
183+
"branch_mapping": {}
184+
},
167185
{
168186
"path_to_root": "azure-docs-pr-policy-includes",
169187
"url": "https://github.com/MicrosoftDocs/azure-docs-pr",

‎.openpublishing.redirection.json

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,11 @@
2424
"source_path_from_root": "/articles/search/search-howto-large-index.md",
2525
"redirect_url": "/azure/search/search-how-to-large-index",
2626
"redirect_document_id": false
27+
},
28+
{
29+
"source_path_from_root": "/articles/ai-services/openai/how-to/use-your-data-securely.md",
30+
"redirect_url": "/azure/ai-services/openai/how-to/on-your-data-configuration",
31+
"redirect_document_id": false
2732
}
2833
]
2934
}

‎articles/ai-foundry/model-inference/breadcrumb/toc.yml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
- name: Azure
2+
tocHref: /azure/
3+
topicHref: /azure/index
4+
items:
5+
- name: AI Foundry
6+
tocHref: /azure/ai-foundry/
7+
topicHref: /azure/ai-studio/index
8+
items:
9+
- name: Model Inference
10+
tocHref: /azure/ai-foundry/model-inference/
11+
topicHref: /azure/ai-foundry/model-inference/index

‎articles/ai-foundry/model-inference/concepts/content-filter.md

Lines changed: 309 additions & 0 deletions
Large diffs are not rendered by default.

‎articles/ai-foundry/model-inference/concepts/default-safety-policies.md

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
---
2+
title: Default content safety policies for Azure AI Model Inference
3+
titleSuffix: Azure AI Foundry
4+
description: Learn about the default content safety policies that Azure AI Model Inference uses to flag content.
5+
author: PatrickFarley
6+
ms.author: fasantia
7+
ms.service: azure-ai-model-inference
8+
ms.topic: conceptual
9+
ms.date: 1/21/2025
10+
manager: nitinme
11+
---
12+
13+
# Default content safety policies for Azure AI Model Inference
14+
15+
Azure AI model inference includes default safety applied to all models, excluding Azure OpenAI Whisper. These configurations provide you with a responsible experience by default.
16+
17+
Default safety aims to mitigate risks such as hate and fairness, sexual, violence, self-harm, protected material content, and user prompt injection attacks. To learn more about content filtering, read [our documentation describing categories and severity levels](content-filter.md).
18+
19+
This document describes the default configuration.
20+
21+
> [!TIP]
22+
> By default, all model deployments use the default configuration. However, you can configure content filtering per model deployment as explained at [Configuring content filtering](../how-to/configure-content-filters.md).
23+
24+
## Text models
25+
26+
Text models in Azure AI model inference can take in and generate both text and code. These models apply Azure's text content filtering models to detect and prevent harmful content. This system works on both prompt and completion.
27+
28+
| Risk Category | Prompt/Completion | Severity Threshold |
29+
|-------------------------------------------|------------------------|---------------------|
30+
| Hate and Fairness | Prompts and Completions| Medium |
31+
| Violence | Prompts and Completions| Medium |
32+
| Sexual | Prompts and Completions| Medium |
33+
| Self-Harm | Prompts and Completions| Medium |
34+
| User prompt injection attack (Jailbreak) | Prompts | N/A |
35+
| Protected Material – Text | Completions | N/A |
36+
| Protected Material – Code | Completions | N/A |
37+
38+
## Vision and chat with vision models
39+
40+
Vision models can take both text and images at the same time as part of the input. Default content filtering capabilities vary per model and provider.
41+
42+
### Azure OpenAI: GPT-4o and GPT-4 Turbo
43+
44+
| Risk Category | Prompt/Completion | Severity Threshold |
45+
|---------------------------------------------------------------------|------------------------|---------------------|
46+
| Hate and Fairness | Prompts and Completions| Medium |
47+
| Violence | Prompts and Completions| Medium |
48+
| Sexual | Prompts and Completions| Medium |
49+
| Self-Harm | Prompts and Completions| Medium |
50+
| Identification of Individuals and Inference of Sensitive Attributes | Prompts | N/A |
51+
| User prompt injection attack (Jailbreak) | Prompts | N/A |
52+
53+
### Azure OpenAI: DALL-E 3 and DALL-E 2
54+
55+
| Risk Category | Prompt/Completion | Severity Threshold |
56+
|---------------------------------------------------|------------------------|---------------------|
57+
| Hate and Fairness | Prompts and Completions| Low |
58+
| Violence | Prompts and Completions| Low |
59+
| Sexual | Prompts and Completions| Low |
60+
| Self-Harm | Prompts and Completions| Low |
61+
| Content Credentials | Completions | N/A |
62+
| Deceptive Generation of Political Candidates | Prompts | N/A |
63+
| Depictions of Public Figures | Prompts | N/A |
64+
| User prompt injection attack (Jailbreak) | Prompts | N/A |
65+
| Protected Material – Art and Studio Characters | Prompts | N/A |
66+
| Profanity | Prompts | N/A |
67+
68+
69+
In addition to the previous safety configurations, Azure OpenAI DALL-E also comes with [prompt transformation](../../../ai-services/openai/concepts/prompt-transformation.md) by default. This transformation occurs on all prompts to enhance the safety of your original prompt, specifically in the risk categories of diversity, deceptive generation of political candidates, depictions of public figures, protected material, and others.
70+
71+
### Meta: Llama-3.2-11B-Vision-Instruct and Llama-3.2-90B-Vision-Instruct
72+
73+
Content filters apply only to text prompts and completions. Images aren't subject to content moderation.
74+
75+
### Microsoft: Phi-3.5-vision-instruct
76+
77+
Content filters apply only to text prompts and completions. Images aren't subject to content moderation.
78+
79+
## Next steps
80+
81+
* [Configure content filters in Azure AI Model Inference](../how-to/configure-content-filters.md)

‎articles/ai-foundry/model-inference/concepts/deployment-types.md

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
---
2+
title: Understanding deployment types in Azure AI model inference
3+
titleSuffix: Azure AI Foundry
4+
description: Learn how to use deployment types in Azure AI model deployments
5+
author: santiagxf
6+
manager: nitinme
7+
ms.service: azure-ai-model-inference
8+
ms.topic: how-to
9+
ms.date: 1/21/2025
10+
ms.author: fasantia
11+
ms.custom: ignite-2024, github-universe-2024
12+
---
13+
14+
# Deployment types in Azure AI model inference
15+
16+
Azure AI model inference makes models available using the *model deployment* concept in Azure AI Services resources. *Model deployments* are also Azure resources and, when created, they give access to a given model under certain configurations. Such configuration includes the infrastructure require to process the requests.
17+
18+
Azure AI model inference provides customers with choices on the hosting structure that fits their business and usage patterns. Those options are translated to different deployments types (or SKUs) that are available at model deployment time in the Azure AI Services resource.
19+
20+
:::image type="content" source="../media/add-model-deployments/models-deploy-deployment-type.png" alt-text="Screenshot showing how to customize the deployment type for a given model deployment." lightbox="../media/add-model-deployments/models-deploy-deployment-type.png":::
21+
22+
Different model providers offer different deployments SKUs that you can select from. When selecting a deployment type, consider your **data residency needs** and **call volume/capacity** requirements.
23+
24+
## Deployment types for Azure OpenAI models
25+
26+
The service offers two main types of deployments: **standard** and **provisioned**. For a given deployment type, customers can align their workloads with their data processing requirements by choosing an Azure geography (`Standard` or `Provisioned-Managed`), Microsoft specified data zone (`DataZone-Standard` or `DataZone Provisioned-Managed`), or Global (`Global-Standard` or `Global Provisioned-Managed`) processing options.
27+
28+
To learn more about deployment options for Azure OpenAI models see [Azure OpenAI documentation](../../../ai-services/openai/how-to/deployment-types.md).
29+
30+
## Deployment types for Models-as-a-Service models
31+
32+
Models from third-party model providers with pay-as-you-go billing (collectively called Models-as-a-Service), makes models available in Azure AI model inference under **standard** deployments with a Global processing option (`Global-Standard`).
33+
34+
### Global-Standard
35+
36+
Global deployments leverage Azure's global infrastructure to dynamically route traffic to the data center with best availability for each request. Global standard provides the highest default quota and eliminates the need to load balance across multiple resources. Data stored at rest remains in the designated Azure geography, while data may be processed for inferencing in any Azure location. Learn more about [data residency](https://azure.microsoft.com/explore/global-infrastructure/data-residency/).
37+
38+
> [!NOTE]
39+
> Models-as-a-Service offers regional deployment options under [Serverless API endpoints](../../../ai-studio/how-to/deploy-models-serverless.md) in Azure AI Foundry. Prompts and outputs are processed within the geography specified during deployment. However, those deployments can't be accessed using the Azure AI model inference endpoint in Azure AI Services.
40+
41+
## Control deployment options
42+
43+
Administrators can control which model deployment types are available to their users by using Azure Policies. Learn more about [How to control AI model deployment with custom policies](../../../ai-studio/how-to/custom-policy-model-deployment.md).
44+
45+
## Related content
46+
47+
- [Quotas & limits](../quotas-limits.md)
48+
- [Data privacy, and security for Models-as-a-Service models](../../../ai-studio/how-to/concept-data-privacy.md)

‎articles/ai-foundry/model-inference/concepts/endpoints.md

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
---
2+
title: Model inference endpoint in Azure AI services
3+
titleSuffix: Azure AI Foundry
4+
description: Learn about the model inference endpoint in Azure AI services
5+
author: santiagxf
6+
manager: nitinme
7+
ms.service: azure-ai-model-inference
8+
ms.topic: how-to
9+
ms.date: 1/21/2025
10+
ms.author: fasantia
11+
ms.custom: ignite-2024, github-universe-2024
12+
---
13+
14+
# Model inference endpoint in Azure AI Services
15+
16+
Azure AI model inference in Azure AI services allows customers to consume the most powerful models from flagship model providers using a single endpoint and credentials. This means that you can switch between models and consume them from your application without changing a single line of code.
17+
18+
The article explains how models are organized inside of the service and how to use the inference endpoint to invoke them.
19+
20+
## Deployments
21+
22+
Azure AI model inference makes models available using the **deployment** concept. **Deployments** are a way to give a model a name under certain configurations. Then, you can invoke such model configuration by indicating its name on your requests.
23+
24+
Deployments capture:
25+
26+
> [!div class="checklist"]
27+
> * A model name
28+
> * A model version
29+
> * A provisioning/capacity type<sup>1</sup>
30+
> * A content filtering configuration<sup>1</sup>
31+
> * A rate limiting configuration<sup>1</sup>
32+
33+
<sup>1</sup> Configurations may vary depending on the selected model.
34+
35+
An Azure AI services resource can have as many model deployments as needed and they don't incur in cost unless inference is performed for those models. Deployments are Azure resources and hence they're subject to Azure policies.
36+
37+
To learn more about how to create deployments see [Add and configure model deployments](../how-to/create-model-deployments.md).
38+
39+
## Azure AI inference endpoint
40+
41+
The Azure AI inference endpoint allows customers to use a single endpoint with the same authentication and schema to generate inference for the deployed models in the resource. This endpoint follows the [Azure AI model inference API](.././reference/reference-model-inference-api.md) which all the models in Azure AI model inference support. It support the following modalidities:
42+
43+
* Text embeddings
44+
* Image embeddings
45+
* Chat completions
46+
47+
You can see the endpoint URL and credentials in the **Overview** section:
48+
49+
:::image type="content" source="../media/overview/overview-endpoint-and-key.png" alt-text="Screenshot showing how to get the URL and key associated with the resource." lightbox="../media/overview/overview-endpoint-and-key.png":::
50+
51+
### Routing
52+
53+
The inference endpoint routes requests to a given deployment by matching the parameter `name` inside of the request to the name of the deployment. This means that *deployments work as an alias of a given model under certain configurations*. This flexibility allows you to deploy a given model multiple times in the service but under different configurations if needed.
54+
55+
:::image type="content" source="../media/endpoint/endpoint-routing.png" alt-text="An illustration showing how routing works for a Meta-llama-3.2-8b-instruct model by indicating such name in the parameter 'model' inside of the payload request." lightbox="../media/endpoint/endpoint-routing.png":::
56+
57+
For example, if you create a deployment named `Mistral-large`, then such deployment can be invoked as:
58+
59+
[!INCLUDE [code-create-chat-client](../includes/code-create-chat-client.md)]
60+
61+
[!INCLUDE [code-create-chat-completion](../includes/code-create-chat-completion.md)]
62+
63+
> [!TIP]
64+
> Deployment routing isn't case sensitive.
65+
66+
### SDKs
67+
68+
The Azure AI model inference endpoint is supported by multiple SDKs, including the **Azure AI Inference SDK**, the **Azure AI Foundry SDK**, and the **Azure OpenAI SDK**; which are available in multiple languages. Multiple integrations are also supported in popular frameworks like LangChain, LangGraph, Llama-Index, Semantic Kernel, and AG2. See [supported programming languages and SDKs](../supported-languages.md) for details.
69+
70+
## Azure OpenAI inference endpoint
71+
72+
Azure OpenAI models deployed to AI services also support the Azure OpenAI API. This API exposes the full capabilities of OpenAI models and supports additional features like assistants, threads, files, and batch inference.
73+
74+
Azure OpenAI inference endpoints work at the deployment level and they have their own URL that is associated with each of them. However, the same authentication mechanism can be used to consume them. Learn more in the reference page for [Azure OpenAI API](../../../ai-services/openai/reference.md)
75+
76+
:::image type="content" source="../media/endpoint/endpoint-openai.png" alt-text="An illustration showing how Azure OpenAI deployments contain a single URL for each deployment." lightbox="../media/endpoint/endpoint-openai.png":::
77+
78+
Each deployment has a URL that is the concatenations of the **Azure OpenAI** base URL and the route `/deployments/<model-deployment-name>`.
79+
80+
> [!IMPORTANT]
81+
> There's no routing mechanism for the Azure OpenAI endpoint, as each URL is exclusive for each model deployment.
82+
83+
### SDKs
84+
85+
The Azure OpenAI endpoint is supported by the **OpenAI SDK (`AzureOpenAI` class)** and **Azure OpenAI SDKs**, which are available in multiple languages. See [supported languages](../supported-languages.md#azure-openai-models) for details.
86+
87+
88+
## Next steps
89+
90+
- [Models](models.md)
91+
- [Deployment types](deployment-types.md)

0 commit comments

Comments
 (0)