You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/ai-foundry/agents/how-to/tools/azure-ai-search.md
+7Lines changed: 7 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -166,6 +166,13 @@ You can add the Azure AI Search tool to an agent programmatically using the code
166
166
167
167
1. The index is created and connected to the [Azure AI Search](/azure/search/) service. You can now use the index with the Azure AI Search tool in your agent. You can also use the index outside of the agent, such as the Azure AI Search REST API or SDKs.
168
168
169
+
170
+
## Limitations
171
+
172
+
Currently, if you want to use the Azure AI Search tool in the Azure AI Foundry portal behind a virtual network, you must create an agent using the SDK or REST API. After creating the agent in a code-based manner, you can then use it in the portal.
173
+
174
+
The Azure AI Search tool can only include one search index. If you want to utilize multiple indexes, consider using [connected agents](../connected-agents.md) with an Azure AI search index configured with each agent.
175
+
169
176
## Next steps
170
177
171
178
* See examples on how to use the [Azure AI Search tool](azure-ai-search-samples.md).
Copy file name to clipboardExpand all lines: articles/ai-foundry/concepts/prompt-flow.md
+3Lines changed: 3 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,6 +7,7 @@ ms.custom:
7
7
- ignite-2023
8
8
- build-2024
9
9
- ignite-2024
10
+
- hub-only
10
11
ms.topic: concept-article
11
12
ms.date: 06/30/2025
12
13
ms.reviewer: none
@@ -24,6 +25,8 @@ Prompt flow is a development tool designed to streamline the entire development
24
25
25
26
Prompt flow is available independently as an open-source project on [GitHub](https://github.com/microsoft/promptflow), with its own SDK and [VS Code extension](https://marketplace.visualstudio.com/items?itemName=prompt-flow.prompt-flow). Prompt flow is also available and recommended to use as a feature within both [Azure AI Foundry](https://ai.azure.com/?cid=learnDocs) and [Azure Machine Learning studio](https://ml.azure.com). This set of documentation focuses on prompt flow in Azure AI Foundry portal.
Copy file name to clipboardExpand all lines: articles/ai-foundry/foundry-models/quotas-limits.md
+35-27Lines changed: 35 additions & 27 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,19 +6,19 @@ author: msakande
6
6
ms.service: azure-ai-model-inference
7
7
ms.custom: ignite-2024, github-universe-2024
8
8
ms.topic: concept-article
9
-
ms.date: 05/19/2025
9
+
ms.date: 08/14/2025
10
10
ms.author: mopeakande
11
-
ms.reviewer: fasantia
12
-
reviewer: santiagxf
11
+
ms.reviewer: shiyingfu
12
+
reviewer: swingfu
13
13
---
14
14
15
15
# Azure AI Foundry Models quotas and limits
16
16
17
-
This article contains a quick reference and a detailed description of the quotas and limits for Azure AI Foundry Models. For quotas and limits specific to the Azure OpenAI in Foundry Models, see [Quota and limits in Azure OpenAI](../openai/quotas-limits.md).
17
+
This article provides a quick reference and detailed description of the quotas and limits for Azure AI Foundry Models. For quotas and limits specific to the Azure OpenAI in Foundry Models, see [Quota and limits in Azure OpenAI](../openai/quotas-limits.md).
18
18
19
19
## Quotas and limits reference
20
20
21
-
Azure uses quotas and limits to prevent budget overruns due to fraud, and to honor Azure capacity constraints. Consider these limits as you scale for production workloads. The following sections provide you with a quick guide to the default quotas and limits that apply to Azure AI model's inference service in Azure AI Foundry:
21
+
Azure uses quotas and limits to prevent budget overruns due to fraud and to honor Azure capacity constraints. Consider these limits as you scale for production workloads. The following sections provide a quick guide to the default quotas and limits that apply to Azure AI model inference service in Azure AI Foundry:
22
22
23
23
### Resource limits
24
24
@@ -30,58 +30,66 @@ Azure uses quotas and limits to prevent budget overruns due to fraud, and to hon
The following table lists limits for Foundry Models for the following rates:
43
34
44
-
You can [request increases to the default limits](#request-increases-to-the-default-limits). Due to high demand, limit increase requests can be submitted and evaluated per request.
35
+
- Tokens per minute
36
+
- Requests per minute
37
+
- Concurrent request
38
+
39
+
| Models | Tokens per minute | Requests per minute | Concurrent requests |
| Azure OpenAI models | Varies per model and SKU. See [limits for Azure OpenAI](../openai/quotas-limits.md). | Varies per model and SKU. See [limits for Azure OpenAI](../openai/quotas-limits.md). | not applicable |
| - Flux-Pro 1.1<br />- Flux.1-Kontext Pro | not applicable | 2 capacity units (6 requests per minute) | not applicable |
45
+
| Rest of models | 400,000 | 1,000 | 300 |
46
+
47
+
To increase your quota:
48
+
49
+
- For Azure OpenAI, use [Azure AI Foundry Service: Request for Quota Increase](https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR4xPXO648sJKt4GoXAed-0pUMFE1Rk9CU084RjA0TUlVSUlMWEQzVkJDNCQlQCN0PWcu) to submit your request.
50
+
- For other models, see [request increases to the default limits](#request-increases-to-the-default-limits).
51
+
52
+
Due to high demand, we evaluate limit increase requests per request.
45
53
46
54
### Other limits
47
55
48
56
| Limit name | Limit value |
49
57
|--|--|
50
58
| Max number of custom headers in API requests<sup>1</sup> | 10 |
51
59
52
-
<sup>1</sup> Our current APIs allow up to 10 custom headers, which are passed through the pipeline, and returned. We have noticed some customers now exceed this header count resulting in HTTP 431 errors. There is no solution for this error, other than to reduce header volume. **In future API versions we will no longer pass through custom headers**. We recommend customers not depend on custom headers in future system architectures.
60
+
<sup>1</sup> Our current APIs allow up to 10 custom headers, which the pipeline passes through and returns. If you exceed this header count, your request results in an HTTP 431 error. To resolve this error, reduce the header volume. **Future API versions won't pass through custom headers**. We recommend that you don't depend on custom headers in future system architectures.
53
61
54
62
## Usage tiers
55
63
56
-
Global Standard deployments use Azure's global infrastructure, dynamically routing customer traffic to the data center with best availability for the customer's inference requests. This enables more consistent latency for customers with low to medium levels of traffic. Customers with high sustained levels of usage might see more variabilities in response latency.
64
+
Global Standard deployments use Azure's global infrastructure to dynamically route customer traffic to the data center with best availability for the customer's inference requests. This infrastructure enables more consistent latency for customers with low to medium levels of traffic. Customers with high sustained levels of usage might see more variabilities in response latency.
57
65
58
66
The Usage Limit determines the level of usage above which customers might see larger variability in response latency. A customer's usage is defined per model and is the total tokens consumed across all deployments in all subscriptions in all regions for a given tenant.
59
67
60
68
## Request increases to the default limits
61
69
62
-
Limit increase requests can be submitted and evaluated per request. [Open an online customer support request](https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest/). When requesting for endpoint limit increase, provide the following information:
70
+
You can submit limit increase requests, which we evaluate one at a time. [Open an online customer support request](https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest/). When you request an endpoint limit increase, provide the following information:
63
71
64
-
1.When opening the support request, select **Service and subscription limits (quotas)** as the **Issue type**.
72
+
1.Select **Service and subscription limits (quotas)** as the **Issue type** when you open the support request.
65
73
66
-
1. Select the subscription of your choice.
74
+
1. Select the subscription you want to use.
67
75
68
76
1. Select **Cognitive Services** as **Quota type**.
69
77
70
78
1. Select **Next**.
71
79
72
-
1. On the **Additional details** tab, you need to provide detailed reasons for the limit increase in order for your request to be processed. Be sure to add the following information into the reason for limit increase:
80
+
1. On the **Additional details** tab, provide detailed reasons for the limit increase so that your request can be processed. Be sure to add the following information to the reason for limit increase:
73
81
74
82
* Model name, model version (if applicable), and deployment type (SKU).
75
83
* Description of your scenario and workload.
76
84
* Rationale for the requested increase.
77
-
*Provide the target throughput: Tokens per minute, requests per minute, etc.
78
-
*Provide planned time plan (by when you need increased limits).
85
+
*Target throughput: Tokens per minute, requests per minute, and other relevant metrics.
86
+
*Planned time plan (by when you need increased limits).
79
87
80
-
1.Finally, select **Save and continue** to continue.
88
+
1.Select **Save and continue**.
81
89
82
-
## General best practices to remain within rate limits
90
+
## General best practices to stay within rate limits
83
91
84
-
To minimize issues related to rate limits, it's a good idea to use the following techniques:
92
+
To minimize issues related to rate limits, use the following techniques:
85
93
86
94
- Implement retry logic in your application.
87
95
- Avoid sharp changes in the workload. Increase the workload gradually.
For a batch run and to use an evaluation method, you need to have the following ready:
36
39
37
40
- A test dataset for batch run. Your dataset should be in one of these formats: `.csv`, `.tsv`, or `.jsonl`. Your data should also include headers that match the input names of your flow. If your flow inputs include a complex structure like a list or dictionary, use `jsonl` format to represent your data.
Copy file name to clipboardExpand all lines: articles/ai-foundry/how-to/flow-tune-prompts-using-variants.md
+3Lines changed: 3 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,6 +6,7 @@ ms.service: azure-ai-foundry
6
6
ms.custom:
7
7
- ignite-2023
8
8
- build-2024
9
+
- hub-only
9
10
ms.topic: how-to
10
11
ms.date: 3/31/2025
11
12
ms.reviewer: none
@@ -25,6 +26,8 @@ Crafting a good prompt is a challenging task that requires much creativity, clar
25
26
26
27
Variants can help you test the model’s behavior under different conditions, such as different wording, formatting, context, temperature, or top-k. You can compare and find the best prompt and configuration that maximizes the model's accuracy, diversity, or coherence.
With prompt flow, you can use variants to tune your prompt. A variant refers to a specific version of a tool node that has distinct settings. Currently, variants are supported only in the [LLM tool](prompt-flow-tools/llm-tool.md). For example, in the LLM tool, a new variant can represent either a different prompt content or different connection settings.
Copy file name to clipboardExpand all lines: articles/ai-foundry/how-to/prompt-flow-tools/index-lookup-tool.md
+5Lines changed: 5 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,6 +5,7 @@ description: This article introduces you to the Index Lookup tool for flows in A
5
5
ms.service: azure-ai-foundry
6
6
ms.custom:
7
7
- build-2024
8
+
- hub-only
8
9
ms.topic: reference
9
10
ms.date: 6/30/2025
10
11
ms.reviewer: none
@@ -20,6 +21,10 @@ ms.update-cycle: 180-days
20
21
21
22
The prompt flow Index Lookup tool enables the use of common vector indices (such as Azure AI Search, Faiss, and Pinecone) for retrieval augmented generation in prompt flow. The tool automatically detects the indices in the workspace and allows the selection of the index to be used in the flow.
Copy file name to clipboardExpand all lines: articles/ai-foundry/how-to/prompt-flow-tools/llm-tool.md
+4-1Lines changed: 4 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,6 +6,7 @@ ms.service: azure-ai-foundry
6
6
ms.custom:
7
7
- ignite-2023
8
8
- build-2024
9
+
- hub-only
9
10
ms.topic: reference
10
11
ms.date: 6/30/2025
11
12
ms.reviewer: none
@@ -21,11 +22,13 @@ ms.update-cycle: 180-days
21
22
22
23
To use large language models (LLMs) for natural language processing, you use the prompt flow LLM tool.
23
24
24
-
> [!NOTE]
25
+
> [!TIP]
25
26
> For embeddings to convert text into dense vector representations for various natural language processing tasks, see [Embedding tool](embedding-tool.md).
Prepare a prompt as described in the [Prompt tool](prompt-tool.md#prerequisites) documentation. The LLM tool and Prompt tool both support [Jinja](https://jinja.palletsprojects.com/en/stable/) templates. For more information and best practices, see [Prompt engineering techniques](../../openai/concepts/advanced-prompt-engineering.md).
0 commit comments