You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
1. Make sure to sign in using the CLI `az login` (or `az login --use-device-code`) command to authenticate before running your Java scripts.
80
80
1. Download [POM.XML](https://github.com/azure-ai-foundry/foundry-samples/blob/main/samples/microsoft/java/mslearn-resources/quickstart/pom.xml) to your Java IDE.
@@ -90,7 +90,7 @@ No installation is necessary to use the Azure AI Foundry portal.
90
90
1. Install packages with `npm install`
91
91
1. Set these environment variables to use in your scripts:
Azure AI Foundry Agent Service allows you to create AI agents tailored to your needs through custom instructions and augmented by advanced tools like code interpreter, and custom functions.
Copy file name to clipboardExpand all lines: articles/ai-services/openai/concepts/model-router.md
+18-10Lines changed: 18 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
---
2
-
title: Azure OpenAI model router (preview) concepts
2
+
title: Model router for Azure AI Foundry (preview) concepts
3
3
titleSuffix: Azure OpenAI
4
4
description: Learn about the model router feature in Azure OpenAI in Azure AI Foundry Models.
5
5
author: PatrickFarley
@@ -11,13 +11,13 @@ ms.custom:
11
11
manager: nitinme
12
12
---
13
13
14
-
# Azure OpenAI model router (preview)
14
+
# Model router for Azure AI Foundry (preview)
15
15
16
-
Azure OpenAI model router is a deployable AI chat model that is trained to select the best large language model (LLM) to respond to a given prompt in real time. By evaluating factors like query complexity, cost, and performance, it intelligently routes requests to the most suitable model.
16
+
Model router for Azure AI Foundry is a deployable AI chat model that is trained to select the best large language model (LLM) to respond to a given prompt in real time. By evaluating factors like query complexity, cost, and performance, it intelligently routes requests to the most suitable model. Thus, it delivers high performance while saving on compute costs where possible, all packaged as a single model deployment.
17
17
18
18
## Why use model router?
19
19
20
-
Model router intelligently selects the best underlying model for a given prompt to optimize costs while maintaining quality. Smaller and cheaper models are used when they're sufficient for the task, but larger and more expensive models are available for more complex tasks. Also, reasoning models are available for tasks that require complex reasoning, and non-reasoning models are used otherwise. Model router provides a single chat experience that combines the best features from all of the underlying chat models.
20
+
Model router intelligently selects the best underlying model for a given prompt to optimize costs while maintaining quality. Smaller and cheaper models are used when they're sufficient for the task, but larger and more expensive models are available for more complex tasks. Also, reasoning models are available for tasks that require complex reasoning, and non-reasoning models are used otherwise. Model router provides a single deployment and chat experience that combines the best features from all of the underlying chat models.
21
21
22
22
## Versioning
23
23
@@ -29,24 +29,32 @@ If you select **Auto-update** at the deployment step (see [Manage models](/azure
See the [Models](../concepts/models.md#model-router) page for the region availability and deployment types for model router.
40
+
41
+
### Technical limitations
42
+
37
43
See [Quotas and limits](/azure/ai-services/openai/quotas-limits) for rate limit information.
38
44
39
-
The context window limit listed on the [Models](../concepts/models.md) page is the limit of the smallest underlying model. Other underlying models are compatible with larger context windows, which means an API call with a larger context will succeed only if the prompt happens to be routed to the right model, otherwise the call will fail. To shorten the context window, you can do one of the following:
40
-
- Summarize the prompt before passing it to the model
41
-
- Truncate the prompt into more relevant parts
42
-
- Use document embeddings and have the chat model retrieve relevant sections: see [Azure AI Search](/azure/search/search-what-is-azure-search)
45
+
> [!NOTE]
46
+
> The context window limit listed on the [Models](../concepts/models.md#model-router) page is the limit of the smallest underlying model. Other underlying models are compatible with larger context windows, which means an API call with a larger context will succeed only if the prompt happens to be routed to the right model, otherwise the call will fail. To shorten the context window, you can do one of the following:
47
+
> - Summarize the prompt before passing it to the model
48
+
> - Truncate the prompt into more relevant parts
49
+
> - Use document embeddings and have the chat model retrieve relevant sections: see [Azure AI Search](/azure/search/search-what-is-azure-search)
43
50
51
+
Model router accepts image inputs for [Vision enabled chats](/azure/ai-services/openai/how-to/gpt-with-vision) (all of the underlying models can accept image input), but the routing decision is based on the text input only.
44
52
45
53
Model router doesn't process audio input.
46
54
47
55
## Billing information
48
56
49
-
When you use Azure OpenAI model router, you're only billed for the use of the underlying models as they're recruited to respond to prompts. The model router itself doesn't incur any extra charges.
57
+
When you use model router, you're only billed for the use of the underlying models as they're recruited to respond to prompts. The model routing function itself doesn't incur any extra charges.
50
58
51
59
You can monitor the costs of your model router deployment in the Azure portal.
Copy file name to clipboardExpand all lines: articles/ai-services/openai/concepts/models.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -56,13 +56,13 @@ A model that intelligently selects from a set of underlying chat models to respo
56
56
57
57
| Model | Region |
58
58
|---|---|
59
-
|`model-router` (2025-04-15) | East US 2 (Global Standard), Sweden Central (Global Standard)|
59
+
|`model-router` (2025-05-19) | East US 2 (Global Standard), Sweden Central (Global Standard)|
60
60
61
61
### Capabilities
62
62
63
63
| Model ID | Description | Context Window | Max Output Tokens | Training Data (up to) |
64
64
| --- | :--- |:--- |:---|:---: |
65
-
|`model-router` (2025-04-15) | A model that intelligently selects from a set of underlying chat models to respond to a given prompt. | 200,000*| 32768 (GPT 4.1 series)</br> 100 K (o4-mini) | May 31, 2024 |
65
+
|`model-router` (2025-05-19) | A model that intelligently selects from a set of underlying chat models to respond to a given prompt. | 200,000*| 32768 (GPT 4.1 series)</br> 100 K (o4-mini) | May 31, 2024 |
66
66
67
67
*Larger context windows are compatible with _some_ of the underlying models, which means an API call with a larger context will succeed only if the prompt happens to be routed to the right model, otherwise the call will fail.
Copy file name to clipboardExpand all lines: articles/ai-services/openai/how-to/model-router.md
+7-8Lines changed: 7 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,5 @@
1
1
---
2
-
title: How to use model router (preview) in Azure OpenAI in Azure AI Foundry Models
3
-
titleSuffix: Azure OpenAI
2
+
title: How to use model router for Azure AI Foundry (preview)
4
3
description: Learn how to use the model router in Azure OpenAI to select the best model for your task.
5
4
author: PatrickFarley
6
5
ms.author: pafarley
@@ -11,20 +10,20 @@ ms.date: 04/17/2025
11
10
manager: nitinme
12
11
---
13
12
14
-
# Use Azure OpenAI model router (preview)
13
+
# Use model router for Azure AI Foundry (preview)
15
14
16
-
Azure OpenAI model router is a deployable AI chat model that is trained to select the best large language model (LLM) to respond to a given prompt in real time. It uses a combination of preexisting models to provide high performance while saving on compute costs where possible. For more information on how model router works and its advantages and limitations, see the [Model router concepts guide](../concepts/model-router.md).
15
+
Model router for Azure AI Foundry is a deployable AI chat model that is trained to select the best large language model (LLM) to respond to a given prompt in real time. It uses a combination of preexisting models to provide high performance while saving on compute costs where possible, all packaged as a single model deployment. For more information on how model router works and its advantages and limitations, see the [Model router concepts guide](../concepts/model-router.md).
17
16
18
-
You can access model router through the Completions API just as you would use a single base model like GPT-4.
17
+
You can access model router through the Completions API just as you would use a single base model like GPT-4. The steps are the same as in the [Chat completions guide](/azure/ai-services/openai/how-to/chatgpt).
19
18
20
19
## Deploy a model router model
21
20
22
-
Model router is packaged as a single OpenAI model that you deploy. Follow the steps in the [resource deployment guide](/azure/ai-services/openai/how-to/create-resource), and in the **Create new deployment** step, find `Azure OpenAI modelrouter` in the **Model** list. Select it, and then complete the rest of the deployment steps.
21
+
Model router is packaged as a single Azure AI Foundry model that you deploy. Follow the steps in the [resource deployment guide](/azure/ai-services/openai/how-to/create-resource). In the **Create new deployment** step, find `model-router` in the **Models** list. Select it, and then complete the rest of the deployment steps.
23
22
24
23
> [!NOTE]
25
24
> Consider that your deployment settings apply to all underlying chat models that model router uses.
26
25
> - You don't need to deploy the underlying chat models separately. Model router works independently of your other deployed models.
27
-
> - You select a content filter when you deploy the model router model (or you can apply a filter later). The content filter is applied to all activity to and from the model router: you don't set content filters for each of the underlying chat models.
26
+
> - You select a content filter when you deploy the model router model (or you can apply a filter later). The content filter is applied to all content passed to and from the model router: you don't set content filters for each of the underlying chat models.
28
27
> - Your tokens-per-minute rate limit setting is applied to all activity to and from the model router: you don't set rate limits for each of the underlying chat models.
29
28
30
29
## Use model router in chats
@@ -44,7 +43,7 @@ In the [Azure AI Foundry portal](https://ai.azure.com/), you can navigate to you
44
43
45
44
### Output format
46
45
47
-
The JSON response you receive from a model router model looks like the following.
46
+
The JSON response you receive from a model router model is identical to the standard chat completions API response. Note that the `"model"` field reveals which underlying model was selected to respond to the prompt.
Copy file name to clipboardExpand all lines: articles/ai-services/openai/whats-new.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -31,7 +31,7 @@ Spotlighting is a sub-feature of prompt shields that enhances protection against
31
31
32
32
### Model router (preview)
33
33
34
-
Azure OpenAI model router is a deployable AI chat model that automatically selects the best underlying chat model to respond to a given prompt. For more information on how model router works and its advantages and limitations, see the [Model router concepts guide](./concepts/model-router.md). To use model router with the Completions API, follow the [How-to guide](./concepts/model-router.md).
34
+
Model router for Azure AI Foundry is a deployable AI chat model that automatically selects the best underlying chat model to respond to a given prompt. For more information on how model router works and its advantages and limitations, see the [Model router concepts guide](./concepts/model-router.md). To use model router with the Completions API, follow the [How-to guide](./concepts/model-router.md).
0 commit comments