You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# How to deploy Mistral models with Azure AI Studio
17
17
18
18
Mistral AI offers two categories of models in AI Studio:
19
-
* Premium models: Mistral-large. These models are available with pay-as-you-go token based billing with Models as a Service in the AI Studio model catalog.
19
+
* Premium models: Mistral Large. These models are available with pay-as-you-go token based billing with Models as a Service in the AI Studio model catalog.
20
20
* Open models: Mixtral-8x7B-Instruct-v01, Mixtral-8x7B-v01, Mistral-7B-Instruct-v01, and Mistral-7B-v01. These models are also available in the AI Studio model catalog and can be deployed to dedicated VM instances in your own Azure subscription with Managed Online Endpoints.
21
21
22
22
You can browse the Mistral family of models in the Model Catalog by filtering on the Mistral collection.
23
23
24
-
## Mistral-large
24
+
## Mistral Large
25
25
26
-
In this article, you learn how to use Azure AI Studio to deploy the Mistral-large model as a service with pay-as you go billing.
26
+
In this article, you learn how to use Azure AI Studio to deploy the Mistral Large model as a service with pay-as you go billing.
27
27
28
-
Mistral-large is Mistral AI's most advanced Large Language Model (LLM). It can be used on any language-based task thanks to its state-of-the-art reasoning and knowledge capabilities.
28
+
Mistral Large is Mistral AI's most advanced Large Language Model (LLM). It can be used on any language-based task thanks to its state-of-the-art reasoning and knowledge capabilities.
29
29
30
30
Additionally, mistral-large is:
31
31
32
-
* Straight-to-the-point. Purposely trained to eliminate unnecessary verbosity and generate concise outputs.
33
32
* Specialized in RAG. Crucial information isn't lost in the middle of long context windows (up to 32 K tokens).
34
-
* Strong in coding. Code generation, review, and comments. Can output results as JSON and do function calling.
33
+
* Strong in coding. Code generation, review, and comments. Supports all mainstream coding languages.
35
34
* Multi-lingual by design. Best-in-class performance in French, German, Spanish, and Italian - in addition to English. Dozens of other languages are supported.
35
+
* Responsible AI. Efficient guardrails baked in the model, with additional safety layer with safe_mode option.
36
36
37
37
[!INCLUDE [Azure AI Studio preview](../includes/preview-ai-studio.md)]
38
38
39
-
## Deploy Mistral-large with pay-as-you-go
39
+
## Deploy Mistral Large with pay-as-you-go
40
40
41
41
Certain models in the model catalog can be deployed as a service with pay-as-you-go, providing a way to consume them as an API without hosting them on your subscription, while keeping the enterprise security and compliance organizations need. This deployment option doesn't require quota from your subscription.
42
42
43
-
Mistral-large can be deployed as a service with pay-as-you-go, and is offered by Mistral AI through the Microsoft Azure Marketplace. Note that Mistral AI can change or update the terms of use and pricing of this model.
43
+
Mistral Large can be deployed as a service with pay-as-you-go, and is offered by Mistral AI through the Microsoft Azure Marketplace. Note that Mistral AI can change or update the terms of use and pricing of this model.
44
44
45
45
### Prerequisites
46
46
@@ -51,22 +51,7 @@ Mistral-large can be deployed as a service with pay-as-you-go, and is offered by
51
51
> Pay-as-you-go model deployment offering is only available in AI hubs created in **East US 2** and **France Central** regions.
52
52
53
53
- An [Azure AI project](../how-to/create-projects.md) in Azure AI Studio.
54
-
- Azure role-based access controls (Azure RBAC) are used to grant access to operations in Azure AI Studio. To perform the steps in this article, your user account must be assigned the __owner__ or __contributor__ role for the Azure subscription. Alternatively, your account can be assigned a custom role that has the following permissions:
55
-
56
-
- On the Azure subscription—to subscribe the Azure AI project to the Azure Marketplace offering, once for each project, per offering:
- Azure role-based access controls (Azure RBAC) are used to grant access to operations in Azure AI Studio. To perform the steps in this article, your user account must be assigned the __Azure AI Developer role__ on the Resouce Group.
70
55
71
56
For more information on permissions, see [Role-based access control in Azure AI Studio](../concepts/rbac-ai-studio.md).
72
57
@@ -87,14 +72,11 @@ To create a deployment:
87
72
1. Select the project in which you want to deploy your model. To deploy the Mistral-large model your project must belong to the **East US 2** or **France Central** regions.
88
73
1. In the deployment wizard, select the link to **Azure Marketplace Terms** to learn more about the terms of use.
89
74
1. You can also select the **Marketplace offer details** tab to learn about pricing for the selected model.
90
-
1. If this is your first time deploying the model in the project, you have to subscribe your project for the particular offering. This step requires that your account has the Azure subscription permissions and resource group permissions listed in the prerequisites. Each project has its own subscription to the particular Azure Marketplace offering of the model, which allows you to control and monitor spending. Select **Subscribe and Deploy**. Currently you can have only one deployment for each model within a project.
91
-
92
-
> [!NOTE]
93
-
> Subscribing a project to a particular Azure Marketplace offering (in this case, Mistral-large) requires that your account has **Contributor** or **Owner** access at the subscription level where the project is created. Alternatively, your user account can be assigned a custom role that has the Azure subscription permissions and resource group permissions listed in the [prerequisites](#prerequisites).
75
+
1. If this is your first time deploying the model in the project, you have to subscribe your project for the particular offering. This step requires that your account has the **Azure AI Developer role** permissions on the Resource Group, as listed in the prerequisites. Each project has its own subscription to the particular Azure Marketplace offering of the model, which allows you to control and monitor spending. Select **Subscribe and Deploy**. Currently you can have only one deployment for each model within a project.
94
76
95
77
:::image type="content" source="../media/deploy-monitor/mistral/mistral-deploy-marketplace-terms.png" alt-text="A screenshot showing the terms and conditions of a given model." lightbox="../media/deploy-monitor/mistral/mistral-deploy-marketplace-terms.png":::
96
78
97
-
1. Once you subscribe the project for the particular Azure Marketplace offering, subsequent deployments of the _same_ offering in the _same_ project don't require subscribing again. Therefore, you don't need to have the subscription-level or resource group-level permissions for subsequent deployments as mentioned in the prerequisites. If this scenario applies to you, you will see a **Continue to deploy** option to select.
79
+
1. Once you subscribe the project for the particular Azure Marketplace offering, subsequent deployments of the _same_ offering in the _same_ project don't require subscribing again. If this scenario applies to you, you will see a **Continue to deploy** option to select (Currently you can have only one deployment for each model within a project).
98
80
99
81
:::image type="content" source="../media/deploy-monitor/mistral/mistral-deploy-pay-as-you-go-project.png" alt-text="A screenshot showing a project that is already subscribed to the offering." lightbox="../media/deploy-monitor/mistral/mistral-deploy-pay-as-you-go-project.png":::
100
82
@@ -104,28 +86,26 @@ To create a deployment:
104
86
105
87
1. Select **Deploy**. Wait until the deployment is ready and you're redirected to the Deployments page.
106
88
1. Select **Open in playground** to start interacting with the model.
107
-
1. You can return to the Deployments page, select the deployment, and note the endpoint's **Target** URL and the Secret **Key**, which you can use to call the deployment for chat completions.
89
+
1. You can return to the Deployments page, select the deployment, and note the endpoint's **Target** URL and the Secret **Key**, which you can use to call the deployment for chat completions using the [`<target_url>/v1/chat/completions`](#chat-api) API.
108
90
1. You can always find the endpoint's details, URL, and access keys by navigating to the **Build** tab and selecting **Deployments** from the Components section.
109
91
110
-
To learn about billing for the Mistral AI model deployed with pay-as-you-go, see [Cost and quota considerations for Mistral-large deployed as a service](#cost-and-quota-considerations-for-mistral-large-deployed-as-a-service).
92
+
To learn about billing for the Mistral AI model deployed with pay-as-you-go, see [Cost and quota considerations for Mistral Large deployed as a service](#cost-and-quota-considerations-for-mistral-large-deployed-as-a-service).
111
93
112
-
### Consume the Mistral-large model as a service
94
+
### Consume the Mistral Large model as a service
113
95
114
-
Mistral-large can be consumed using the chat API.
96
+
Mistral Large can be consumed using the chat API.
115
97
116
98
1. On the **Build** page, select **Deployments**.
117
99
118
100
1. Find and select the deployment you created.
119
101
120
-
1. Select **Open in playground**.
121
-
122
-
1. Select **View code** and copy the **Endpoint** URL and the **Key** value.
102
+
1. Copy the **Target** URL and the **Key** value.
123
103
124
-
1. Make an API request using the [`/v1/chat/completions`](#chat-api) API.
104
+
1. Make an API request using the [`/v1/chat/completions`](#chat-api) API using [`<target_url>/v1/chat/completions`](#chat-api).
125
105
126
106
For more information on using the APIs, see the [reference](#reference-for-mistral-large-deployed-as-a-service) section.
127
107
128
-
### Reference for Mistral-large deployed as a service
108
+
### Reference for Mistral Large deployed as a service
129
109
130
110
#### Chat API
131
111
@@ -148,7 +128,7 @@ Payload is a JSON formatted string containing the following parameters:
148
128
|-----|-----|-----|-----|
149
129
|`messages`|`string`| No default. This value must be specified. | The message or history of messages to use to prompt the model. |
150
130
|`stream`|`boolean`|`False`| Streaming allows the generated tokens to be sent as data-only server-sent events whenever they become available. |
151
-
|`max_tokens`|`integer`|`16`| The maximum number of tokens to generate in the completion. The token count of your prompt plus `max_tokens` can't exceed the model's context length. |
131
+
|`max_tokens`|`integer`|`8192`| The maximum number of tokens to generate in the completion. The token count of your prompt plus `max_tokens` can't exceed the model's context length. |
152
132
|`top_p`|`float`|`1`| An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with `top_p` probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. We generally recommend altering `top_p` or `temperature`, but not both. |
153
133
|`temperature`|`float`|`1`| The sampling temperature to use, between 0 and 2. Higher values mean the model samples more broadly the distribution of tokens. Zero means greedy sampling. We recommend altering this or `top_p`, but not both. |
154
134
|`ignore_eos`|`boolean`|`False`| Whether to ignore the EOS token and continue generating tokens after the EOS token is generated. |
@@ -259,15 +239,15 @@ The following is an example response:
### Cost and quota considerations for Mistral-large deployed as a service
250
+
### Cost and quota considerations for Mistral Large deployed as a service
271
251
272
252
Mistral models deployed as a service are offered by Mistral AI through the Azure Marketplace and integrated with Azure AI Studio for use. You can find the Azure Marketplace pricing when deploying the model.
273
253
@@ -277,10 +257,6 @@ For more information on how to track costs, see [monitor costs for models offere
277
257
278
258
Quota is managed per deployment. Each deployment has a rate limit of 200,000 tokens per minute and 1,000 API requests per minute. However, we currently limit one deployment per model per project. Contact Microsoft Azure Support if the current rate limits aren't sufficient for your scenarios.
279
259
280
-
## Data and policy
281
-
282
-
No data from the user using models deployed as a service with pay-as-you-go is sent to the model provider (in this case Mistral AI).
283
-
284
260
## Content filtering
285
261
286
262
Models deployed as a service with pay-as-you-go are protected by Azure AI Content Safety. With Azure AI content safety, both the prompt and completion pass through an ensemble of classification models aimed at detecting and preventing the output of harmful content. The content filtering system detects and takes action on specific categories of potentially harmful content in both input prompts and output completions. Learn more about [Azure AI Content Safety](../concepts/content-filtering.md).
![prmerger-automator[bot]](https://avatars.githubusercontent.com/u/22479449?v=4&size=40){kind=avatar}
0 commit comments