You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# How to deploy Mistral models with Azure Machine Learning studio
18
18
19
-
In this article, you learn how to use Azure Machine Learning studio to deploy the Mistral family of models as a service with pay-as-you-go billing.
19
+
In this article, you learn how to use Azure Machine Learning studio to deploy the Mistral family of models as serverless APIs with pay-as-you-go token-based billing.
20
20
21
-
Mistral AI offers two categories of models in Azure Machine Learning studio:
21
+
Mistral AI offers two categories of models in Azure Machine Learning studio. These models are available in the [model catalog](concept-model-catalog.md).
22
22
23
-
-__Premium models__: Mistral Large and Mistral Small. These models are available with pay-as-you-go tokenbased billing with Models as a Service in the studio model catalog.
24
-
-__Open models__: Mixtral-8x7B-Instruct-v01, Mixtral-8x7B-v01, Mistral-7B-Instruct-v01, and Mistral-7B-v01. These models are also available in the studio model catalog and can be deployed to dedicated VM instances in your own Azure subscription with managed online endpoints.
25
-
26
-
You can browse the Mistral family of models in the [model catalog](concept-model-catalog.md) by filtering on the Mistral collection.
23
+
-__Premium models__: Mistral Large and Mistral Small. These models can be deployed as serverless APIs with pay-as-you-go token-based billing.
24
+
-__Open models__: Mixtral-8x7B-Instruct-v01, Mixtral-8x7B-v01, Mistral-7B-Instruct-v01, and Mistral-7B-v01. These models can be deployed to managed computes in your own Azure subscription.
25
+
26
+
You can browse the Mistral family of models in the model catalog by filtering on the Mistral collection.
27
27
28
28
## Mistral family of models
29
29
@@ -37,7 +37,7 @@ Additionally, Mistral Large is:
37
37
-__Strong in coding.__ Code generation, review, and comments. Supports all mainstream coding languages.
38
38
-__Multi-lingual by design.__ Best-in-class performance in French, German, Spanish, and Italian - in addition to English. Dozens of other languages are supported.
39
39
-__Responsible AI compliant.__ Efficient guardrails baked in the model, and extra safety layer with the `safe_mode` option.
40
-
40
+
41
41
# [Mistral Small](#tab/mistral-small)
42
42
43
43
Mistral Small is Mistral AI's most efficient Large Language Model (LLM). It can be used on any language-based task that requires high efficiency and low latency.
## Deploy Mistral family of models with pay-as-you-go
57
+
## Deploy Mistral family of models as a serverless API
58
+
59
+
Certain models in the model catalog can be deployed as a serverless API with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need. This deployment option doesn't require quota from your subscription.
58
60
59
-
Certain models in the model catalog can be deployed as a service with pay-as-you-go. Pay-as-you-go deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need. This deployment option doesn't require quota from your subscription.
61
+
**Mistral Large** and **Mistral Small**can be deployed as a serverless API with pay-as-you-go billing and are offered by Mistral AI through the Microsoft Azure Marketplace. Mistral AI can change or update the terms of use and pricing of these models.
60
62
61
-
**Mistral Large** and **Mistral Small** are eligible to be deployed as a service with pay-as-you-go and are offered by Mistral AI through the Microsoft Azure Marketplace. Mistral AI can change or update the terms of use and pricing of these models.
62
63
63
64
### Prerequisites
64
65
65
66
- An Azure subscription with a valid payment method. Free or trial Azure subscriptions won't work. If you don't have an Azure subscription, create a [paid Azure account](https://azure.microsoft.com/pricing/purchase-options/pay-as-you-go) to begin.
66
-
- An Azure Machine Learning workspace. If you don't have a workspace, use the steps in the [Quickstart: Create workspace resources](quickstart-create-resources.md) article to create one.
67
+
- An Azure Machine Learning workspace. If you don't have a workspace, use the steps in the [Quickstart: Create workspace resources](quickstart-create-resources.md) article to create one. The serverless API model deployment offering for eligible models in the Mistral family is only available in workspaces created in these regions:
67
68
68
-
> [!IMPORTANT]
69
-
> The pay-as-you-go model deployment offering for eligible models in the Mistral family is only available in workspaces created in the **East US 2** and **Sweden Central** regions.
69
+
- East US
70
+
- East US 2
71
+
- North Central US
72
+
- South Central US
73
+
- West US
74
+
- West US 3
75
+
- Sweden Central
76
+
77
+
For a list of regions that are available for each of the models supporting serverless API endpoint deployments, see [Region availability for models in serverless API endpoints](concept-endpoint-serverless-availability.md)
70
78
71
79
- Azure role-based access controls (Azure RBAC) are used to grant access to operations in Azure Machine Learning. To perform the steps in this article, your user account must be assigned the __Azure AI Developer role__ on the resource group. For more information on permissions, see [Manage access to an Azure Machine Learning workspace](how-to-assign-roles.md).
72
80
@@ -77,24 +85,23 @@ The following steps demonstrate the deployment of Mistral Large, but you can use
77
85
To create a deployment:
78
86
79
87
1. Go to [Azure Machine Learning studio](https://ml.azure.com/home).
80
-
1. Select the workspace in which you want to deploy your models. To use the pay-as-you-go model deployment offering, your workspace must belong to the **East US 2** or **Sweden Central** region.
81
-
1. Choose the model (Mistral-large) that you want to deploy from the [model catalog](https://ml.azure.com/model/catalog).
88
+
1. Select the workspace in which you want to deploy your model. To use the serverless API model deployment offering, your workspace must belong to one of the regions listed in the [prerequisites](#prerequisites).
89
+
1. Choose the model you want to deploy, for example Mistral-large, from the [model catalog](https://ml.azure.com/model/catalog).
82
90
83
91
Alternatively, you can initiate deployment by going to your workspace and selecting **Endpoints** > **Serverless endpoints** > **Create**.
84
92
85
-
1. On the model's overview page in the model catalog, select **Deploy** and then **Pay-as-you-go**.
93
+
1. On the model's overview page in the model catalog, select **Deploy** to open a serverless API deployment window for the model.
94
+
1. Select the checkbox to acknowledge the Microsoft purchase policy.
86
95
87
-
:::image type="content" source="media/how-to-deploy-models-mistral/mistral-deploy-pay-as-you-go.png" alt-text="A screenshot showing how to deploy a model with the pay-as-you-go option." lightbox="media/how-to-deploy-models-mistral/mistral-deploy-pay-as-you-go.png":::
96
+
:::image type="content" source="media/how-to-deploy-models-mistral/mistral-deploy-serverless-api.png" alt-text="A screenshot showing how to deploy a model as a serverless API." lightbox="media/how-to-deploy-models-mistral/mistral-deploy-serverless-api.png":::
88
97
89
98
1. In the deployment wizard, select the link to **Azure Marketplace Terms** to learn more about the terms of use.
90
-
1. You can also select the **Marketplace offer details** tab to learn about pricing for the selected model.
99
+
1. You can also select the **Pricing and terms** tab to learn about pricing for the selected model.
91
100
1. If this is your first time deploying the model in the workspace, you have to subscribe your workspace for the particular offering (for example, Mistral-large). This step requires that your account has the **Azure AI Developer role** permissions on the Resource Group, as listed in the prerequisites. Each workspace has its own subscription to the particular Azure Marketplace offering, which allows you to control and monitor spending. Select **Subscribe and Deploy**. Currently you can have only one deployment for each model within a workspace.
92
101
93
-
:::image type="content" source="media/how-to-deploy-models-mistral/mistral-deploy-marketplace-terms.png" alt-text="A screenshot showing the terms and conditions of a given model." lightbox="media/how-to-deploy-models-mistral/mistral-deploy-marketplace-terms.png":::
94
-
95
102
1. Once you subscribe the workspace for the particular Azure Marketplace offering, subsequent deployments of the _same_ offering in the _same_ workspace don't require subscribing again. If this scenario applies to you, you'll see a **Continue to deploy** option to select.
96
103
97
-
:::image type="content" source="media/how-to-deploy-models-mistral/mistral-deploy-pay-as-you-go-project.png" alt-text="A screenshot showing a workspace that is already subscribed to the offering." lightbox="media/how-to-deploy-models-mistral/mistral-deploy-pay-as-you-go-project.png":::
104
+
:::image type="content" source="media/how-to-deploy-models-mistral/mistral-deploy-serverless-api-project.png" alt-text="A screenshot showing a workspace that is already subscribed to the offering." lightbox="media/how-to-deploy-models-mistral/mistral-deploy-serverless-api-project.png":::
98
105
99
106
1. Give the deployment a name. This name becomes part of the deployment API URL. This URL must be unique in each Azure region.
100
107
@@ -105,7 +112,7 @@ To create a deployment:
105
112
1. Select the **Test** tab to start interacting with the model.
106
113
1. You can always find the endpoint's details, URL, and access keys by navigating to **Workspace** > **Endpoints** > **Serverless endpoints**.
107
114
108
-
To learn about billing for Mistral models deployed with pay-as-you-go, see [Cost and quota considerations for Mistral family of models deployed as a service](#cost-and-quota-considerations-for-mistral-family-of-models-deployed-as-a-service).
115
+
To learn about billing for Mistral models deployed as a serverless API with pay-as-you-go token-based billing, see [Cost and quota considerations for Mistral family of models deployed as a service](#cost-and-quota-considerations-for-mistral-family-of-models-deployed-as-a-service).
109
116
110
117
### Consume the Mistral family of models as a service
111
118
@@ -118,15 +125,15 @@ You can consume Mistral Large by using the chat API.
118
125
119
126
For more information on using the APIs, see the [reference](#reference-for-mistral-family-of-models-deployed-as-a-service) section.
120
127
121
-
###Reference for Mistral family of models deployed as a service
128
+
## Reference for Mistral family of models deployed as a service
122
129
123
130
Mistral models accept both the [Azure AI Model Inference API](reference-model-inference-api.md) on the route `/chat/completions` and the native [Mistral Chat API](#mistral-chat-api) on `/v1/chat/completions`.
124
131
125
132
### Azure AI Model Inference API
126
133
127
134
The [Azure AI Model Inference API](reference-model-inference-api.md) schema can be found in the [reference for Chat Completions](reference-model-inference-chat-completions.md) article and an [OpenAPI specification can be obtained from the endpoint itself](reference-model-inference-api.md?tabs=rest#getting-started).
128
135
129
-
####Mistral Chat API
136
+
### Mistral Chat API
130
137
131
138
Use the method `POST` to send the request to the `/v1/chat/completions` route:
132
139
@@ -161,7 +168,7 @@ The `messages` object has the following fields:
161
168
|`role`|`string`| The role of the message's author. One of `system`, `user`, or `assistant`. |
162
169
163
170
164
-
#### Example
171
+
#### Request example
165
172
166
173
__Body__
167
174
@@ -227,7 +234,7 @@ The `logprobs` object is a dictionary with the following fields:
227
234
|`tokens`|`array` of `string`| Selected tokens. |
228
235
|`top_logprobs`|`array` of `dictionary`| Array of dictionary. In each dictionary, the key is the token and the value is the prob. |
229
236
230
-
#### Example
237
+
#### Response example
231
238
232
239
The following JSON is an example response:
233
240
@@ -284,5 +291,6 @@ Models deployed as a service with pay-as-you-go are protected by Azure AI conten
284
291
## Related content
285
292
286
293
-[Model Catalog and Collections](concept-model-catalog.md)
294
+
-[Region availability for models in serverless API endpoints](concept-endpoint-serverless-availability.md)
287
295
-[Deploy and score a machine learning model by using an online endpoint](how-to-deploy-online-endpoints.md)
288
-
-[Plan and manage costs for Azure AI Studio](concept-plan-manage-cost.md)
296
+
-[Plan and manage costs for Azure AI Studio](concept-plan-manage-cost.md)
0 commit comments