Skip to content

Commit 4b36b41

Browse files
committed
update azureml docs
1 parent 19d2f4c commit 4b36b41

File tree

5 files changed

+35
-27
lines changed

5 files changed

+35
-27
lines changed

articles/machine-learning/how-to-deploy-models-mistral.md

Lines changed: 35 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -16,14 +16,14 @@ ms.custom: [references_regions]
1616
---
1717
# How to deploy Mistral models with Azure Machine Learning studio
1818

19-
In this article, you learn how to use Azure Machine Learning studio to deploy the Mistral family of models as a service with pay-as-you-go billing.
19+
In this article, you learn how to use Azure Machine Learning studio to deploy the Mistral family of models as serverless APIs with pay-as-you-go token-based billing.
2020

21-
Mistral AI offers two categories of models in Azure Machine Learning studio:
21+
Mistral AI offers two categories of models in Azure Machine Learning studio. These models are available in the [model catalog](concept-model-catalog.md).
2222

23-
- __Premium models__: Mistral Large and Mistral Small. These models are available with pay-as-you-go token based billing with Models as a Service in the studio model catalog.
24-
- __Open models__: Mixtral-8x7B-Instruct-v01, Mixtral-8x7B-v01, Mistral-7B-Instruct-v01, and Mistral-7B-v01. These models are also available in the studio model catalog and can be deployed to dedicated VM instances in your own Azure subscription with managed online endpoints.
25-
26-
You can browse the Mistral family of models in the [model catalog](concept-model-catalog.md) by filtering on the Mistral collection.
23+
- __Premium models__: Mistral Large and Mistral Small. These models can be deployed as serverless APIs with pay-as-you-go token-based billing.
24+
- __Open models__: Mixtral-8x7B-Instruct-v01, Mixtral-8x7B-v01, Mistral-7B-Instruct-v01, and Mistral-7B-v01. These models can be deployed to managed computes in your own Azure subscription.
25+
26+
You can browse the Mistral family of models in the model catalog by filtering on the Mistral collection.
2727

2828
## Mistral family of models
2929

@@ -37,7 +37,7 @@ Additionally, Mistral Large is:
3737
- __Strong in coding.__ Code generation, review, and comments. Supports all mainstream coding languages.
3838
- __Multi-lingual by design.__ Best-in-class performance in French, German, Spanish, and Italian - in addition to English. Dozens of other languages are supported.
3939
- __Responsible AI compliant.__ Efficient guardrails baked in the model, and extra safety layer with the `safe_mode` option.
40-
40+
4141
# [Mistral Small](#tab/mistral-small)
4242

4343
Mistral Small is Mistral AI's most efficient Large Language Model (LLM). It can be used on any language-based task that requires high efficiency and low latency.
@@ -54,19 +54,27 @@ Mistral Small is:
5454

5555
[!INCLUDE [machine-learning-preview-generic-disclaimer](includes/machine-learning-preview-generic-disclaimer.md)]
5656

57-
## Deploy Mistral family of models with pay-as-you-go
57+
## Deploy Mistral family of models as a serverless API
58+
59+
Certain models in the model catalog can be deployed as a serverless API with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need. This deployment option doesn't require quota from your subscription.
5860

59-
Certain models in the model catalog can be deployed as a service with pay-as-you-go. Pay-as-you-go deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need. This deployment option doesn't require quota from your subscription.
61+
**Mistral Large** and **Mistral Small** can be deployed as a serverless API with pay-as-you-go billing and are offered by Mistral AI through the Microsoft Azure Marketplace. Mistral AI can change or update the terms of use and pricing of these models.
6062

61-
**Mistral Large** and **Mistral Small** are eligible to be deployed as a service with pay-as-you-go and are offered by Mistral AI through the Microsoft Azure Marketplace. Mistral AI can change or update the terms of use and pricing of these models.
6263

6364
### Prerequisites
6465

6566
- An Azure subscription with a valid payment method. Free or trial Azure subscriptions won't work. If you don't have an Azure subscription, create a [paid Azure account](https://azure.microsoft.com/pricing/purchase-options/pay-as-you-go) to begin.
66-
- An Azure Machine Learning workspace. If you don't have a workspace, use the steps in the [Quickstart: Create workspace resources](quickstart-create-resources.md) article to create one.
67+
- An Azure Machine Learning workspace. If you don't have a workspace, use the steps in the [Quickstart: Create workspace resources](quickstart-create-resources.md) article to create one. The serverless API model deployment offering for eligible models in the Mistral family is only available in workspaces created in these regions:
6768

68-
> [!IMPORTANT]
69-
> The pay-as-you-go model deployment offering for eligible models in the Mistral family is only available in workspaces created in the **East US 2** and **Sweden Central** regions.
69+
- East US
70+
- East US 2
71+
- North Central US
72+
- South Central US
73+
- West US
74+
- West US 3
75+
- Sweden Central
76+
77+
For a list of regions that are available for each of the models supporting serverless API endpoint deployments, see [Region availability for models in serverless API endpoints](concept-endpoint-serverless-availability.md)
7078

7179
- Azure role-based access controls (Azure RBAC) are used to grant access to operations in Azure Machine Learning. To perform the steps in this article, your user account must be assigned the __Azure AI Developer role__ on the resource group. For more information on permissions, see [Manage access to an Azure Machine Learning workspace](how-to-assign-roles.md).
7280

@@ -77,24 +85,23 @@ The following steps demonstrate the deployment of Mistral Large, but you can use
7785
To create a deployment:
7886

7987
1. Go to [Azure Machine Learning studio](https://ml.azure.com/home).
80-
1. Select the workspace in which you want to deploy your models. To use the pay-as-you-go model deployment offering, your workspace must belong to the **East US 2** or **Sweden Central** region.
81-
1. Choose the model (Mistral-large) that you want to deploy from the [model catalog](https://ml.azure.com/model/catalog).
88+
1. Select the workspace in which you want to deploy your model. To use the serverless API model deployment offering, your workspace must belong to one of the regions listed in the [prerequisites](#prerequisites).
89+
1. Choose the model you want to deploy, for example Mistral-large, from the [model catalog](https://ml.azure.com/model/catalog).
8290

8391
Alternatively, you can initiate deployment by going to your workspace and selecting **Endpoints** > **Serverless endpoints** > **Create**.
8492

85-
1. On the model's overview page in the model catalog, select **Deploy** and then **Pay-as-you-go**.
93+
1. On the model's overview page in the model catalog, select **Deploy** to open a serverless API deployment window for the model.
94+
1. Select the checkbox to acknowledge the Microsoft purchase policy.
8695

87-
:::image type="content" source="media/how-to-deploy-models-mistral/mistral-deploy-pay-as-you-go.png" alt-text="A screenshot showing how to deploy a model with the pay-as-you-go option." lightbox="media/how-to-deploy-models-mistral/mistral-deploy-pay-as-you-go.png":::
96+
:::image type="content" source="media/how-to-deploy-models-mistral/mistral-deploy-serverless-api.png" alt-text="A screenshot showing how to deploy a model as a serverless API." lightbox="media/how-to-deploy-models-mistral/mistral-deploy-serverless-api.png":::
8897

8998
1. In the deployment wizard, select the link to **Azure Marketplace Terms** to learn more about the terms of use.
90-
1. You can also select the **Marketplace offer details** tab to learn about pricing for the selected model.
99+
1. You can also select the **Pricing and terms** tab to learn about pricing for the selected model.
91100
1. If this is your first time deploying the model in the workspace, you have to subscribe your workspace for the particular offering (for example, Mistral-large). This step requires that your account has the **Azure AI Developer role** permissions on the Resource Group, as listed in the prerequisites. Each workspace has its own subscription to the particular Azure Marketplace offering, which allows you to control and monitor spending. Select **Subscribe and Deploy**. Currently you can have only one deployment for each model within a workspace.
92101

93-
:::image type="content" source="media/how-to-deploy-models-mistral/mistral-deploy-marketplace-terms.png" alt-text="A screenshot showing the terms and conditions of a given model." lightbox="media/how-to-deploy-models-mistral/mistral-deploy-marketplace-terms.png":::
94-
95102
1. Once you subscribe the workspace for the particular Azure Marketplace offering, subsequent deployments of the _same_ offering in the _same_ workspace don't require subscribing again. If this scenario applies to you, you'll see a **Continue to deploy** option to select.
96103

97-
:::image type="content" source="media/how-to-deploy-models-mistral/mistral-deploy-pay-as-you-go-project.png" alt-text="A screenshot showing a workspace that is already subscribed to the offering." lightbox="media/how-to-deploy-models-mistral/mistral-deploy-pay-as-you-go-project.png":::
104+
:::image type="content" source="media/how-to-deploy-models-mistral/mistral-deploy-serverless-api-project.png" alt-text="A screenshot showing a workspace that is already subscribed to the offering." lightbox="media/how-to-deploy-models-mistral/mistral-deploy-serverless-api-project.png":::
98105

99106
1. Give the deployment a name. This name becomes part of the deployment API URL. This URL must be unique in each Azure region.
100107

@@ -105,7 +112,7 @@ To create a deployment:
105112
1. Select the **Test** tab to start interacting with the model.
106113
1. You can always find the endpoint's details, URL, and access keys by navigating to **Workspace** > **Endpoints** > **Serverless endpoints**.
107114

108-
To learn about billing for Mistral models deployed with pay-as-you-go, see [Cost and quota considerations for Mistral family of models deployed as a service](#cost-and-quota-considerations-for-mistral-family-of-models-deployed-as-a-service).
115+
To learn about billing for Mistral models deployed as a serverless API with pay-as-you-go token-based billing, see [Cost and quota considerations for Mistral family of models deployed as a service](#cost-and-quota-considerations-for-mistral-family-of-models-deployed-as-a-service).
109116

110117
### Consume the Mistral family of models as a service
111118

@@ -118,15 +125,15 @@ You can consume Mistral Large by using the chat API.
118125

119126
For more information on using the APIs, see the [reference](#reference-for-mistral-family-of-models-deployed-as-a-service) section.
120127

121-
### Reference for Mistral family of models deployed as a service
128+
## Reference for Mistral family of models deployed as a service
122129

123130
Mistral models accept both the [Azure AI Model Inference API](reference-model-inference-api.md) on the route `/chat/completions` and the native [Mistral Chat API](#mistral-chat-api) on `/v1/chat/completions`.
124131

125132
### Azure AI Model Inference API
126133

127134
The [Azure AI Model Inference API](reference-model-inference-api.md) schema can be found in the [reference for Chat Completions](reference-model-inference-chat-completions.md) article and an [OpenAPI specification can be obtained from the endpoint itself](reference-model-inference-api.md?tabs=rest#getting-started).
128135

129-
#### Mistral Chat API
136+
### Mistral Chat API
130137

131138
Use the method `POST` to send the request to the `/v1/chat/completions` route:
132139

@@ -161,7 +168,7 @@ The `messages` object has the following fields:
161168
| `role` | `string` | The role of the message's author. One of `system`, `user`, or `assistant`. |
162169

163170

164-
#### Example
171+
#### Request example
165172

166173
__Body__
167174

@@ -227,7 +234,7 @@ The `logprobs` object is a dictionary with the following fields:
227234
| `tokens` | `array` of `string` | Selected tokens. |
228235
| `top_logprobs` | `array` of `dictionary` | Array of dictionary. In each dictionary, the key is the token and the value is the prob. |
229236

230-
#### Example
237+
#### Response example
231238

232239
The following JSON is an example response:
233240

@@ -284,5 +291,6 @@ Models deployed as a service with pay-as-you-go are protected by Azure AI conten
284291
## Related content
285292

286293
- [Model Catalog and Collections](concept-model-catalog.md)
294+
- [Region availability for models in serverless API endpoints](concept-endpoint-serverless-availability.md)
287295
- [Deploy and score a machine learning model by using an online endpoint](how-to-deploy-online-endpoints.md)
288-
- [Plan and manage costs for Azure AI Studio](concept-plan-manage-cost.md)
296+
- [Plan and manage costs for Azure AI Studio](concept-plan-manage-cost.md)
34.3 KB
Loading
76.7 KB
Loading
6.16 KB
Loading

0 commit comments

Comments
 (0)