You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In this article, you learn how to use Azure AI Studio to deploy the Mistral family of models as serverless APIs with pay-as-you-go token-based billing.
21
21
Mistral AI offers two categories of models in the [Azure AI Studio](https://ai.azure.com). These models are available in the [model catalog](model-catalog-overview.md):
22
22
23
-
*__Premium models__: Mistral Large and Mistral Small. These models can be deployed as serverless APIs with pay-as-you-go token-based billing.
24
-
*__Open models__: Mixtral-8x7B-Instruct-v01, Mixtral-8x7B-v01, Mistral-7B-Instruct-v01, and Mistral-7B-v01. These models can be deployed to managed computes in your own Azure subscription.
23
+
*__Premium models__: Mistral Large (2402), Mistral Large (2407), and Mistral Small.
24
+
*__Open models__: Mistral Nemo, Mixtral-8x7B-Instruct-v01, Mixtral-8x7B-v01, Mistral-7B-Instruct-v01, and Mistral-7B-v01.
25
+
26
+
All the premium models and Mistral Nemo (an open model) can be deployed as serverless APIs with pay-as-you-go token-based billing. The other open models can be deployed to managed computes in your own Azure subscription.
25
27
26
28
You can browse the Mistral family of models in the model catalog by filtering on the Mistral collection.
27
29
28
30
## Mistral family of models
29
31
30
32
# [Mistral Large](#tab/mistral-large)
31
33
32
-
Mistral Large is Mistral AI's most advanced Large Language Model (LLM). It can be used on any language-based task, thanks to its state-of-the-art reasoning and knowledge capabilities.
34
+
Mistral Large is Mistral AI's most advanced Large Language Model (LLM). It can be used on any language-based task, thanks to its state-of-the-art reasoning and knowledge capabilities. There are two variants available for the Mistral Large model version:
35
+
36
+
- Mistral Large (2402)
37
+
- Mistral Large (2407)
33
38
34
-
Additionally, Mistral Large is:
39
+
Additionally, some attributes of _Mistral Large (2402)_ include:
35
40
36
41
*__Specialized in RAG.__ Crucial information isn't lost in the middle of long context windows (up to 32-K tokens).
37
42
*__Strong in coding.__ Code generation, review, and comments. Supports all mainstream coding languages.
38
43
*__Multi-lingual by design.__ Best-in-class performance in French, German, Spanish, Italian, and English. Dozens of other languages are supported.
39
44
*__Responsible AI compliant.__ Efficient guardrails baked in the model and extra safety layer with the `safe_mode` option.
40
45
46
+
And attributes of _Mistral Large (2407)_ include:
47
+
48
+
-**Multi-lingual by design.** Supports dozens of languages, including English, French, German, Spanish, and Italian.
49
+
-**Proficient in coding.** Trained on more than 80 coding languages, including Python, Java, C, C++, JavaScript, and Bash. Also trained on more specific languages such as Swift and Fortran.
50
+
-**Agent-centric.** Possesses agentic capabilities with native function calling and JSON outputting.
51
+
-**Advanced in reasoning.** Demonstrates state-of-the-art mathematical and reasoning capabilities.
52
+
53
+
41
54
# [Mistral Small](#tab/mistral-small)
42
55
43
56
Mistral Small is Mistral AI's most efficient Large Language Model (LLM). It can be used on any language-based task that requires high efficiency and low latency.
44
57
45
58
Mistral Small is:
46
59
47
-
-**A small model optimized for low latency.**Very efficient for high volume and low latency workloads. Mistral Small is Mistral's smallest proprietary model, it outperforms Mixtral-8x7B and has lower latency.
60
+
-**A small model optimized for low latency.**Efficient for high volume and low latency workloads. Mistral Small is Mistral's smallest proprietary model, it outperforms Mixtral-8x7B and has lower latency.
48
61
-**Specialized in RAG.** Crucial information isn't lost in the middle of long context windows (up to 32K tokens).
49
62
-**Strong in coding.** Code generation, review, and comments. Supports all mainstream coding languages.
50
63
-**Multi-lingual by design.** Best-in-class performance in French, German, Spanish, Italian, and English. Dozens of other languages are supported.
51
64
-**Responsible AI compliant.** Efficient guardrails baked in the model, and extra safety layer with the `safe_mode` option.
52
65
66
+
67
+
# [Mistral Nemo](#tab/mistral-nemo)
68
+
69
+
Mistral Nemo is a cutting-edge Language Model (LLM) boasting state-of-the-art reasoning, world knowledge, and coding capabilities within its size category.
70
+
71
+
Mistral Nemo is a 12B model, making it a powerful drop-in replacement for any system using Mistral 7B, which it supersedes. It supports a context length of 128K, and it accepts only text inputs and generates text outputs.
72
+
73
+
Additionally, Mistral Nemo is:
74
+
75
+
-**Jointly developed with Nvidia.** This collaboration has resulted in a powerful 12B model that pushes the boundaries of language understanding and generation.
76
+
-**Multilingual proficient.** Mistral Nemo is equipped with a tokenizer called Tekken, which is designed for multilingual applications. It supports over 100 languages, such as English, French, German, and Spanish. Tekken is more efficient than the Llama 3 tokenizer in compressing text for approximately 85% of all languages, with significant improvements in Malayalam, Hindi, Arabic, and prevalent European languages.
77
+
-**Agent-centric.** Mistral Nemo possesses top-tier agentic capabilities, including native function calling and JSON outputting.
78
+
-**Advanced in reasoning.** Mistral Nemo demonstrates state-of-the-art mathematical and reasoning capabilities within its size category.
79
+
53
80
---
54
81
55
82
## Deploy Mistral family of models as a serverless API
56
83
57
84
Certain models in the model catalog can be deployed as a serverless API with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need. This deployment option doesn't require quota from your subscription.
58
85
59
-
**Mistral Large** and **Mistral Small** can be deployed as a serverless API with pay-as-you-go billing and are offered by Mistral AI through the Microsoft Azure Marketplace. Mistral AI can change or update the terms of use and pricing of these models.
86
+
**Mistral Large (2402)**, **Mistral Large (2407)**, **Mistral Small**, and **Mistral Nemo** can be deployed as a serverless API with pay-as-you-go billing and are offered by Mistral AI through the Microsoft Azure Marketplace. Mistral AI can change or update the terms of use and pricing of these models.
60
87
61
88
### Prerequisites
62
89
@@ -78,13 +105,13 @@ Certain models in the model catalog can be deployed as a serverless API with pay
78
105
79
106
### Create a new deployment
80
107
81
-
The following steps demonstrate the deployment of Mistral Large, but you can use the same steps to deploy Mistral Small by replacing the model name.
108
+
The following steps demonstrate the deployment of Mistral Large (2402), but you can use the same steps to deploy Mistral Nemo or any of the premium Mistral models by replacing the model name.
82
109
83
110
To create a deployment:
84
111
85
112
1. Sign in to [Azure AI Studio](https://ai.azure.com).
86
113
1. Select **Model catalog** from the left sidebar.
87
-
1. Search for and select **Mistral-large** to open its Details page.
114
+
1. Search for and select the Mistral Large (2402) model to open its Details page.
88
115
89
116
:::image type="content" source="../media/deploy-monitor/mistral/mistral-large-deploy-directly-from-catalog.png" alt-text="A screenshot showing how to access the model details page by going through the model catalog." lightbox="../media/deploy-monitor/mistral/mistral-large-deploy-directly-from-catalog.png":::
90
117
@@ -93,7 +120,7 @@ To create a deployment:
93
120
94
121
1. From the left sidebar of your project, select **Components** > **Deployments**.
95
122
1. Select **+ Create deployment**.
96
-
1. Search for and select **Mistral-large**. to open the Model's Details page.
123
+
1. Search for and select the Mistral Large (2402) model to open the Model's Details page.
97
124
98
125
:::image type="content" source="../media/deploy-monitor/mistral/mistral-large-deploy-starting-from-project.png" alt-text="A screenshot showing how to access the model details page by going through the Deployments page in your project." lightbox="../media/deploy-monitor/mistral/mistral-large-deploy-starting-from-project.png":::
99
126
@@ -121,7 +148,7 @@ To learn about billing for the Mistral AI model deployed as a serverless API wit
121
148
122
149
### Consume the Mistral family of models as a service
123
150
124
-
You can consume Mistral family models by using the chat API.
151
+
You can consume Mistral models by using the chat API.
125
152
126
153
1. From your **Project overview** page, go to the left sidebar and select **Components** > **Deployments**.
Llama-2-7b <br> Llama-2-13b <br> Llama-2-70b | [Microsoft Managed Countries](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) | East US, East US 2, North Central US, South Central US, West US, West US 3 | West US 3
131
131
Llama-2-7b-chat <br> Llama-2-13b-chat <br> Llama-2-70b-chat | [Microsoft Managed Countries](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) | East US, East US 2, North Central US, South Central US, West US, West US 3, | Not available
132
132
Mistral Small | [Microsoft Managed Countries](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) | East US, East US 2, North Central US, South Central US, Sweden Central, West US, West US 3 | Not available
133
-
Mistral-Large | [Microsoft Managed Countries](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) <br> Brazil <br> Hong Kong <br> Israel| East US, East US 2, North Central US, South Central US, Sweden Central, West US, West US 3 | Not available
133
+
Mistral Large (2402) <br> Mistral-Large (2407) | [Microsoft Managed Countries](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) <br> Brazil <br> Hong Kong <br> Israel| East US, East US 2, North Central US, South Central US, Sweden Central, West US, West US 3 | Not available
134
+
Mistral Nemo | [Microsoft Managed Countries](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) <br> Brazil <br> Hong Kong <br> Israel| East US, East US 2, North Central US, South Central US, Sweden Central, West US, West US 3 | Not available
134
135
Cohere-command-r-plus <br> Cohere-command-r <br> Cohere-embed-v3-english <br> Cohere-embed-v3-multilingual | [Microsoft Managed Countries](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) <br> Japan | East US, East US 2, North Central US, South Central US, Sweden Central, West US, West US 3 | Not available
135
136
TimeGEN-1 | [Microsoft Managed Countries](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) <br> Mexico <br> Israel| East US, East US 2, North Central US, South Central US, Sweden Central, West US, West US 3 | Not available
136
137
jais-30b-chat | [Microsoft Managed Countries](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) | East US, East US 2, North Central US, South Central US, Sweden Central, West US, West US 3 | Not available
137
-
Phi-3-mini-4k-instruct | [Microsoft Managed Countries](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) | East US 2, Canada Central, Sweden Central, West US 3 | Not available
138
-
Phi-3-mini-128k-instruct <br> Phi-3-medium-4k-instruct <br> Phi-3-medium-128k-instruct | [Microsoft Managed Countries](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) | East US 2, Sweden Central | Not available
138
+
Phi-3-mini-4k-instruct <br> Phi-3-mini-128k-instruct | [Microsoft Managed Countries](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) | East US 2, Sweden Central | Not available
139
+
Phi-3-small-8k-instruct <br> Phi-3-small-128k-Instruct | [Microsoft Managed Countries](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) | East US 2, Sweden Central | Not available
140
+
Phi-3-medium-4k-instruct <br> Phi-3-medium-128k-instruct | [Microsoft Managed Countries](/partner-center/marketplace/tax-details-marketplace#microsoft-managed-countriesregions) | East US 2, Sweden Central | Not available
![prmerger-automator[bot]](https://avatars.githubusercontent.com/u/22479449?v=4&size=40){kind=avatar}
0 commit comments