You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Azure AI Studio supports deploying large language models (LLMs), flows, and web apps. Deploying an LLM or flow makes it available for use in a website, an application, or other production environments. This typically involves hosting the model on a server or in the cloud, and creating an API or other interface for users to interact with the model.
In this article, you learn about which regions are available for each of the models supporting serverless API endpoint deployments.
24
22
25
23
Certain models in the model catalog can be deployed as a serverless API with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need. This deployment option doesn't require quota from your subscription.
In this article, you learn how to configure an existing serverless API endpoint in a different project or hub than the one that was used to create the deployment.
23
21
24
22
[Certain models in the model catalog](deploy-models-serverless-availability.md) can be deployed as serverless APIs. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need. This deployment option doesn't require quota from your subscription.
@@ -235,4 +233,4 @@ Follow these steps to create a connection:
235
233
## Related content
236
234
237
235
- [What is Azure AI Studio?](../what-is-ai-studio.md)
In this article, you learn how to deploy a model from the model catalog as a serverless API with pay-as-you-go token based billing.
21
19
22
20
[Certain models in the model catalog](deploy-models-serverless-availability.md) can be deployed as a serverless API with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need. This deployment option doesn't require quota from your subscription.
@@ -86,16 +84,13 @@ In this article, you learn how to deploy a model from the model catalog as a ser
86
84
You can use any compatible web browser to [deploy ARM templates](../../azure-resource-manager/templates/deploy-portal.md) in the Microsoft Azure portal or use any of the deployment tools. This tutorial uses the [Azure CLI](/cli/azure/).
87
85
88
86
89
-
## Subscribe your project to the model offering
90
-
91
-
For models offered through the Azure Marketplace, you can deploy them to serverless API endpoints to consume their predictions. If it's your first time deploying the model in the project, you have to subscribe your project for the particular model offering from the Azure Marketplace. Each project has its own subscription to the particular Azure Marketplace offering of the model, which allows you to control and monitor spending.
92
-
93
-
> [!NOTE]
94
-
> Models offered through the Azure Marketplace are available for deployment to serverless API endpoints in specific regions. Check [Model and region availability for Serverless API deployments](deploy-models-serverless-availability.md) to verify which models and regions are available. If the one you need is not listed, you can deploy to a workspace in a supported region and then [consume serverless API endpoints from a different workspace](deploy-models-serverless-connect.md).
87
+
## Find your model and model ID in the model catalog
95
88
96
89
1. Sign in to [Azure AI Studio](https://ai.azure.com).
97
90
98
-
1. Ensure your account has the **Azure AI Developer** role permissions on the resource group, or that you meet the [permissions required to subscribe to model offerings](#permissions-required-to-subscribe-to-model-offerings).
91
+
1. For models offered through the Azure Marketplace, ensure that your account has the **Azure AI Developer** role permissions on the resource group, or that you meet the [permissions required to subscribe to model offerings](#permissions-required-to-subscribe-to-model-offerings).
92
+
93
+
Models that are offered by non-Microsoft providers (for example, Llama and Mistral models) are billed through the Azure Marketplace. For such models, you're required to subscribe your project to the particular model offering. Models that are offered by Microsoft (for example, Phi-3 models) don't have this requirement, as billing is done differently. For details about billing for serverless deployment of models in the model catalog, see [Billing for serverless APIs](model-catalog-overview.md#billing).
99
94
100
95
1. Select **Model catalog** from the left sidebar and find the model card of the model you want to deploy. In this article, you select a **Meta-Llama-3-8B-Instruct** model.
101
96
@@ -106,13 +101,23 @@ For models offered through the Azure Marketplace, you can deploy them to serverl
The next section covers the steps for subscribing your project to a model offering. You can skip this section and go to [Deploy the model to a serverless API endpoint](#deploy-the-model-to-a-serverless-api-endpoint), if you're deploying a Microsoft model.
106
+
107
+
## Subscribe your project to the model offering
108
+
109
+
For non-Microsoft models offered through the Azure Marketplace, you can deploy them to serverless API endpoints to consume their predictions. If it's your first time deploying the model in the project, you have to subscribe your project for the particular model offering from the Azure Marketplace. Each project has its own subscription to the particular Azure Marketplace offering of the model, which allows you to control and monitor spending.
110
+
111
+
> [!NOTE]
112
+
> Models offered through the Azure Marketplace are available for deployment to serverless API endpoints in specific regions. Check [Model and region availability for Serverless API deployments](deploy-models-serverless-availability.md) to verify which models and regions are available. If the one you need is not listed, you can deploy to a workspace in a supported region and then [consume serverless API endpoints from a different workspace](deploy-models-serverless-connect.md).
113
+
109
114
1. Create the model's marketplace subscription. When you create a subscription, you accept the terms and conditions associated with the model offer.
110
115
111
116
# [AI Studio](#tab/azure-ai-studio)
112
117
113
-
1. On the model's **Details** page, select **Deploy** and then select **Serverless API** to open the deployment wizard.
118
+
1. On the model's **Details** page, select **Deploy** and then select **Serverless API with Azure AI Content Safety (preview)** to open the deployment wizard.
114
119
115
-
1. Select the project in which you want to deploy your models. Notice that not all the regions are supported.
120
+
1. Select the project in which you want to deploy your models. To use the serverless API model deployment offering, your project must belong to one of the [regions that are supported for serverless deployment](deploy-models-serverless-availability.md) for the particular model.
116
121
117
122
:::image type="content" source="../media/deploy-monitor/serverless/deploy-pay-as-you-go.png" alt-text="A screenshot showing how to deploy a model with the serverless API option." lightbox="../media/deploy-monitor/serverless/deploy-pay-as-you-go.png":::
118
123
@@ -191,7 +196,7 @@ For models offered through the Azure Marketplace, you can deploy them to serverl
191
196
}
192
197
```
193
198
194
-
1. Once you sign up the project for the particular Azure Marketplace offering, subsequent deployments of the same offering in the same project don't require subscribing again.
199
+
1. Once you subscribe the project for the particular Azure Marketplace offering, subsequent deployments of the same offering in the same project don't require subscribing again.
195
200
196
201
1. At any point, you can see the model offers to which your project is currently subscribed:
197
202
@@ -233,15 +238,21 @@ For models offered through the Azure Marketplace, you can deploy them to serverl
233
238
234
239
## Deploy the model to a serverless API endpoint
235
240
236
-
Once you've created a model's subscription, you can deploy the associated model to a serverless API endpoint. The serverless API endpoint provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance organizations need. This deployment option doesn't require quota from your subscription.
241
+
Once you've created a subscription for a non-Microsoft model, you can deploy the associated model to a serverless API endpoint. For Microsoft models (such as Phi-3 models), you don't need to create a subscription.
237
242
238
-
In this article, you create an endpoint with name **meta-llama3-8b-qwerty**.
243
+
The serverless API endpoint provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance organizations need. This deployment option doesn't require quota from your subscription.
244
+
245
+
In this section, you create an endpoint with the name **meta-llama3-8b-qwerty**.
239
246
240
247
1. Create the serverless endpoint
241
248
242
249
# [AI Studio](#tab/azure-ai-studio)
243
250
244
-
1. From the previous wizard, select **Deploy** (if you've just subscribed the project to the model offer in the previous section), or select **Continue to deploy** (if your deployment wizard had the note *You already have an Azure Marketplace subscription for this project*).
251
+
1. To deploy a Microsoft model that doesn't require subscribing to a model offering:
252
+
1. Select **Deploy** and then select **Serverless API with Azure AI Content Safety (preview)** to open the deployment wizard.
253
+
1. Select the project in which you want to deploy your model. Notice that not all the regions are supported.
254
+
255
+
1. Alternatively, for a non-Microsoft model that requires a model subscription, if you've just subscribed your project to the model offer in the previous section, continue to select **Deploy**. Alternatively, select **Continue to deploy** (if your deployment wizard had the note *You already have an Azure Marketplace subscription for this project*).
245
256
246
257
:::image type="content" source="../media/deploy-monitor/serverless/deploy-pay-as-you-go-subscribed-project.png" alt-text="A screenshot showing a project that is already subscribed to the offering." lightbox="../media/deploy-monitor/serverless/deploy-pay-as-you-go-subscribed-project.png":::
247
258
@@ -418,11 +429,11 @@ In this article, you create an endpoint with name **meta-llama3-8b-qwerty**.
418
429
> [!TIP]
419
430
> If you're using prompt flow in the same project or hub where the deployment was deployed, you still need to create the connection.
420
431
421
-
## Using the serverless API endpoint
432
+
## Use the serverless API endpoint
422
433
423
434
Models deployed in Azure Machine Learning and Azure AI studio in Serverless API endpoints support the [Azure AI Model Inference API](../reference/reference-model-inference-api.md) that exposes a common set of capabilities for foundational models and that can be used by developers to consume predictions from a diverse set of models in a uniform and consistent way.
424
435
425
-
Read more about the [capabilities of this API](../reference/reference-model-inference-api.md#capabilities) and how [you can leverage it when building applications](../reference/reference-model-inference-api.md#getting-started).
436
+
Read more about the [capabilities of this API](../reference/reference-model-inference-api.md#capabilities) and how [you can use it when building applications](../reference/reference-model-inference-api.md#getting-started).
426
437
427
438
## Delete endpoints and subscriptions
428
439
@@ -495,7 +506,15 @@ az resource delete --name <resource-name>
495
506
496
507
## Cost and quota considerations for models deployed as serverless API endpoints
497
508
498
-
Models deployed as serverless API endpoints are offered through the Azure Marketplace and integrated with Azure AI Studio for use. You can find the Azure Marketplace pricing when deploying or fine-tuning the models.
509
+
Quota is managed per deployment. Each deployment has a rate limit of 200,000 tokens per minute and 1,000 API requests per minute. However, we currently limit one deployment per model per project. Contact Microsoft Azure Support if the current rate limits aren't sufficient for your scenarios.
510
+
511
+
#### Cost for Microsoft models
512
+
513
+
You can find the pricing information on the __Pricing and terms__ tab of the deployment wizard when deploying Microsoft models (such as Phi-3 models) as serverless API endpoints.
514
+
515
+
#### Cost for non-Microsoft models
516
+
517
+
Non-Microsoft models deployed as serverless API endpoints are offered through the Azure Marketplace and integrated with Azure AI Studio for use. You can find the Azure Marketplace pricing when deploying or fine-tuning these models.
499
518
500
519
Each time a project subscribes to a given offer from the Azure Marketplace, a new resource is created to track the costs associated with its consumption. The same resource is used to track costs associated with inference and fine-tuning; however, multiple meters are available to track each scenario independently.
501
520
@@ -504,8 +523,6 @@ For more information on how to track costs, see [Monitor costs for models offere
504
523
:::image type="content" source="../media/deploy-monitor/serverless/costs-model-as-service-cost-details.png" alt-text="A screenshot showing different resources corresponding to different model offers and their associated meters." lightbox="../media/deploy-monitor/serverless/costs-model-as-service-cost-details.png":::
505
524
506
525
507
-
Quota is managed per deployment. Each deployment has a rate limit of 200,000 tokens per minute and 1,000 API requests per minute. However, we currently limit one deployment per model per project. Contact Microsoft Azure Support if the current rate limits aren't sufficient for your scenarios.
508
-
509
526
## Permissions required to subscribe to model offerings
510
527
511
528
Azure role-based access controls (Azure RBAC) are used to grant access to operations in Azure AI Studio. To perform the steps in this article, your user account must be assigned the __Owner__, __Contributor__, or __Azure AI Developer__ role for the Azure subscription. Alternatively, your account can be assigned a custom role that has the following permissions:
@@ -527,6 +544,7 @@ Azure role-based access controls (Azure RBAC) are used to grant access to operat
527
544
528
545
For more information on permissions, see [Role-based access control in Azure AI Studio](../concepts/rbac-ai-studio.md).
529
546
530
-
## Next step
547
+
## Related content
531
548
549
+
*[Region availability for models in serverless API endpoints](deploy-models-serverless-availability.md)
532
550
*[Fine-tune a Meta Llama 2 model in Azure AI Studio](fine-tune-model-llama.md)
Copy file name to clipboardExpand all lines: articles/aks/manage-ssh-node-access.md
+3Lines changed: 3 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -125,6 +125,9 @@ To improve security and support your corporate security requirements or strategy
125
125
126
126
When you disable SSH at cluster creation time, it takes effect after the cluster is created. However, when you disable SSH on an existing cluster or node pool, AKS doesn't automatically disable SSH. At any time, you can choose to perform a nodepool upgrade operation. The disable/enable SSH keys operation takes effect after the node image update is complete.
127
127
128
+
> [!NOTE]
129
+
> When you disable SSH at the cluster level, it applies to all existing node pools. Any node pools created after this operation will have SSH enabled by default, and you'll need to run these commands again in order to disable it.
0 commit comments