You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
| Which models can be deployed? |[Azure OpenAI models](../../ai-services/openai/concepts/models.md)|[Azure OpenAI models and Standard deployment](../../ai-foundry/model-inference/concepts/models.md)|[Standard deployment](../how-to/model-catalog-overview.md#content-safety-for-models-deployed-via-serverless-apis)|[Open and custom models](../how-to/model-catalog-overview.md#availability-of-models-for-deployment-as-managed-compute)|
30
+
| Which models can be deployed? |[Azure OpenAI models](../../ai-services/openai/concepts/models.md)|[Azure OpenAI models and Standard deployment](../../ai-foundry/model-inference/concepts/models.md)|[Standard deployment](../how-to/model-catalog-overview.md#content-safety-for-models-deployed-via-standard-deployments)|[Open and custom models](../how-to/model-catalog-overview.md#availability-of-models-for-deployment-as-managed-compute)|
31
31
| Deployment resource | Azure OpenAI resource | Azure AI services resource | AI project resource | AI project resource |
32
32
| Requires Hubs/Projects | No | No | Yes | Yes |
33
33
| Data processing options | Regional <br /> Data-zone <br /> Global | Global | Regional | Regional |
| Region | East US/East US2 |[Standard deployments](../how-to/model-catalog-overview.md#serverless-api-pay-per-token-billing) and [Azure OpenAI](/azure/ai-services/openai/overview)|
76
+
| Region | East US/East US2 |[Standard deployments](../how-to/model-catalog-overview.md#standard-deployment-pay-per-token-offer-billing) and [Azure OpenAI](/azure/ai-services/openai/overview)|
77
77
| Tokens per minute (TPM) rate limit | 30k (180 RPM based on Azure OpenAI) for non-reasoning and 100k for reasoning models <br> N/A (standard deployments) | For Azure OpenAI models, selection is available for users with rate limit ranges based on deployment type (standard, global, global standard, and so on.) <br> For standard deployments, this setting is abstracted. |
78
78
| Number of requests | Two requests in a trail for every hour (24 trails per day) | Standard deployments, Azure OpenAI |
79
79
| Number of trails/runs | 14 days with 24 trails per day for 336 runs | Standard deployments, Azure OpenAI |
The next section covers the steps for subscribing your project to a model offering. You can skip this section and go to [Deploy the model to a standard deployment](#deploy-the-model-to-a-serverless-api-endpoint), if you're deploying a Microsoft model.
125
+
The next section covers the steps for subscribing your project to a model offering. You can skip this section and go to [Deploy the model to a standard deployment](#deploy-the-model-to-a-standard-deployment), if you're deploying a Microsoft model.
126
126
127
127
## Subscribe your project to the model offering
128
128
129
-
Standard deployments can deploy both Microsoft and non-Microsoft offered models. For Microsoft models (such as Phi-3 models), you don't need to create an Azure Marketplace subscription and you can [deploy them to standard deployment directly](#deploy-the-model-to-a-serverless-api-endpoint) to consume their predictions. For non-Microsoft models, you need to create the subscription first. If it's your first time deploying the model in the project, you have to subscribe your project for the particular model offering from the Azure Marketplace. Each project has its own subscription to the particular Azure Marketplace offering of the model, which allows you to control and monitor spending.
129
+
Standard deployments can deploy both Microsoft and non-Microsoft offered models. For Microsoft models (such as Phi-3 models), you don't need to create an Azure Marketplace subscription and you can [deploy them to standard deployment directly](#deploy-the-model-to-a-standard-deployment) to consume their predictions. For non-Microsoft models, you need to create the subscription first. If it's your first time deploying the model in the project, you have to subscribe your project for the particular model offering from the Azure Marketplace. Each project has its own subscription to the particular Azure Marketplace offering of the model, which allows you to control and monitor spending.
130
130
131
131
> [!TIP]
132
-
> Skip this step if you are deploying models from the Phi-3 family of models. Directly [deploy the model to a standard deployment](#deploy-the-model-to-a-serverless-api-endpoint).
132
+
> Skip this step if you are deploying models from the Phi-3 family of models. Directly [deploy the model to a standard deployment](#deploy-the-model-to-a-standard-deployment).
133
133
134
134
> [!NOTE]
135
135
> Models offered through the Azure Marketplace are available for deployment to standard deployment in specific regions. Check [Model and region availability for standard deployment](deploy-models-serverless-availability.md) to verify which models and regions are available. If the one you need is not listed, you can deploy to a workspace in a supported region and then [consume standard deployment from a different workspace](deploy-models-serverless-connect.md).
@@ -148,7 +148,7 @@ Standard deployments can deploy both Microsoft and non-Microsoft offered models.
148
148
149
149
:::image type="content" source="../media/deploy-monitor/serverless/deploy-pay-as-you-go.png" alt-text="A screenshot showing how to deploy a model with the standard deployment option." lightbox="../media/deploy-monitor/serverless/deploy-pay-as-you-go.png":::
150
150
151
-
1. If you see the note *You already have an Azure Marketplace subscription for this project*, you don't need to create the subscription since you already have one. You can proceed to [Deploy the model to a standard deployment](#deploy-the-model-to-a-serverless-api-endpoint).
151
+
1. If you see the note *You already have an Azure Marketplace subscription for this project*, you don't need to create the subscription since you already have one. You can proceed to [Deploy the model to a standard deployment](#deploy-the-model-to-a-standard-deployment).
152
152
153
153
1. In the deployment wizard, select the link to **Azure Marketplace Terms** to learn more about the terms of use. You can also select the **Pricing and terms** tab to learn about pricing for the selected model.
| Public Network Access = Enabled | Account key enabled | SAS/Account Key | Yes, UX and SDK |
222
+
| Public Network Access = Enabled | Account key enabled | SAS/Account Key | Yes, UX and SDK |
223
223
| Public Network Access = Enabled | Account key disabled | Entra-Based Auth (Credentialless) | Yes, UX and SDK <br><br> *Note:* for UX, you may need to add Storage Blob Data Reader or Storage Blob Data Contributor for your user ID on the storage account, or change the connection's authentication to use Account key/SAS token ||||
224
224
| Enabled from selected virtual networks and IP addresses | Account key enabled | Account key | Yes, UX and SDK <br><br> *Note:* for UX, the IP of the compute running the browser must be in the selected list |
225
225
| Enabled from selected virtual networks and IP addresses | Account key enabled | SAS | Yes, UX and SDK <br><br> *Note:* for UX, the IP of the compute running the browser must be in the selected list |
@@ -584,9 +584,9 @@ Several enterprise scenarios are supported for standard deployment fine-tuning.
584
584
>- Storage auth can be changed in Azure Storage >**Settings**>**Configurations** page >**Allow storage account key access**.
585
585
>- Storage networking can be changed in Azure Storage >**Networking** page.
| Public Network Access = Enabled | Account key disabled | Entra-Based Auth (Credentialless) | Yes, UXandSDK<br><br>*Note:*forUX, you may need to add Storage Blob Data Reader or Storage Blob Data Contributor for your user ID on the storage account, or change the connection's authentication to use Account key/SAS token | | | |
591
591
| Enabled from selected virtual networks andIP addresses | Account key enabled | Account key | Yes, UXandSDK<br><br>*Note:*forUX, the IP of the compute running the browser must be in the selected list|
592
592
| Enabled from selected virtual networks andIP addresses | Account key enabled |SAS| Yes, UXandSDK<br><br>*Note:*forUX, the IP of the compute running the browser must be in the selected list|
Copy file name to clipboardExpand all lines: articles/ai-foundry/how-to/model-catalog-overview.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -69,7 +69,7 @@ Features | Managed compute | standard deployment
69
69
Deployment experience and billing | Model weights are deployed to dedicated virtual machines with managed compute. A managed compute, which can have one or more deployments, makes available a REST API for inference. You're billed for the virtual machine core hours that the deployments use. | Access to models is through a deployment that provisions an API to access the model. The API provides access to the model that Microsoft hosts and manages, for inference. You're billed for inputs and outputs to the APIs, typically in tokens. Pricing information is provided before you deploy.
70
70
API authentication | Keys and Microsoft Entra authentication. | Keys only.
71
71
Content safety | Use Azure AI Content Safety service APIs. | Azure AI Content Safety filters are available integrated with inference APIs. Azure AI Content Safety filters are billed separately.
72
-
Network isolation | [Configure managed networks for Azure AI Foundry hubs](configure-managed-network.md). | Managed compute follow your hub's public network access (PNA) flag setting. For more information, see the [Network isolation for models deployed via standard deployments](#network-isolation-for-models-deployed-via-serverless-apis) section later in this article.
72
+
Network isolation | [Configure managed networks for Azure AI Foundry hubs](configure-managed-network.md). | Managed compute follow your hub's public network access (PNA) flag setting. For more information, see the [Network isolation for models deployed via standard deployments](#network-isolation-for-models-deployed-via-standard-deployments) section later in this article.
73
73
74
74
### Available models for supported deployment options
75
75
@@ -125,7 +125,7 @@ The [Azure AI Content Safety](../../ai-services/content-safety/overview.md) serv
125
125
126
126
You can refer to [this notebook](https://github.com/Azure/azureml-examples/blob/main/sdk/python/foundation-models/system/inference/text-generation/llama-safe-online-deployment.ipynb) for reference integration with Azure AI Content Safety for Llama 2. Or you can use the Content Safety (Text) tool in prompt flow to pass responses from the model to Azure AI Content Safety for screening. You're billed separately for such use, as described in [Azure AI Content Safety pricing](https://azure.microsoft.com/pricing/details/cognitive-services/content-safety/).
127
127
128
-
## standard deployment (pay per token offer) billing
128
+
## Standard deployment (pay per token offer) billing
129
129
130
130
You can deploy certain models in the model catalog with pay-per-token billing. This deployment method, also called *standard deployment*, provides a way to consume the models as APIs without hosting them on your subscription. Models are hosted in a Microsoft-managed infrastructure, which enables API-based access to the model provider's model. API-based access can dramatically reduce the cost of accessing a model and simplify the provisioning experience.
The next section covers the steps for subscribing your workspace to a model offering. You can skip this section and go to [Deploy the model to a standard deployment](#deploy-the-model-to-a-serverless-api-endpoint), if you're deploying a Microsoft model.
108
+
The next section covers the steps for subscribing your workspace to a model offering. You can skip this section and go to [Deploy the model to a standard deployment](#deploy-the-model-to-a-standard-deployment), if you're deploying a Microsoft model.
109
109
110
110
## Subscribe your workspace to the model offering
111
111
112
-
standard deployments can deploy both Microsoft and non-Microsoft offered models. For Microsoft models (such as Phi-3 models), you don't need to create an Azure Marketplace subscription and you can [deploy them to standard deployments directly](#deploy-the-model-to-a-serverless-api-endpoint) to consume their predictions. For non-Microsoft models, you need to create the subscription first. If it's your first time deploying the model in the workspace, you have to subscribe your workspace for the particular model offering from the Azure Marketplace. Each workspace has its own subscription to the particular Azure Marketplace offering of the model, which allows you to control and monitor spending.
112
+
standard deployments can deploy both Microsoft and non-Microsoft offered models. For Microsoft models (such as Phi-3 models), you don't need to create an Azure Marketplace subscription and you can [deploy them to standard deployments directly](#deploy-the-model-to-a-standard-deployment) to consume their predictions. For non-Microsoft models, you need to create the subscription first. If it's your first time deploying the model in the workspace, you have to subscribe your workspace for the particular model offering from the Azure Marketplace. Each workspace has its own subscription to the particular Azure Marketplace offering of the model, which allows you to control and monitor spending.
113
113
114
114
> [!NOTE]
115
115
> Models offered through the Azure Marketplace are available for deployment to standard deployments in specific regions. Check [Region availability for models in standard deployments](concept-endpoint-serverless-availability.md) to verify which models and regions are available. If the one you need is not listed, you can deploy to a workspace in a supported region and then [consume standard deployments from a different workspace](how-to-connect-models-serverless.md).
@@ -128,7 +128,7 @@ standard deployments can deploy both Microsoft and non-Microsoft offered models.
128
128
129
129
:::image type="content" source="media/how-to-deploy-models-serverless/deploy-pay-as-you-go.png" alt-text="A screenshot showing how to deploy a model with the standard deployment option." lightbox="media/how-to-deploy-models-serverless/deploy-pay-as-you-go.png":::
130
130
131
-
1. If you see the note *You already have an Azure Marketplace subscription for this workspace*, you don't need to create the subscription since you already have one. You can proceed to [Deploy the model to a standard deployment](#deploy-the-model-to-a-serverless-api-endpoint).
131
+
1. If you see the note *You already have an Azure Marketplace subscription for this workspace*, you don't need to create the subscription since you already have one. You can proceed to [Deploy the model to a standard deployment](#deploy-the-model-to-a-standard-deployment).
132
132
133
133
1. In the deployment wizard, select the link to **Azure Marketplace Terms** to learn more about the terms of use. You can also select the **Pricing and terms** tab to learn about pricing for the selected model.
0 commit comments