You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/concept-endpoints-online.md
+7-7Lines changed: 7 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,7 @@ author: s-polly
10
10
ms.author: scottpolly
11
11
ms.reviewer: sehan
12
12
ms.custom: devplatv2, FY25Q1-Linter
13
-
ms.date: 09/23/2024
13
+
ms.date: 10/06/2025
14
14
15
15
#Customer intent: As an ML pro, I want to understand what an online endpoint is and why I need it.
16
16
---
@@ -48,12 +48,12 @@ Managed online endpoints are the recommended way to use online endpoints in Azur
48
48
|Attributes |Managed online endpoints (v2) |Container Instances or AKS (v1) |
49
49
|---------|---------|---------|
50
50
|Network security/isolation |Easy inbound/outbound control with quick toggle |Virtual network not supported or requires complex manual configuration |
51
-
|Managed service |• Fully managed compute provisioning/scaling<br>• Network configuration for data exfiltration prevention<br>• Host OS upgrade, controlled rollout of in-place updates |• Scaling is limited<br>• User must manage network configuration or upgrade |
51
+
|Managed service |* Fully managed compute provisioning/scaling<br>* Network configuration for data exfiltration prevention<br>* Host OS upgrade, controlled rollout of in-place updates |* Scaling is limited<br>* User must manage network configuration or upgrade |
52
52
|Endpoint/deployment concept |Distinction between endpoint and deployment enables complex scenarios such as safe rollout of models |No concept of endpoint |
53
-
|Diagnostics and Monitoring |• Local endpoint debugging possible with Docker and Visual Studio Code<br>• Advanced metrics and logs analysis with chart/query to compare between deployments<br>• Cost breakdown to deployment level |No easy local debugging |
54
-
|Scalability |Elastic, and automatic scaling (not bound by the default cluster size) |• Container Instances isn't scalable <br>• AKS v1 supports in-cluster scale only and requires scalability configuration |
53
+
|Diagnostics and Monitoring |* Local endpoint debugging possible with Docker and Visual Studio Code<br>* Advanced metrics and logs analysis with chart/query to compare between deployments<br>* Cost breakdown to deployment level |No easy local debugging |
54
+
|Scalability |Elastic, and automatic scaling (not bound by the default cluster size) |* Container Instances isn't scalable <br>* AKS v1 supports in-cluster scale only and requires scalability configuration |
|Advanced ML features |• Model data collection<br>• Model monitoring<br>• Champion-challenger model, safe rollout, traffic mirroring<br>• Responsible AI extensibility |Not supported |
56
+
|Advanced ML features |* Model data collection<br>* Model monitoring<br>* Champion-challenger model, safe rollout, traffic mirroring<br>* Responsible AI extensibility |Not supported |
57
57
58
58
### Managed online endpoints vs Kubernetes online endpoints
59
59
@@ -79,7 +79,7 @@ Managed online endpoints can help streamline your deployment process and provide
79
79
> [!NOTE]
80
80
> Managed online endpoints are based on Azure Machine Learning compute. When you use a managed online endpoint, you pay for the compute and networking charges. There's no added surcharge. For more information on pricing, see the [Azure pricing calculator](https://azure.microsoft.com/pricing/calculator/).
81
81
>
82
-
> If you use an Azure Machine Learning virtual network to secure outbound traffic from the managed online endpoint, you're charged for the Azure private link and fully-qualified domain name (FQDN) outbound rules that the managed virtual network uses. For more information, see [Pricing for managed virtual network](how-to-managed-network.md#pricing).
82
+
> If you use an Azure Machine Learning virtual network to secure outbound traffic from the managed online endpoint, you're charged for the Azure private link and fullyqualified domain name (FQDN) outbound rules that the managed virtual network uses. For more information, see [Pricing for managed virtual network](how-to-managed-network.md#pricing).
83
83
84
84
The following table highlights the key differences between managed online endpoints and Kubernetes online endpoints.
85
85
@@ -150,7 +150,7 @@ The following table describes the key attributes of a deployment:
To deploy Llama-2, Phi, Nemotron, Mistral, Dolly, and Deci-DeciLM models from the model catalog by using the shared quota, you must have an [Enterprise Agreement subscription](/azure/cost-management-billing/manage/create-enterprise-subscription). For more information on how to use the shared quota for online endpoint deployment, see [How to deploy foundation models using the studio](how-to-use-foundation-models.md#shared-quota).
153
+
To deploy Llama, Phi, Mistral, and other models from the model catalog by using the shared quota, you must have an [Enterprise Agreement subscription](/azure/cost-management-billing/manage/create-enterprise-subscription). For more information on how to use the shared quota for online endpoint deployment, see [How to deploy foundation models using the studio](how-to-use-foundation-models.md#shared-quota).
154
154
155
155
For more information on quotas and limits for resources in Azure Machine Learning, see [Manage and increase quotas and limits for resources with Azure Machine Learning](how-to-manage-quotas.md).
0 commit comments