Skip to content

Commit 5d861a4

Browse files
dem108dem108
authored
Update Supported VM SKU list for online endpoint
1 parent c879455 commit 5d861a4

File tree

1 file changed

+83
-15
lines changed

1 file changed

+83
-15
lines changed

‎articles/machine-learning/reference-managed-online-endpoints-vm-sku-list.md

Lines changed: 83 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -21,22 +21,90 @@ The following table shows the virtual machine (VM) stock keeping units (SKUs) th
2121

2222
* For more information on configuration details such as CPU and RAM, see [Azure Machine Learning Pricing](https://azure.microsoft.com/pricing/details/machine-learning/) and [VM sizes](/azure/virtual-machines/sizes).
2323

24-
| Relative Size | General Purpose | Compute Optimized | Memory Optimized | GPU |
25-
| --- | --- | --- | --- | --- |
26-
| X-Small | Standard_DS1_v2 <br/> Standard_DS2_v2 <br/> Standard_D2a_v4 <br/> Standard_D2as_v4 | Standard_F2s_v2 | Standard_E2s_v3 | Standard_NC4as_T4_v3 |
27-
| Small | Standard_DS3_v2 <br/> Standard_D4a_v4 <br/> Standard_D4as_v4 | Standard_F4s_v2 <br/> Standard_FX4mds | Standard_E4s_v3 | Standard_NC6s_v2 <br/> Standard_NC6s_v3 <br/> Standard_NC8as_T4_v3 |
28-
| Medium | Standard_DS4_v2 </br> Standard_D8a_v4 </br> Standard_D8as_v4 | Standard_F8s_v2 </br> Standard_FX12mds | Standard_E8s_v3 | Standard_NC12s_v2 <br/> Standard_NC12s_v3 <br/> Standard_NC16as_T4_v3 |
29-
| Large | Standard_DS5_v2 </br> Standard_D16a_v4 </br> Standard_D16as_v4 | Standard_F16s_v2 | Standard_E16s_v3 | Standard_NC24s_v2 <br/> Standard_NC24s_v3 <br/> Standard_NC64as_T4_v3 </br> Standard_NC24ads_A100_v4 |
30-
| X-Large | Standard_D32a_v4 </br> Standard_D32as_v4 </br> Standard_D48a_v4 </br> Standard_D48as_v4 </br> Standard_D64a_v4 </br> Standard_D64as_v4 </br> Standard_D96a_v4 </br> Standard_D96as_v4 | Standard_F32s_v2 <br/> Standard_F48s_v2 <br/> Standard_F64s_v2 <br/> Standard_F72s_v2 <br/> Standard_FX24mds <br/> Standard_FX36mds <br/> Standard_FX48mds | Standard_E32s_v3 <br/> Standard_E48s_v3 <br/> Standard_E64s_v3 | Standard_NC48ads_A100_v4 </br> Standard_NC96ads_A100_v4 </br> Standard_ND96asr_v4 </br> Standard_ND96amsr_A100_v4 </br> Standard_ND40rs_v2 |
24+
| Family Name | VM Size Name | Supports Infiniband | Architecture | numberOfGPUs | numberOfCores | Skip 20% Reservation |
25+
| --- | --- | --- | --- | --- | --- | --- |
26+
| standardDASv4Family | STANDARD_D2AS_V4 | - | Cpu | 0 | 2 | - |
27+
| standardDASv4Family | STANDARD_D4AS_V4 | - | Cpu | 0 | 4 | - |
28+
| standardDASv4Family | STANDARD_D8AS_V4 | - | Cpu | 0 | 8 | - |
29+
| standardDASv4Family | STANDARD_D16AS_V4 | - | Cpu | 0 | 16 | - |
30+
| standardDASv4Family | STANDARD_D32AS_V4 | - | Cpu | 0 | 32 | - |
31+
| standardDASv4Family | STANDARD_D48AS_V4 | - | Cpu | 0 | 48 | - |
32+
| standardDASv4Family | STANDARD_D64AS_V4 | - | Cpu | 0 | 64 | - |
33+
| standardDASv4Family | STANDARD_D96AS_V4 | - | Cpu | 0 | 96 | - |
34+
| standardDAv4Family | STANDARD_D2A_V4 | - | Cpu | 0 | 2 | - |
35+
| standardDAv4Family | STANDARD_D4A_V4 | - | Cpu | 0 | 4 | - |
36+
| standardDAv4Family | STANDARD_D8A_V4 | - | Cpu | 0 | 8 | - |
37+
| standardDAv4Family | STANDARD_D16A_V4 | - | Cpu | 0 | 16 | - |
38+
| standardDAv4Family | STANDARD_D32A_V4 | - | Cpu | 0 | 32 | - |
39+
| standardDAv4Family | STANDARD_D48A_V4 | - | Cpu | 0 | 48 | - |
40+
| standardDAv4Family | STANDARD_D64A_V4 | - | Cpu | 0 | 64 | - |
41+
| standardDAv4Family | STANDARD_D96A_V4 | - | Cpu | 0 | 96 | - |
42+
| standardDSv2Family | STANDARD_DS1_V2 | - | Cpu | 0 | 1 | - |
43+
| standardDSv2Family | STANDARD_DS2_V2 | - | Cpu | 0 | 2 | - |
44+
| standardDSv2Family | STANDARD_DS3_V2 | - | Cpu | 0 | 4 | - |
45+
| standardDSv2Family | STANDARD_DS4_V2 | - | Cpu | 0 | 8 | - |
46+
| standardDSv2Family | STANDARD_DS5_V2 | - | Cpu | 0 | 16 | - |
47+
| standardESv3Family | STANDARD_E2S_V3 | - | Cpu | 0 | 2 | - |
48+
| standardESv3Family | STANDARD_E4S_V3 | - | Cpu | 0 | 4 | - |
49+
| standardESv3Family | STANDARD_E8S_V3 | - | Cpu | 0 | 8 | - |
50+
| standardESv3Family | STANDARD_E16S_V3 | - | Cpu | 0 | 16 | - |
51+
| standardESv3Family | STANDARD_E32S_V3 | - | Cpu | 0 | 32 | - |
52+
| standardESv3Family | STANDARD_E48S_V3 | - | Cpu | 0 | 48 | - |
53+
| standardESv3Family | STANDARD_E64S_V3 | - | Cpu | 0 | 64 | - |
54+
| standardFSv2Family | STANDARD_F2S_V2 | - | Cpu | 0 | 2 | - |
55+
| standardFSv2Family | STANDARD_F4S_V2 | - | Cpu | 0 | 4 | - |
56+
| standardFSv2Family | STANDARD_F8S_V2 | - | Cpu | 0 | 8 | - |
57+
| standardFSv2Family | STANDARD_F16S_V2 | - | Cpu | 0 | 16 | - |
58+
| standardFSv2Family | STANDARD_F32S_V2 | - | Cpu | 0 | 32 | - |
59+
| standardFSv2Family | STANDARD_F48S_V2 | - | Cpu | 0 | 48 | - |
60+
| standardFSv2Family | STANDARD_F64S_V2 | - | Cpu | 0 | 64 | - |
61+
| standardFSv2Family | STANDARD_F72S_V2 | - | Cpu | 0 | 72 | - |
62+
| standardFXMDVSFamily | STANDARD_FX4MDS | - | Cpu | 0 | 4 | - |
63+
| standardFXMDVSFamily | STANDARD_FX12MDS | - | Cpu | 0 | 12 | - |
64+
| standardFXMDVSFamily | STANDARD_FX24MDS | - | Cpu | 0 | 24 | - |
65+
| standardFXMDVSFamily | STANDARD_FX36MDS | - | Cpu | 0 | 36 | - |
66+
| standardFXMDVSFamily | STANDARD_FX48MDS | - | Cpu | 0 | 48 | - |
67+
| standardLASv3Family | STANDARD_L8AS_V3 | - | Cpu | 0 | 8 | - |
68+
| standardLASv3Family | STANDARD_L16AS_V3 | - | Cpu | 0 | 16 | - |
69+
| standardLASv3Family | STANDARD_L32AS_V3 | - | Cpu | 0 | 32 | - |
70+
| standardLASv3Family | STANDARD_L48AS_V3 | - | Cpu | 0 | 48 | - |
71+
| standardLASv3Family | STANDARD_L64AS_V3 | - | Cpu | 0 | 64 | - |
72+
| standardLASv3Family | STANDARD_L80AS_V3 | - | Cpu | 0 | 80 | - |
73+
| standardLSv2Family | STANDARD_L8S_V2 | - | Cpu | 0 | 8 | - |
74+
| standardLSv2Family | STANDARD_L16S_V2 | - | Cpu | 0 | 16 | - |
75+
| standardLSv2Family | STANDARD_L32S_V2 | - | Cpu | 0 | 32 | - |
76+
| standardLSv2Family | STANDARD_L48S_V2 | - | Cpu | 0 | 48 | - |
77+
| standardLSv2Family | STANDARD_L64S_V2 | - | Cpu | 0 | 64 | - |
78+
| standardLSv2Family | STANDARD_L80S_V2 | - | Cpu | 0 | 80 | - |
79+
| standardLSv3Family | STANDARD_L8S_V3 | - | Cpu | 0 | 8 | - |
80+
| standardLSv3Family | STANDARD_L16S_V3 | - | Cpu | 0 | 16 | - |
81+
| standardLSv3Family | STANDARD_L32S_V3 | - | Cpu | 0 | 32 | - |
82+
| standardLSv3Family | STANDARD_L48S_V3 | - | Cpu | 0 | 48 | - |
83+
| standardLSv3Family | STANDARD_L64S_V3 | - | Cpu | 0 | 64 | - |
84+
| standardLSv3Family | STANDARD_L80S_V3 | - | Cpu | 0 | 80 | - |
85+
| standardNCADSA100v4Family | STANDARD_NC24ADS_A100_V4 | - | NvidiaGpu | 1 | 24 | Yes |
86+
| standardNCADSA100v4Family | STANDARD_NC48ADS_A100_V4 | - | NvidiaGpu | 2 | 48 | Yes |
87+
| standardNCADSA100v4Family | STANDARD_NC96ADS_A100_V4 | - | NvidiaGpu | 4 | 96 | Yes |
88+
| Standard NCASv3_T4 Family | STANDARD_NC4AS_T4_V3 | - | NvidiaGpu | 1 | 4 | - |
89+
| Standard NCASv3_T4 Family | STANDARD_NC8AS_T4_V3 | - | NvidiaGpu | 1 | 8 | - |
90+
| Standard NCASv3_T4 Family | STANDARD_NC16AS_T4_V3 | - | NvidiaGpu | 1 | 16 | - |
91+
| Standard NCASv3_T4 Family | STANDARD_NC64AS_T4_V3 | - | NvidiaGpu | 4 | 64 | - |
92+
| standardNCSv2Family | STANDARD_NC6S_V2 | - | NvidiaGpu | 1 | 6 | - |
93+
| standardNCSv2Family | STANDARD_NC12S_V2 | - | NvidiaGpu | 2 | 12 | - |
94+
| standardNCSv2Family | STANDARD_NC24S_V2 | - | NvidiaGpu | 4 | 24 | - |
95+
| standardNCSv3Family | STANDARD_NC6S_V3 | - | NvidiaGpu | 1 | 6 | - |
96+
| standardNCSv3Family | STANDARD_NC12S_V3 | - | NvidiaGpu | 2 | 12 | - |
97+
| standardNCSv3Family | STANDARD_NC24S_V3 | - | NvidiaGpu | 4 | 24 | - |
98+
| standardNCADSH100v5Family | STANDARD_NC40ADS_H100_V5 | - | NvidiaGpu | 1 | 40 | Yes |
99+
| standardNCADSH100v5Family | STANDARD_NC80ADIS_H100_V5 | - | NvidiaGpu | 2 | 80 | Yes |
100+
| standard NDAMSv4_A100Family | STANDARD_ND96AMSR_A100_V4 | Yes | NvidiaGpu | 8 | 96 | Yes |
101+
| Standard NDASv4_A100 Family | STANDARD_ND96ASR_V4 | Yes | NvidiaGpu | 8 | 96 | Yes |
102+
| standardNDSv2Family | STANDARD_ND40RS_V2 | Yes | NvidiaGpu | 8 | 40 | Yes |
103+
| standardNDv5H100Family | STANDARD_ND96IS_H100_v5 | - | NvidiaGpu | 8 | 96 | Yes |
104+
| standardNDv5H100Family | STANDARD_ND96ISR_H100_v5 | Yes | NvidiaGpu | 8 | 96 | Yes |
31105

32106
> [!CAUTION]
33-
> `Standard_DS1_v2` and `Standard_F2s_v2` may be too small for bigger models and may lead to container termination due to insufficient memory, not enough space on the disk, or probe failure as it takes too long to initiate the container. If you face [OutOfQuota errors](how-to-troubleshoot-online-endpoints.md?tabs=cli#error-outofquota) or [ReourceNotReady errors](how-to-troubleshoot-online-endpoints.md?tabs=cli#error-resourcenotready), try bigger VM SKUs. If you want to reduce the cost of deploying multiple models with managed online endpoint, see [Deployment for several local models](concept-online-deployment-model-specification.md#deployment-for-several-local-models).
107+
> Small VM SKUs such as `Standard_DS1_v2` and `Standard_F2s_v2` may be too small for bigger models and may lead to container termination due to insufficient memory, not enough space on the disk, or probe failure as it takes too long to initiate the container. If you face [OutOfQuota errors](how-to-troubleshoot-online-endpoints.md?tabs=cli#error-outofquota) or [ReourceNotReady errors](how-to-troubleshoot-online-endpoints.md?tabs=cli#error-resourcenotready), try bigger VM SKUs. If you want to reduce the cost of deploying multiple models with managed online endpoint, see [Deployment for several local models](concept-online-deployment-model-specification.md#deployment-for-several-local-models).
34108
35109
> [!NOTE]
36-
> We recommend having more than 3 instances for deployments in production scenarios. In addition, Azure Machine Learning reserves 20% of your compute resources for performing upgrades on some VM SKUs as described in [Virtual machine quota allocation for deployment](how-to-manage-quotas.md#virtual-machine-quota-allocation-for-deployment). VM SKUs that are exempted from this extra quota reservation are listed below:
37-
> - Standard_NC24ads_A100_v4
38-
> - Standard_NC48ads_A100_v4
39-
> - Standard_NC96ads_A100_v4
40-
> - Standard_ND96asr_v4
41-
> - Standard_ND96amsr_A100_v4
42-
> - Standard_ND40rs_v2
110+
> We recommend having more than 3 instances for deployments in production scenarios. In addition, Azure Machine Learning reserves 20% of your compute resources for performing upgrades on some VM SKUs as described in [Virtual machine quota allocation for deployment](how-to-manage-quotas.md#virtual-machine-quota-allocation-for-deployment). VM SKUs that are exempted from this extra quota reservation are specified in the "Skip 20% Reservation" column.

0 commit comments

Comments
 (0)