Skip to content

Commit 5674302

Browse files
v-albemiv-albemi
authored
Merge pull request #97104 from hrasheed-msft/hdi_minvmsizes
minimum recommended sizes
2 parents c6b39c4 + 9ee604e commit 5674302

File tree

1 file changed

+31
-15
lines changed

1 file changed

+31
-15
lines changed

‎articles/hdinsight/hdinsight-supported-node-configuration.md

Lines changed: 31 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -14,19 +14,33 @@ ms.date: 08/26/2019
1414

1515
This article discusses default and recommended node configurations for Azure HDInsight clusters.
1616

17-
## Default and recommended node configuration and virtual machine sizes for clusters
17+
## Default and minimum recommended node configuration and virtual machine sizes for clusters
1818

19-
The following tables list default and recommended virtual machine (VM) sizes for HDInsight clusters. This information is necessary to understand the VM sizes to use when you are creating PowerShell or Azure CLI scripts to deploy HDInsight clusters.
19+
The following tables list default and recommended virtual machine (VM) sizes for HDInsight clusters. This information is necessary to understand the VM sizes to use when you are creating PowerShell or Azure CLI scripts to deploy HDInsight clusters.
2020

2121
If you need more than 32 worker nodes in a cluster, select a head node size with at least 8 cores and 14 GB of RAM.
2222

2323
The only cluster types that have data disks are Kafka and HBase clusters with the Accelerated Writes feature enabled. HDInsight supports P30 and S30 disk sizes in these scenarios.
2424

25-
For more details on the specification of each VM type, see the following documents:
25+
The specifications of all minimum recommended VM types used in this document are summarized in the following table.
2626

27-
* [General purpose virtual machine sizes: Dv2 series 1-5](https://docs.microsoft.com/azure/virtual-machines/linux/sizes-general#dv2-series)
28-
* [Memory optimized virtual machine sizes: Dv2 series 11-15](https://docs.microsoft.com/azure/virtual-machines/linux/sizes-memory#dv2-series-11-15)
29-
* [General purpose virtual machine sizes: Av2 series 1-8](https://docs.microsoft.com/azure/virtual-machines/linux/sizes-general#av2-series)
27+
| Size | vCPU | Memory: GiB | Temp storage (SSD) GiB | Max temp storage throughput: IOPS / Read MBps / Write MBps | Max data disks / throughput: IOPS | Max NICs / Expected network bandwidth (Mbps) |
28+
|-------------------|-----------|-------------|----------------|----------------------------------------------------------|-----------------------------------|------------------------------|
29+
| Standard_D3_v2 | 4 | 14 | 200 | 12000 / 187 / 93 | 16 / 16x500 | 4 / 3000 |
30+
| Standard_D4_v2 | 8 | 28 | 400 | 24000 / 375 / 187 | 32 / 32x500 | 8 / 6000 |
31+
| Standard_D5_v2 | 16 | 56 | 800 | 48000 / 750 / 375 | 64 / 64x500 | 8 / 12000 |
32+
| Standard_D12_v2 | 4 | 28 | 200 | 12000 / 187 / 93 | 16 / 16x500 | 4 / 3000 |
33+
| Standard_D13_v2 | 8 | 56 | 400 | 24000 / 375 / 187 | 32 / 32x500 | 8 / 6000 |
34+
| Standard_D14_v2 | 16 | 112 | 800 | 48000 / 750 / 375 | 64 / 64x500 | 8 / 12000 |
35+
| Standard_A1_v2 | 1         | 2           | 10             | 1000 / 20 / 10                                           | 2 / 2x500               | 2 / 250                 |
36+
| Standard_A2_v2 | 2 | 4 | 20 | 2000 / 40 / 20 | 4 / 4x500 | 2 / 500 |
37+
| Standard_A4_v2 | 4 | 8 | 40 | 4000 / 80 / 40 | 8 / 8x500 | 4 / 1000 |
38+
39+
For more details on the specifications of each VM type, see the following documents:
40+
41+
* [General purpose virtual machine sizes: Dv2 series 1-5](../virtual-machines/linux/sizes-general.md#dv2-series)
42+
* [Memory optimized virtual machine sizes: Dv2 series 11-15](../virtual-machines/linux/sizes-memory.md#dv2-series-11-15)
43+
* [General purpose virtual machine sizes: Av2 series 1-8](../virtual-machines/linux/sizes-general.md#av2-series)
3044

3145
### All supported regions except Brazil south and Japan west
3246

@@ -35,27 +49,29 @@ For more details on the specification of each VM type, see the following documen
3549
3650
| Cluster type | Hadoop | HBase | Interactive Query | Storm | Spark | ML Server | Kafka |
3751
|---|---|---|---|---|---|---|---|
38-
| Head: default VM size | D12_v2 | D12_v2 | D13_v2 | A4_v2 | D12_v2 | D12_v2 | D3_v2 |
39-
| Head: recommended VM sizes | D13_v2,<br/>D14_v2,<br/>D5_v2 | D3_v2,<br/>D4_v2,<br/>D12_v2 | D13_v2,<br/>D14_v2 | A4_v2,<br/>A8_v2 | D12_v2,<br/>D13_v2,<br/>D14_v2 | D12_v2,<br/>D13_v2,<br/>D14_v2 | D3_v2,<br/>D4_v2,<br/>D12_v2 |
52+
| Head: default VM size | D12_v2 | D12_v2 | D13_v2 | A4_v2 | D12_v2, <br/>D13_v2* | D12_v2 | D3_v2 |
53+
| Head: minimum recommended VM sizes | D5_v2 | D3_v2 | D13_v2 | A4_v2 | D12_v2, <br/>D13_v2* | D12_v2 | D3_v2 |
4054
| Worker: default VM size | D4_v2 | D4_v2 | D14_v2 | D3_v2 | D13_v2 | D4_v2 | 4 D12_v2 with 2 S30 disks per broker |
41-
| Worker: recommended VM sizes | D5_v2,<br>D12_v2,<br/>D13_v2 | D3_v2,<br/>D4_v2,<br/>D13_v2 | D13_v2,<br/>D14_v2 | D3_v2<br/>D4_v2,<br/>D12_v2 | D12_v2,<br>D13_v2,<br>D14_v2 | D4_v2,<br/>D12_v2,<br>D13_v2,<br>D14_v2 | D3_v2,<br/>D4_v2,<br/>DS3_v2,<br/>DS4_v2 |
55+
| Worker: minimum recommended VM sizes | D5_v2 | D3_v2 | D13_v2 | D3_v2 | D12_v2 | D4_v2 | D3_v2 |
4256
| ZooKeeper: default VM size | | A4_v2 | A4_v2 | A4_v2 | | A2_v2 | A4_v2 |
43-
| ZooKeeper: recommended VM sizes | | A4_v2, <br/>A8_v2, <br/>A2m_v2 | A4_v2,<br/>A8_v2,<br/>A2m_v2 | A4_v2,<br/>A2_v2,<br/>A8_v2 | | A2_v2 | A4_v2,<br/> A8_v2,<br/>A2m_v2 |
57+
| ZooKeeper: minimum recommended VM sizes | | A4_v2 | A4_v2 | A2_v2 | | A2_v2 | A4_v2 |
4458
| ML Services: default VM size | | | | | | D4_v2 | |
45-
| ML Services: recommended VM size | | | | | | D4_v2,<br/> D12_v2,<br/> D13_v2,<br/>D14_v2 | |
59+
| ML Services: minimum recommended VM size | | | | | | D4_v2 | |
60+
61+
\* = VM Sizes for Spark Enterprise Security Package (ESP) clusters
4662

4763
### Brazil south and Japan west only
4864

4965
| Cluster type | Hadoop | HBase | Interactive Query | Storm | Spark | ML Services |
5066
|---|---|---|---|---|---|---|
5167
| Head: default VM size | D12 | D12 | D13 | A4_v2 | D12 | D12 |
52-
| Head: recommended VM sizes | D5_v2,<br/> D13_v2,<br/> D14_v2 | D3_v2,<br/> D4_v2,<br/> D12_v2 | D13_v2,<br/> D14_v2 | A4_v2,<br/> A8_v2 | D12_v2,<br/> D13_v2,<br/> D14_v2 | D12_v2,<br/> D13_v2,<br/> D14_v2 |
68+
| Head: minimum recommended VM sizes | D5_v2 | D3_v2 | D13_v2 | A4_v2 | D12_v2 | D12_v2 |
5369
| Worker: default VM size | D4 | D4 | D14 | D3 | D13 | D4 |
54-
| Worker: recommended VM sizes | D5_v2,<br/> D12_v2,<br/> D13_v2 | D3_v2,<br/> D4_v2,<br/> D13_v2 | D13_v2,<br/> D14_v2 | D3_v2,<br/> D4_v2,<br/> D12_v2 | D12_v2,<br/> D13_v2,<br/> D14_v2 | D4_v2,<br/> D12_v2,<br/> D13_v2,<br/> D14_v2 |
70+
| Worker: minimum recommended VM sizes | D5_v2 | D3_v2 | D13_v2 | D3_v2 | D12_v2 | D4_v2 |
5571
| ZooKeeper: default VM size | | A4_v2 | A4_v2 | A4_v2 | | A2_v2 |
56-
| ZooKeeper: recommended VM sizes | | A4_v2,<br/> A8_v2,<br/> A2m_v2 | A4_v2,<br/> A8_v2,<br/> A2m_v2 | A4_v2,<br/> A8_v2 | | A2_v2 |
72+
| ZooKeeper: minimum recommended VM sizes | | A4_v2 | A4_v2 | A4_v2 | | A2_v2 |
5773
| ML Services: default VM sizes | | | | | | D4 |
58-
| ML Services: recommended VM sizes | | | | | | D4_v2,<br/> D12_v2,<br/> D13_v2,<br/> D14_v2 |
74+
| ML Services: minimum recommended VM sizes | | | | | | D4_v2 |
5975

6076
> [!NOTE]
6177
> - Head is known as *Nimbus* for the Storm cluster type.

0 commit comments

Comments
 (0)