You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Set up clusters in HDInsight with Apache Hadoop, Apache Spark, Apache Kafka, and more
@@ -243,17 +243,9 @@ For more information, see [Sizes for virtual machines](../virtual-machines/sizes
243
243
244
244
### Disk attachment
245
245
246
-
HDInsight cluster comes with pre-defined disk space based on SKU. This space may not be sufficient in large job scenarios.
246
+
HDInsight cluster comes with pre-defined disk space based on SKU. Running some large applications, can lead to insufficient disk space, (with disk full error - ```LinkId=221672#ERROR_NOT_ENOUGH_DISK_SPACE```) and job failures.
247
247
248
-
This new feature allows you to add more disks in cluster, which will be used as node manager local directory. Add number of disks to worker nodes during HIVE and Spark cluster creation, while the selected disks will be part of node manager’s local directories.
249
-
250
-
On each of the **NodeManager** machines, **LocalResources** are ultimately localized in the target directories.
251
-
252
-
By normal configuration only the default disk is added as the local disk in NodeManager. For large applications this disk space may not be enough which can result in job failure.
253
-
254
-
If the cluster is expected to run large data application, you can choose to add extra disks to the **NodeManager**.
255
-
256
-
You can add number of disks per VM and each disk will be of 1 TB size.
248
+
More discs can be added to the cluster using the new feature **NodeManager**’s local directory. At the time of Hive and Spark cluster creation, the number of discs can be selected and added to the worker nodes. The selected disk, which will be of size 1TB each, would be part of **NodeManager**'s local directories.
0 commit comments