Skip to content

Commit ef7538e

Browse files
authored
Merge pull request #205998 from guyhay/Elastic-pool-storage2
Updates adding elastic pool storage details
2 parents 0bab7cf + 726f8d7 commit ef7538e

File tree

1 file changed

+12
-6
lines changed

1 file changed

+12
-6
lines changed

articles/synapse-analytics/spark/apache-spark-pool-configurations.md

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
---
22
title: Apache Spark pool concepts
33
description: Introduction to Apache Spark pool sizes and configurations in Azure Synapse Analytics.
4+
ms.topic: conceptual
45
services: synapse-analytics
5-
author: DaniBunny
66
ms.service: synapse-analytics
7-
ms.topic: conceptual
87
ms.subservice: spark
9-
ms.date: 07/7/2022
10-
ms.author: dacoelho
8+
author: guyhay
9+
ms.author: guyhay
1110
ms.reviewer: martinle
11+
ms.date: 07/26/2022
1212
---
1313

1414
# Apache Spark pool configurations in Azure Synapse Analytics
@@ -48,8 +48,14 @@ A Spark pool can be defined with node sizes that range from a Small compute node
4848

4949
## Autoscale
5050

51-
Apache Spark pools provide the ability to automatically scale up and down compute resources based on the amount of activity. When the autoscale feature is enabled, you can set the minimum and maximum number of nodes to scale.
52-
When the autoscale feature is disabled, the number of nodes set will remain fixed. This setting can be altered after pool creation although the instance may need to be restarted.
51+
Apache Spark pools provide the ability to automatically scale up and down compute resources based on the amount of activity. When the autoscale feature is enabled, you can set the minimum and maximum number of nodes to scale. When the autoscale feature is disabled, the number of nodes set will remain fixed. This setting can be altered after pool creation although the instance may need to be restarted.
52+
53+
## Elastic pool storage
54+
55+
Apache Spark pools utilize temporary disk storage while the pool is instantiated. For many Spark jobs, it is difficult to estimate cluster storage requirements, which may cause your Spark jobs to fail if the worker nodes exhaust storage. Elastic pool storage allows the Spark engine to monitor worker node temporary cluster storage, and attach additional disks if needed. No action is required by customers. Customers should see fewer job failures as a result of elastic pool storage.
56+
57+
> [!NOTE]
58+
> Azure Synapse Elastic pool storage is currently in Public Preview. During Public Preview there is no charge for use of Elastic Pool Storage.
5359
5460
## Automatic pause
5561

0 commit comments

Comments
 (0)