Skip to content

Commit 70d98ed

Browse files
committed
Updates adding elastic pool storage details
1 parent 3ef3c30 commit 70d98ed

File tree

1 file changed

+12
-5
lines changed

1 file changed

+12
-5
lines changed

articles/synapse-analytics/spark/apache-spark-pool-configurations.md

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
---
22
title: Apache Spark pool concepts
33
description: Introduction to Apache Spark pool sizes and configurations in Azure Synapse Analytics.
4+
ms.topic: conceptual
45
services: synapse-analytics
5-
author: mlee3gsd
66
ms.service: synapse-analytics
7-
ms.topic: conceptual
87
ms.subservice: spark
9-
ms.date: 08/19/2021
10-
ms.author: martinle
11-
ms.reviewer: euang
8+
author: guyhay
9+
ms.author: guyhay
10+
ms.reviewer: martinle
11+
ms.date: 07/26/2022
1212
---
1313

1414
# Apache Spark pool configurations in Azure Synapse Analytics
@@ -51,6 +51,13 @@ A Spark pool can be defined with node sizes that range from a Small compute node
5151
Apache Spark pools provide the ability to automatically scale up and down compute resources based on the amount of activity. When the autoscale feature is enabled, you can set the minimum and maximum number of nodes to scale.
5252
When the autoscale feature is disabled, the number of nodes set will remain fixed. This setting can be altered after pool creation although the instance may need to be restarted.
5353

54+
## Elastic pool storage
55+
56+
Apache Spark pools utilize temporary disk storage while the pool is instantiated. For many Spark jobs, it is difficult to estimate cluster storage requirements, and in some cases causes your Spark jobs to fail if the worker nodes exhausts it's temporary storage. Elastic pool storage allows the Spark engine to monitor worker node temporary cluster storage, and attach additional disks if needed. No action is required by customers, and customers should see less job failures as a result.
57+
58+
> [!NOTE]
59+
> Azure Synapse Elastic pool storage is currently in Public Preview. During Public Preview there is no charge for use of Elastic Pool Storage.
60+
5461
## Automatic pause
5562

5663
The automatic pause feature releases resources after a set idle period reducing the overall cost of an Apache Spark pool. The number of minutes of idle time can be set once this feature is enabled. The automatic pause feature is independent of the autoscale feature. Resources can be paused whether the autoscale is enabled or disabled. This setting can be altered after pool creation although the instance may need to be restarted.

0 commit comments

Comments
 (0)