You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/synapse-analytics/plan-manage-costs.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -128,9 +128,9 @@ You can control costs for a dedicated SQL pool by pausing the resource when it i
128
128
129
129
### Serverless Apache Spark pool
130
130
131
-
To control costs for your serverless Apache Spark pool, enable the serverless Apache Spark automatic pause feature and set your timeout value accordingly. Choose the [node sizes](../synapse-analytics/spark/apache-spark-pool-configurations.md) appropriately to match your preference for performance vs. cost. Use [autoscale](../synapse-analytics/spark/apache-spark-autoscale.md) to re-size pools when needed. When using Synapse Studio for your development, the studio sends a keep alive message to keep the session active, which is also configurable, so set a short timeout value for automatic pause. When you are done, close your session and the Apache Spark pool will automatically pause once the timeout value is reached. Billing starts for a serverless Apache Spark Pool when it starts thru the ide timeout. Autoscale up or down will increase the pool runtime.
131
+
To control costs for your serverless Apache Spark pool, enable the serverless Apache Spark automatic pause feature and set your timeout value accordingly. Choose the [node sizes](../synapse-analytics/spark/apache-spark-pool-configurations.md) appropriately to match your preference for performance vs. cost. Use [autoscale](../synapse-analytics/spark/apache-spark-autoscale.md) to re-size pools when needed. When using Synapse Studio for your development, the studio sends a keep alive message to keep the session active, which is also configurable, so set a short timeout value for automatic pause. When you are done, close your session and the Apache Spark pool will automatically pause once the timeout value is reached. Billing starts for a serverless Apache Spark Pool when it starts until the ide timeout. Autoscale up or down will increase the pool runtime.
132
132
133
-
During development, create multiple Apache Spark pool definitions of various sizes. Creating Apache Spark pool definitions are free and you will only be charged for usage. Apache Spark usage in Azure Synapse is charged per vCore hour and prorated by the minute. For example, use small pool sizes for code development and validation while using larger pool sizes for performance testing.
133
+
During development, create multiple Apache Spark pool definitions of various sizes. Creating Apache Spark pool definitions are free and you will only be charged for usage. Apache Spark pool usage in Azure Synapse is charged per vCore hour and prorated by the minute. For example, use small pool sizes for code development and validation while using larger pool sizes for performance testing.
Copy file name to clipboardExpand all lines: articles/synapse-analytics/spark/apache-spark-concepts.md
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
---
2
2
title: Apache Spark core concepts
3
3
description: Introduction to core concepts for Apache Spark in Azure Synapse Analytics.
4
-
ms.service: synapse-analytics
4
+
ms.service: synapse-analytics
5
5
ms.subservice: spark
6
6
ms.topic: overview
7
7
author: guyhay
@@ -31,7 +31,7 @@ Spark instances are created when you connect to a Spark pool, create a session,
31
31
32
32
When you submit a second job, if there's capacity in the pool, the existing Spark instance also has capacity. Then, the existing instance will process the job. Otherwise, if capacity is available at the pool level, then a new Spark instance will be created.
33
33
34
-
Billing for the instances starts when the Azure VM(s) starts. Billing for the Spark pool instances stops when pool instances changes to terminating. For more details on how Azure VMs are started, de-allocated see [States and billing status of Azure Virtual Machines](../virtual-machines/states-billing)
34
+
Billing for the instances starts when the Azure VM(s) starts. Billing for the Spark pool instances stops when pool instances changes to terminating. For more details on how Azure VMs are started, de-allocated see [States and billing status of Azure Virtual Machines](../virtual-machines/states-billing)
35
35
36
36
## Examples
37
37
@@ -41,10 +41,10 @@ Billing for the instances starts when the Azure VM(s) starts. Billing for the S
41
41
- You submit a notebook job, J1 that uses 10 nodes, a Spark instance, SI1 is created to process the job
42
42
- You now submit another job, J2, that uses 10 nodes because there's still capacity in the pool and the instance, the J2, is processed by SI1
43
43
- If J2 had asked for 11 nodes, there wouldn't have been capacity in SP1 or SI1. In this case, if J2 comes from a notebook, then the job will be rejected; if J2 comes from a batch job, then it will be queued.
44
-
- Billing starts at the submission of notebook job J1.
44
+
- Billing starts at the submission of notebook job J1.
45
45
- The Spark pool is instantiated with 20 medium nodes, each with 8 vCores, and typically takes ~3 minutes to start. 20 x 8 = 160 vCores.
46
46
- Depending on the exact Spark pool start-up time, idle timeout and the runtime of the two notebook jobs; the pool is likely to run for between 18 and 20 minutes (Spark pool instantiation time + notebook job runtime + idle timeout).
- Note: vCore hours are billed per second, vCore pricing varies by Azure region. For more information, see [Azure Synapse Pricing](https://azure.microsoft.com/pricing/details/synapse-analytics/#pricing)
49
49
50
50
### Example 2
@@ -54,7 +54,7 @@ Billing for the instances starts when the Azure VM(s) starts. Billing for the S
54
54
- You now submit another job J2 that uses 10 nodes; because there's still capacity in the pool the instance autoscales to 20 nodes and processes J2.
55
55
- Billing starts at the submission of notebook job J1.
56
56
- The Spark pool is instantiated with 10 medium nodes, each with 8 vCores, and typically takes ~3 minutes to start. 10 x 8, 80 vCores.
57
-
- At the submission of J2, the pool autoscales by adding another 10 medium nodes, and typically takes 4 minutes to autoscale. Adding 10 x 8, 80 vCores for a total of 160 vCores.
57
+
- At the submission of J2, the pool autoscales by adding another 10 medium nodes, and typically takes 4 minutes to autoscale. Adding 10 x 8, 80 vCores for a total of 160 vCores.
58
58
- Depending on the Spark pool start-up time, runtime of the first notebook job J1, the time to scale-up the pool, runtime of the second notebook, and finally the idle timeout; the pool is likely to run between 22 and 24 minutes (Spark pool instantiation time + J1 notebook job runtime all at 80 vCores) + (Spark pool autoscale-up time + J2 notebook job runtime + idle timeout all at 160 vCores).
59
59
- 80 vCores for 4 minutes + 160 vCores for 20 minutes = 58.67 vCore hours.
60
60
- Note: vCore hours are billed per second, vCore pricing varies by Azure region. For more information, see [Azure Synapse Pricing](https://azure.microsoft.com/pricing/details/synapse-analytics/#pricing)
0 commit comments