Skip to content

Commit 66a80d8

Browse files
Merge pull request #268502 from guyhay/Update-for-billing-clarity
Changing per second to per minute
2 parents fd0154a + b8db86a commit 66a80d8

File tree

1 file changed

+6
-6
lines changed

1 file changed

+6
-6
lines changed

articles/synapse-analytics/spark/apache-spark-concepts.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ ms.subservice: spark
66
ms.topic: overview
77
author: guyhay
88
ms.author: guyhay
9-
ms.date: 02/09/2023
9+
ms.date: 04/02/2024
1010
---
1111

1212
# Apache Spark in Azure Synapse Analytics Core Concepts
@@ -29,7 +29,7 @@ You can read how to create a Spark pool and see all their properties here [Get s
2929

3030
Spark instances are created when you connect to a Spark pool, create a session, and run a job. As multiple users may have access to a single Spark pool, a new Spark instance is created for each user that connects.
3131

32-
When you submit a second job, if there's capacity in the pool, the existing Spark instance also has capacity. Then, the existing instance will process the job. Otherwise, if capacity is available at the pool level, then a new Spark instance will be created.
32+
When you submit a second job, if there's capacity in the pool, the existing Spark instance also has capacity. Then, the existing instance processes the job. Otherwise, if capacity is available at the pool level, a new Spark instance is created.
3333

3434
Billing for the instances starts when the Azure VM(s) starts. Billing for the Spark pool instances stops when pool instances change to terminating. For more information on how Azure VMs are started and deallocated, see [States and billing status of Azure Virtual Machines](/azure/virtual-machines/states-billing).
3535

@@ -40,12 +40,12 @@ Billing for the instances starts when the Azure VM(s) starts. Billing for the Sp
4040
- You create a Spark pool called SP1; it has a fixed cluster size of 20 medium nodes
4141
- You submit a notebook job, J1 that uses 10 nodes, a Spark instance, SI1 is created to process the job
4242
- You now submit another job, J2, that uses 10 nodes because there's still capacity in the pool and the instance, the J2, is processed by SI1
43-
- If J2 had asked for 11 nodes, there wouldn't have been capacity in SP1 or SI1. In this case, if J2 comes from a notebook, then the job will be rejected; if J2 comes from a batch job, then it will be queued.
43+
- If J2 had asked for 11 nodes, there wouldn't have been capacity in SP1 or SI1. In this case, if J2 comes from a notebook, then the job is rejected; if J2 comes from a batch job, it is queued.
4444
- Billing starts at the submission of notebook job J1.
4545
- The Spark pool is instantiated with 20 medium nodes, each with 8 vCores, and typically takes ~3 minutes to start. 20 x 8 = 160 vCores.
4646
- Depending on the exact Spark pool start-up time, idle timeout and the runtime of the two notebook jobs; the pool is likely to run for between 18 and 20 minutes (Spark pool instantiation time + notebook job runtime + idle timeout).
4747
- Assuming 20-minute runtime, 160 x 0.3 hours = 48 vCore hours.
48-
- Note: vCore hours are billed per second, vCore pricing varies by Azure region. For more information, see [Azure Synapse Pricing](https://azure.microsoft.com/pricing/details/synapse-analytics/#pricing)
48+
- Note: vCore hours are billed per minute and vCore pricing varies by Azure region. For more information, see [Azure Synapse Pricing](https://azure.microsoft.com/pricing/details/synapse-analytics/#pricing)
4949

5050
### Example 2
5151

@@ -57,7 +57,7 @@ Billing for the instances starts when the Azure VM(s) starts. Billing for the Sp
5757
- At the submission of J2, the pool autoscales by adding another 10 medium nodes, and typically takes 4 minutes to autoscale. Adding 10 x 8, 80 vCores for a total of 160 vCores.
5858
- Depending on the Spark pool start-up time, runtime of the first notebook job J1, the time to scale-up the pool, runtime of the second notebook, and finally the idle timeout; the pool is likely to run between 22 and 24 minutes (Spark pool instantiation time + J1 notebook job runtime all at 80 vCores) + (Spark pool autoscale-up time + J2 notebook job runtime + idle timeout all at 160 vCores).
5959
- 80 vCores for 4 minutes + 160 vCores for 20 minutes = 58.67 vCore hours.
60-
- Note: vCore hours are billed per second, vCore pricing varies by Azure region. For more information, see [Azure Synapse Pricing](https://azure.microsoft.com/pricing/details/synapse-analytics/#pricing)
60+
- Note: vCore hours are billed per minute and vCore pricing varies by Azure region. For more information, see [Azure Synapse Pricing](https://azure.microsoft.com/pricing/details/synapse-analytics/#pricing)
6161

6262
### Example 3
6363

@@ -71,7 +71,7 @@ Billing for the instances starts when the Azure VM(s) starts. Billing for the Sp
7171
- Another Spark pool SI2 is instantiated with 20 medium nodes, each with 8 vCores, and typically takes ~3 minutes to start. 20 x 8, 160 vCores
7272
- Depending on the exact Spark pool start-up time, the ide timeout and the runtime of the first notebook job; The SI2 pool is likely to run for between 18 and 20 minutes (Spark pool instantiation time + notebook job runtime + idle timeout).
7373
- Assuming the two pools run for 20 minutes each, 160 x .03 x 2 = 96 vCore hours.
74-
- Note: vCore hours are billed per second, vCore pricing varies by Azure region. For more information, see [Azure Synapse Pricing](https://azure.microsoft.com/pricing/details/synapse-analytics/#pricing)
74+
- Note: vCore hours are billed per minute and vCore pricing varies by Azure region. For more information, see [Azure Synapse Pricing](https://azure.microsoft.com/pricing/details/synapse-analytics/#pricing)
7575

7676
## Quotas and resource constraints in Apache Spark for Azure Synapse
7777

0 commit comments

Comments
 (0)