Skip to content

Commit a0d1d93

Browse files
20240311 spark support in quickstarts, edit pass
1 parent 8ad970f commit a0d1d93

File tree

2 files changed

+81
-77
lines changed

2 files changed

+81
-77
lines changed
Lines changed: 45 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,20 @@
11
---
2-
title: 'Quickstart: Create a serverless Apache Spark pool using the Azure portal'
2+
title: "Quickstart: Create a serverless Apache Spark pool using the Azure portal"
33
description: Create a serverless Apache Spark pool using the Azure portal by following the steps in this guide.
44
author: WilliamDAssafMSFT
5+
ms.author: wiassaf
6+
ms.reviewer: sngun, eskot
7+
ms.date: 03/11/2024
58
ms.service: synapse-analytics
6-
ms.topic: quickstart
79
ms.subservice: spark
8-
ms.date: 08/19/2021
9-
ms.author: wiassaf
10-
ms.reviewer: sngun
11-
ms.custom: mode-ui
10+
ms.topic: quickstart
11+
ms.custom:
12+
- mode-ui
1213
---
1314

1415
# Quickstart: Create a new serverless Apache Spark pool using the Azure portal
1516

16-
Azure Synapse Analytics offers various analytics engines to help you ingest, transform, model, analyze, and distribute your data. An Apache Spark pool provides open-source big data compute capabilities. After you've created an Apache Spark pool in your Synapse workspace, data can be loaded, modeled, processed, and distributed for faster analytic insight.
17+
Azure Synapse Analytics offers various analytics engines to help you ingest, transform, model, analyze, and distribute your data. An Apache Spark pool provides open-source big data compute capabilities. After you create an Apache Spark pool in your Synapse workspace, data can be loaded, modeled, processed, and distributed for faster analytic insight.
1718

1819
In this quickstart, you learn how to use the Azure portal to create an Apache Spark pool in a Synapse workspace.
1920

@@ -25,78 +26,80 @@ If you don't have an Azure subscription, [create a free account before you begin
2526
## Prerequisites
2627

2728
- You'll need an Azure subscription. If needed, [create a free Azure account](https://azure.microsoft.com/free/)
28-
- [Synapse Analytics workspace](quickstart-create-workspace.md)
29+
- You'll be using the [Synapse workspace](./quickstart-create-workspace.md).
2930

3031
## Sign in to the Azure portal
3132

3233
Sign in to the [Azure portal](https://portal.azure.com/)
3334

34-
## Navigate to the Synapse workspace
35-
1. Navigate to the Synapse workspace where the Apache Spark pool will be created by typing the service name (or resource name directly) into the search bar.
35+
## Navigate to the Synapse workspace
3636

37-
![Azure portal search bar with Synapse workspaces typed in.](media/quickstart-create-sql-pool/create-sql-pool-00a.png)
37+
1. Navigate to the Synapse workspace where the Apache Spark pool will be created by typing the service name (or resource name directly) into the search bar.
38+
:::image type="content" source="media/quickstart-create-sql-pool/create-sql-pool-00a.png" alt-text="Screenshot of the Azure portal search bar with Synapse workspaces typed in." lightbox="media/quickstart-create-sql-pool/create-sql-pool-00a.png":::
3839

39-
1. From the list of workspaces, type the name (or part of the name) of the workspace to open. For this example, we'll use a workspace named **contosoanalytics**.
40-
![Listing of Synapse workspaces filtered to show those containing the name Contoso.](media/quickstart-create-sql-pool/create-sql-pool-00b.png)
40+
1. From the list of workspaces, type the name (or part of the name) of the workspace to open. For this example, we use a workspace named **contosoanalytics**.
41+
:::image type="content" source="media/quickstart-create-sql-pool/create-sql-pool-00b.png" alt-text="Screenshot from the Azure portal of the list of Synapse workspaces filtered to show those containing the name Contoso." lightbox="media/quickstart-create-sql-pool/create-sql-pool-00b.png":::
4142

4243

4344
## Create new Apache Spark pool
4445

4546
> [!IMPORTANT]
46-
> Azure Synapse Runtime for Apache Spark 2.4 has been deprecated and officially not supported since 09/29/2023. Given [Spark 3.1](/azure/synapse-analytics/spark/apache-spark-3-runtime) and [Spark 3.2](/azure/synapse-analytics/spark/apache-spark-32-runtime) are also End of Life announced (EOLA), we recommend customers migrate to Spark 3.3. Please refer to [this link](/azure/synapse-analytics/spark/apache-spark-33-runtime) for a migration guide.
47-
1. In the Synapse workspace where you want to create the Apache Spark pool, select **New Apache Spark pool**.
48-
![Overview of Synapse workspace with a red box around the command to create a new Apache Spark pool](media/quickstart-create-apache-spark-pool/create-spark-pool-portal-01.png)
47+
> Azure Synapse Runtime for Apache Spark 2.4 has been deprecated and officially not supported since September 2023. Given [Spark 3.1](/azure/synapse-analytics/spark/apache-spark-3-runtime) and [Spark 3.2](/azure/synapse-analytics/spark/apache-spark-32-runtime) are also End of Support announced, [we recommend customers migrate to Spark 3.3](/azure/synapse-analytics/spark/apache-spark-33-runtime).
48+
49+
1. In the Synapse workspace where you want to create the Apache Spark pool, select **New Apache Spark pool**.
50+
:::image type="content" source="media/quickstart-create-apache-spark-pool/create-spark-pool-portal-01.png" alt-text="Screenshot from the Azure portal of a Synapse workspace with a red box around the command to create a new Apache Spark pool." lightbox="media/quickstart-create-apache-spark-pool/create-spark-pool-portal-01.png":::
51+
4952
1. Enter the following details in the **Basics** tab:
5053

5154
|Setting | Suggested value | Description |
5255
| :------ | :-------------- | :---------- |
53-
| **Apache Spark pool name** | A valid pool name | This is the name that the Apache Spark pool will have. |
56+
| **Apache Spark pool name** | A valid pool name, like `contosospark` | This is the name that the Apache Spark pool will have. |
5457
| **Node size** | Small (4 vCPU / 32 GB) | Set this to the smallest size to reduce costs for this quickstart |
5558
| **Autoscale** | Disabled | We don't need autoscale for this quickstart |
5659
| **Number of nodes** | 5 | Use a small size to limit costs for this quickstart |
5760

58-
59-
![Apache Spark pool create flow - basics tab.](media/quickstart-create-apache-spark-pool/create-spark-pool-portal-02.png)
61+
:::image type="content" source="media/quickstart-create-apache-spark-pool/create-spark-pool-portal-02.png" alt-text="Screenshot from the Azure portal of the Apache Spark pool create flow - basics tab." lightbox="media/quickstart-create-apache-spark-pool/create-spark-pool-portal-02.png":::
6062

61-
> [!IMPORTANT]
62-
> Note that there are specific limitations for the names that Apache Spark pools can use. Names must contain letters or numbers only, must be 15 or less characters, must start with a letter, not contain reserved words, and be unique in the workspace.
63+
> [!IMPORTANT]
64+
> There are specific limitations for the names that Apache Spark pools can use. Names must contain letters or numbers only, must be 15 or less characters, must start with a letter, not contain reserved words, and be unique in the workspace.
6365

64-
1. Select **Next: additional settings** and review the default settings. Do not modify any default settings.
65-
![Screenshot that shows the "Create Apache Spark pool" page with the "Additional settings" tab selected.](media/quickstart-create-apache-spark-pool/create-spark-pool-portal-03.png)
66+
1. Select **Next: additional settings** and review the default settings. Don't modify any default settings.
67+
:::image type="content" source="media/quickstart-create-apache-spark-pool/create-spark-pool-portal-03.png" alt-text="Screenshot from the Azure portal that shows the 'Create Apache Spark pool' page with the 'Additional settings' tab selected." lightbox="media/quickstart-create-apache-spark-pool/create-spark-pool-portal-03.png":::
6668

67-
1. Select **Next: tags**. Don't add any tags.
68-
![Apache Spark pool create flow - additional settings tab.](media/quickstart-create-apache-spark-pool/create-spark-pool-03-tags.png)
69+
1. Select **Next: tags**. Consider using Azure tags. For example, the "Owner" or "CreatedBy" tag to identify who created the resource, and the "Environment" tag to identify whether this resource is in Production, Development, etc. For more information, see [Develop your naming and tagging strategy for Azure resources](/azure/cloud-adoption-framework/ready/azure-best-practices/naming-and-tagging).
70+
:::image type="content" source="media/quickstart-create-apache-spark-pool/create-spark-pool-03-tags.png" alt-text="Screenshot from the Azure portal of Apache Spark pool create flow - additional settings tab." lightbox="media/quickstart-create-apache-spark-pool/create-spark-pool-03-tags.png":::
6971

70-
5. Select **Review + create**.
72+
1. Select **Review + create**.
7173

7274
1. Make sure that the details look correct based on what was previously entered, and select **Create**.
73-
![Apache Spark pool create flow - review settings tab.](media/quickstart-create-apache-spark-pool/create-spark-pool-portal-05.png)
75+
:::image type="content" source="media/quickstart-create-apache-spark-pool/create-spark-pool-portal-05.png" alt-text="Screenshot from the Azure portal of Apache Spark pool create flow - review settings tab." lightbox="media/quickstart-create-apache-spark-pool/create-spark-pool-portal-05.png":::
7476

7577
1. At this point, the resource provisioning flow will start, indicating once it's complete.
76-
![Screenshot that shows the "Overview" page with a "Your deployment is complete" message displayed.](media/quickstart-create-apache-spark-pool/create-spark-pool-portal-06.png)
78+
:::image type="content" source="media/quickstart-create-apache-spark-pool/create-spark-pool-portal-06.png" alt-text="Screenshot from the Azure portal of that shows the 'Overview' page with a 'Your deployment is complete' message displayed." lightbox="media/quickstart-create-apache-spark-pool/create-spark-pool-portal-06.png":::
7779

7880
1. After the provisioning completes, navigating back to the workspace will show a new entry for the newly created Apache Spark pool.
79-
![Apache Spark pool create flow - resource provisioning.](media/quickstart-create-apache-spark-pool/create-spark-pool-portal-07.png)
81+
:::image type="content" source="media/quickstart-create-apache-spark-pool/create-spark-pool-portal-07.png" alt-text="Screenshot from the Azure portal of Apache Spark pool create flow - resource provisioning." lightbox="media/quickstart-create-apache-spark-pool/create-spark-pool-portal-07.png":::
8082

81-
9. At this point, there are no resources running, no charges for Spark, you have created metadata about the Spark instances you want to create.
83+
1. At this point, there are no resources running, no charges for Spark, you have created metadata about the Spark instances you want to create.
8284

8385
## Clean up resources
8486

85-
Follow the steps below to delete the Apache Spark pool from the workspace.
87+
The following steps delete the Apache Spark pool from the workspace.
88+
8689
> [!WARNING]
8790
> Deleting an Apache Spark pool will remove the analytics engine from the workspace. It will no longer be possible to connect to the pool, and all queries, pipelines, and notebooks that use this Apache Spark pool will no longer work.
8891
89-
If you want to delete the Apache Spark pool, do the following:
92+
If you want to delete the Apache Spark pool, do the following steps:
9093

91-
1. Navigate to the Apache Spark pools blade in the workspace.
92-
2. Select the Apache Spark pool to be deleted (in this case, **contosospark**).
93-
1. Press **delete**.
94-
![Listing of Apache Spark pools, with the recently created pool selected.](media/quickstart-create-apache-spark-pool/create-spark-pool-portal-08.png)
95-
1. Confirm the deletion, and press **Delete** button.
96-
![Confirmation dialog to delete the selected Apache Spark pool.](media/quickstart-create-apache-spark-pool/create-spark-pool-portal-10.png)
97-
5. When the process completes successfully, the Apache Spark pool will no longer be listed in the workspace resources.
94+
1. Navigate to the Apache Spark pools pane in the workspace.
95+
1. Select the Apache Spark pool to be deleted (in this case, **contosospark**).
96+
1. Select **Delete**.
97+
:::image type="content" source="media/quickstart-create-apache-spark-pool/create-spark-pool-portal-08.png" alt-text="Screenshot from the Azure portal of a list of Apache Spark pools, with the recently created pool selected." lightbox="media/quickstart-create-apache-spark-pool/create-spark-pool-portal-08.png":::
98+
1. Confirm the deletion, and select **Delete** button.
99+
:::image type="content" source="media/quickstart-create-apache-spark-pool/create-spark-pool-portal-10.png" alt-text="Screenshot from the Azure portal of the Confirmation dialog to delete the selected Apache Spark pool.":::
100+
1. When the process completes successfully, the Apache Spark pool will no longer be listed in the workspace resources.
98101

99-
## Next steps
102+
## Related content
100103

101-
- See [Quickstart: Create an Apache Spark notebook](quickstart-apache-spark-notebook.md).
102-
- See [Quickstart: Create a dedicated SQL pool using the Azure portal](quickstart-create-sql-pool-portal.md).
104+
- [Quickstart: Create a serverless Apache Spark pool in Azure Synapse Analytics using web tools](quickstart-apache-spark-notebook.md)
105+
- [Quickstart: Create a dedicated SQL pool using the Azure portal](quickstart-create-sql-pool-portal.md)

0 commit comments

Comments
 (0)