You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
|[Azure Synapse Runtime for Apache Spark 3.5](./apache-spark-35-runtime.md)|N/A| Public Preview | N/A | N/A|
32
-
|[Azure Synapse Runtime for Apache Spark 3.4](./apache-spark-34-runtime.md)| Nov 21, 2023 |GA|Q3 2025| Q1 2026|
31
+
|[Azure Synapse Runtime for Apache Spark 3.5](./apache-spark-35-runtime.md)|May 27, 2025| Public Preview | N/A | N/A|
32
+
|[Azure Synapse Runtime for Apache Spark 3.4](./apache-spark-34-runtime.md)| Nov 21, 2023 |EOSA|April 30, 2025| Q1 2026|
33
33
| [Azure Synapse Runtime for Apache Spark 3.3](./apache-spark-33-runtime.md) | Nov 17, 2022 |**deprecated and soon to be disabled**|July 12, 2024| March 31, 2025
34
34
35
35
## Runtime release stages
@@ -73,10 +73,10 @@ The patch policy differs based on the [runtime lifecycle stage](./runtime-for-ap
73
73
74
74
## Migration between Apache Spark versions - support
75
75
76
-
This guide provides a structured approach for users looking to upgrade their Azure Synapse Runtime for Apache Spark workloads from versions 2.4, 3.1, 3.2, or 3.3 to [the latest GA version, such as 3.4](./apache-spark-34-runtime.md). Upgrading to the most recent version enables users to benefit from performance enhancements, new features, and improved security measures. It's important to note that transitioning to a higher version may require adjustments to your existing Spark code due to incompatibilities or deprecated features.
76
+
This guide provides a structured approach for users looking to upgrade their Azure Synapse Runtime for Apache Spark workloads to [the latest GA version, such as 3.4](./apache-spark-34-runtime.md). Upgrading to the most recent version enables users to benefit from performance enhancements, new features, and improved security measures. It's important to note that transitioning to a higher version may require adjustments to your existing Spark code due to incompatibilities or deprecated features.
77
77
78
78
### Step 1: Evaluate and plan
79
-
-**Assess Compatibility:** Start with reviewing Apache Spark migration guides to identify any potential incompatibilities, deprecated features, and new APIs between your current Spark version (2.4, 3.1, 3.2, or 3.3) and the target version (for example, 3.4).
79
+
-**Assess Compatibility:** Start with reviewing Apache Spark migration guides to identify any potential incompatibilities, deprecated features, and new APIs between your current Spark version and the target version (for example, 3.4).
80
80
-**Analyze Codebase:** Carefully examine your Spark code to identify the use of deprecated or modified APIs. Pay particular attention to SQL queries and User Defined Functions (UDFs), which may be affected by the upgrade.
81
81
82
82
### Step 2: Create a new Spark pool for testing
@@ -88,15 +88,15 @@ This guide provides a structured approach for users looking to upgrade their Azu
88
88
-**Test in Development Environment:** Test your updated code within a development environment in Azure Synapse, not locally. This step is essential for identifying and fixing any issues before moving to production.
89
89
-**Deploy and Monitor:** After thorough testing and validation in the development environment, deploy your application to the new Spark 3.4 pool. It's critical to monitor the application for any unexpected behaviors. Utilize the monitoring tools available in Azure Synapse to keep track of your Spark applications' performance.
90
90
91
-
**Question:** What steps should be taken in migrating from 2.4 to 3.X?
91
+
**Question:** What steps should be taken migrating to 3.X?
92
92
93
93
**Answer:** Refer to the [Apache Spark migration guide](https://spark.apache.org/docs/latest/sql-migration-guide.html).
94
94
95
95
**Question:** I got an error when I tried to upgrade Spark pool runtime using PowerShell cmdlet when they have attached libraries.
96
96
97
97
**Answer:** Don't use PowerShell cmdlet if you have custom libraries installed in your Synapse workspace. Instead follow these steps:
98
-
1. Recreate Spark Pool 3.3 from the ground up.
99
-
1. Downgrade the current Spark Pool 3.3 to 3.1, remove any packages attached, and then upgrade again to 3.3.
98
+
1. Recreate your Spark Pool from the ground up.
99
+
1. Downgrade the current Spark Pool, remove any packages attached, and then upgrade again to [the latest GA version, such as 3.4](./apache-spark-34-runtime.md)
100
100
101
101
**Question:** Why can't I upgrade to 3.4 without recreating a new Spark pool?
Copy file name to clipboardExpand all lines: articles/synapse-analytics/spark/runtime-for-apache-spark-lifecycle-and-supportability.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,10 +1,10 @@
1
1
---
2
2
title: Synapse runtime for Apache Spark lifecycle and supportability
3
3
description: Lifecycle and support policies for Synapse runtime for Apache Spark
4
-
author: juluczni
5
-
ms.author: juluczni
4
+
author: jeacla
5
+
ms.author: arali
6
6
ms.reviewer: arali
7
-
ms.date: 03/08/2024
7
+
ms.date: 05/30/2025
8
8
ms.service: azure-synapse-analytics
9
9
ms.subservice: spark
10
10
ms.topic: reference
@@ -16,7 +16,7 @@ Apache Spark pools in Azure Synapse use runtimes to tie together essential compo
16
16
17
17
18
18
> [!CAUTION]
19
-
> Azure Synapse Runtime for Apache Spark 3.1, 3.2 and 3.3 are unsupported and deprecated. Using these runtimes after the deprecation date is at one's own risk, and with the agreement and acceptance of the risks that existing jobs running on Apache Spark 3.1, 3.2 or 3.3 pools will eventually stop executing.
19
+
> Azure Synapse Runtime for Apache Spark 3.3 is unsupported and deprecated. Using this runtime after the deprecation date is at one's own risk, and with the agreement and acceptance of the risks that existing jobs running on Apache Spark 3.3 pools will eventually stop executing.
0 commit comments