Skip to content

Commit 18e6784

Browse files
authored
Merge pull request #300534 from jeacla/docs-editor/apache-spark-version-support-1748473174
Update apache-spark-version-support.md
2 parents d5aa65d + cd38be8 commit 18e6784

File tree

3 files changed

+13
-13
lines changed

3 files changed

+13
-13
lines changed

articles/synapse-analytics/spark/apache-spark-version-support.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: Apache Spark version support
33
description: Supported versions of Spark, Scala, Python
4-
author: ms-arali
4+
author: arali
55
ms.author: arali
66
ms.reviewer: arali
77
ms.date: 03/08/2024
@@ -28,8 +28,8 @@ The following table lists the runtime name, Apache Spark version, and release da
2828

2929
| Runtime name | Release date | Release stage | End of Support announcement date | End of Support effective date |
3030
| --- | --- |------------------------------| --- | --- |
31-
| [Azure Synapse Runtime for Apache Spark 3.5](./apache-spark-35-runtime.md) | N/A | Public Preview | N/A | N/A|
32-
| [Azure Synapse Runtime for Apache Spark 3.4](./apache-spark-34-runtime.md) | Nov 21, 2023 | GA | Q3 2025| Q1 2026|
31+
| [Azure Synapse Runtime for Apache Spark 3.5](./apache-spark-35-runtime.md) | May 27, 2025 | Public Preview | N/A | N/A|
32+
| [Azure Synapse Runtime for Apache Spark 3.4](./apache-spark-34-runtime.md) | Nov 21, 2023 | EOSA | April 30, 2025| Q1 2026|
3333
| [Azure Synapse Runtime for Apache Spark 3.3](./apache-spark-33-runtime.md) | Nov 17, 2022 |**deprecated and soon to be disabled**|July 12, 2024| March 31, 2025
3434

3535
## Runtime release stages
@@ -73,10 +73,10 @@ The patch policy differs based on the [runtime lifecycle stage](./runtime-for-ap
7373

7474
## Migration between Apache Spark versions - support
7575

76-
This guide provides a structured approach for users looking to upgrade their Azure Synapse Runtime for Apache Spark workloads from versions 2.4, 3.1, 3.2, or 3.3 to [the latest GA version, such as 3.4](./apache-spark-34-runtime.md). Upgrading to the most recent version enables users to benefit from performance enhancements, new features, and improved security measures. It's important to note that transitioning to a higher version may require adjustments to your existing Spark code due to incompatibilities or deprecated features.
76+
This guide provides a structured approach for users looking to upgrade their Azure Synapse Runtime for Apache Spark workloads to [the latest GA version, such as 3.4](./apache-spark-34-runtime.md). Upgrading to the most recent version enables users to benefit from performance enhancements, new features, and improved security measures. It's important to note that transitioning to a higher version may require adjustments to your existing Spark code due to incompatibilities or deprecated features.
7777

7878
### Step 1: Evaluate and plan
79-
- **Assess Compatibility:** Start with reviewing Apache Spark migration guides to identify any potential incompatibilities, deprecated features, and new APIs between your current Spark version (2.4, 3.1, 3.2, or 3.3) and the target version (for example, 3.4).
79+
- **Assess Compatibility:** Start with reviewing Apache Spark migration guides to identify any potential incompatibilities, deprecated features, and new APIs between your current Spark version and the target version (for example, 3.4).
8080
- **Analyze Codebase:** Carefully examine your Spark code to identify the use of deprecated or modified APIs. Pay particular attention to SQL queries and User Defined Functions (UDFs), which may be affected by the upgrade.
8181

8282
### Step 2: Create a new Spark pool for testing
@@ -88,15 +88,15 @@ This guide provides a structured approach for users looking to upgrade their Azu
8888
- **Test in Development Environment:** Test your updated code within a development environment in Azure Synapse, not locally. This step is essential for identifying and fixing any issues before moving to production.
8989
- **Deploy and Monitor:** After thorough testing and validation in the development environment, deploy your application to the new Spark 3.4 pool. It's critical to monitor the application for any unexpected behaviors. Utilize the monitoring tools available in Azure Synapse to keep track of your Spark applications' performance.
9090

91-
**Question:** What steps should be taken in migrating from 2.4 to 3.X?
91+
**Question:** What steps should be taken migrating to 3.X?
9292

9393
**Answer:** Refer to the [Apache Spark migration guide](https://spark.apache.org/docs/latest/sql-migration-guide.html).
9494

9595
**Question:** I got an error when I tried to upgrade Spark pool runtime using PowerShell cmdlet when they have attached libraries.
9696

9797
**Answer:** Don't use PowerShell cmdlet if you have custom libraries installed in your Synapse workspace. Instead follow these steps:
98-
1. Recreate Spark Pool 3.3 from the ground up.
99-
1. Downgrade the current Spark Pool 3.3 to 3.1, remove any packages attached, and then upgrade again to 3.3.
98+
1. Recreate your Spark Pool from the ground up.
99+
1. Downgrade the current Spark Pool, remove any packages attached, and then upgrade again to [the latest GA version, such as 3.4](./apache-spark-34-runtime.md)
100100

101101
**Question:** Why can't I upgrade to 3.4 without recreating a new Spark pool?
102102

articles/synapse-analytics/spark/runtime-for-apache-spark-lifecycle-and-supportability.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
---
22
title: Synapse runtime for Apache Spark lifecycle and supportability
33
description: Lifecycle and support policies for Synapse runtime for Apache Spark
4-
author: juluczni
5-
ms.author: juluczni
4+
author: jeacla
5+
ms.author: arali
66
ms.reviewer: arali
7-
ms.date: 03/08/2024
7+
ms.date: 05/30/2025
88
ms.service: azure-synapse-analytics
99
ms.subservice: spark
1010
ms.topic: reference
@@ -16,7 +16,7 @@ Apache Spark pools in Azure Synapse use runtimes to tie together essential compo
1616

1717

1818
> [!CAUTION]
19-
> Azure Synapse Runtime for Apache Spark 3.1, 3.2 and 3.3 are unsupported and deprecated. Using these runtimes after the deprecation date is at one's own risk, and with the agreement and acceptance of the risks that existing jobs running on Apache Spark 3.1, 3.2 or 3.3 pools will eventually stop executing.
19+
> Azure Synapse Runtime for Apache Spark 3.3 is unsupported and deprecated. Using this runtime after the deprecation date is at one's own risk, and with the agreement and acceptance of the risks that existing jobs running on Apache Spark 3.3 pools will eventually stop executing.
2020
2121
## Release cadence
2222

articles/synapse-analytics/toc.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -738,7 +738,7 @@ items:
738738
href: ./spark/apache-spark-version-support.md
739739
- name: Azure Synapse Apache Spark 3.5 runtime (Public Preview)
740740
href: ./spark/apache-spark-35-runtime.md
741-
- name: Azure Synapse Apache Spark 3.4 runtime (GA)
741+
- name: Azure Synapse Apache Spark 3.4 runtime (EOSA)
742742
href: ./spark/apache-spark-34-runtime.md
743743
- name: Azure Synapse Apache Spark 3.3 runtime (unsupported)
744744
href: ./spark/apache-spark-33-runtime.md

0 commit comments

Comments
 (0)