Skip to content

Commit 7042fc2

Browse files
authored
Update apache-spark-version-support.md
Removing all instances of Azure Synapse for Apache Spark 2.4 -3.2 runtime content as per leadership team. Any questions please contact [email protected]
1 parent beaa237 commit 7042fc2

File tree

1 file changed

+5
-5
lines changed

1 file changed

+5
-5
lines changed

articles/synapse-analytics/spark/apache-spark-version-support.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -73,10 +73,10 @@ The patch policy differs based on the [runtime lifecycle stage](./runtime-for-ap
7373

7474
## Migration between Apache Spark versions - support
7575

76-
This guide provides a structured approach for users looking to upgrade their Azure Synapse Runtime for Apache Spark workloads from versions 2.4, 3.1, 3.2, or 3.3 to [the latest GA version, such as 3.4](./apache-spark-34-runtime.md). Upgrading to the most recent version enables users to benefit from performance enhancements, new features, and improved security measures. It's important to note that transitioning to a higher version may require adjustments to your existing Spark code due to incompatibilities or deprecated features.
76+
This guide provides a structured approach for users looking to upgrade their Azure Synapse Runtime for Apache Spark workloads to [the latest GA version, such as 3.4](./apache-spark-34-runtime.md). Upgrading to the most recent version enables users to benefit from performance enhancements, new features, and improved security measures. It's important to note that transitioning to a higher version may require adjustments to your existing Spark code due to incompatibilities or deprecated features.
7777

7878
### Step 1: Evaluate and plan
79-
- **Assess Compatibility:** Start with reviewing Apache Spark migration guides to identify any potential incompatibilities, deprecated features, and new APIs between your current Spark version (2.4, 3.1, 3.2, or 3.3) and the target version (for example, 3.4).
79+
- **Assess Compatibility:** Start with reviewing Apache Spark migration guides to identify any potential incompatibilities, deprecated features, and new APIs between your current Spark version and the target version (for example, 3.4).
8080
- **Analyze Codebase:** Carefully examine your Spark code to identify the use of deprecated or modified APIs. Pay particular attention to SQL queries and User Defined Functions (UDFs), which may be affected by the upgrade.
8181

8282
### Step 2: Create a new Spark pool for testing
@@ -88,15 +88,15 @@ This guide provides a structured approach for users looking to upgrade their Azu
8888
- **Test in Development Environment:** Test your updated code within a development environment in Azure Synapse, not locally. This step is essential for identifying and fixing any issues before moving to production.
8989
- **Deploy and Monitor:** After thorough testing and validation in the development environment, deploy your application to the new Spark 3.4 pool. It's critical to monitor the application for any unexpected behaviors. Utilize the monitoring tools available in Azure Synapse to keep track of your Spark applications' performance.
9090

91-
**Question:** What steps should be taken in migrating from 2.4 to 3.X?
91+
**Question:** What steps should be taken migrating to 3.X?
9292

9393
**Answer:** Refer to the [Apache Spark migration guide](https://spark.apache.org/docs/latest/sql-migration-guide.html).
9494

9595
**Question:** I got an error when I tried to upgrade Spark pool runtime using PowerShell cmdlet when they have attached libraries.
9696

9797
**Answer:** Don't use PowerShell cmdlet if you have custom libraries installed in your Synapse workspace. Instead follow these steps:
98-
1. Recreate Spark Pool 3.3 from the ground up.
99-
1. Downgrade the current Spark Pool 3.3 to 3.1, remove any packages attached, and then upgrade again to 3.3.
98+
1. Recreate your Spark Pool from the ground up.
99+
1. Downgrade the current Spark Pool, remove any packages attached, and then upgrade again to [the latest GA version, such as 3.4](./apache-spark-34-runtime.md)
100100

101101
**Question:** Why can't I upgrade to 3.4 without recreating a new Spark pool?
102102

0 commit comments

Comments
 (0)