Skip to content

Commit 51a32c9

Browse files
authored
Fix non-standard formatting
1 parent 1ff0fa8 commit 51a32c9

File tree

1 file changed

+9
-15
lines changed

1 file changed

+9
-15
lines changed

articles/synapse-analytics/spark/apache-spark-version-support.md

Lines changed: 9 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -86,34 +86,28 @@ The patch policy differs based on the [runtime lifecycle stage](./runtime-for-ap
8686

8787
This guide provides a structured approach for users looking to upgrade their Azure Synapse Runtime for Apache Spark workloads from versions 2.4, 3.1, 3.2, or 3.3 to [the latest GA version, such as 3.4](./apache-spark-34-runtime.md). Upgrading to the most recent version enables users to benefit from performance enhancements, new features, and improved security measures. It is important to note that transitioning to a higher version may require adjustments to your existing Spark code due to incompatibilities or deprecated features.
8888

89-
### Step 1: Evaluate and Plan
89+
### Step 1: Evaluate and plan
9090
- **Assess Compatibility:** Start with reviewing Apache Spark migration guides to identify any potential incompatibilities, deprecated features, and new APIs between your current Spark version (2.4, 3.1, 3.2, or 3.3) and the target version (e.g., 3.4).
9191
- **Analyze Codebase:** Carefully examine your Spark code to identify the use of deprecated or modified APIs. Pay particular attention to SQL queries and User Defined Functions (UDFs), which may be affected by the upgrade.
9292

93-
### Step 2: Create a New Spark Pool for Testing
93+
### Step 2: Create a new Spark pool for testing
9494
- **Create a New Pool:** In Azure Synapse, go to the Spark pools section and set up a new Spark pool. Select the target Spark version (e.g., 3.4) and configure it according to your performance requirements.
9595
- **Configure Spark Pool Configuration:** Ensure that all libraries and dependencies in your new Spark pool are updated or replaced to be compatible with Spark 3.4.
9696

97-
### Step 3: Migrate and Test Your Code
97+
### Step 3: Migrate and test your code
9898
- **Migrate Code:** Update your code to be compliant with the new or revised APIs in Apache Spark 3.4. This involves addressing deprecated functions and adopting new features as detailed in the official Apache Spark documentation.
9999
- **Test in Development Environment:** Test your updated code within a development environment in Azure Synapse, not locally. This step is essential for identifying and fixing any issues before moving to production.
100100
- **Deploy and Monitor:** After thorough testing and validation in the development environment, deploy your application to the new Spark 3.4 pool. It is critical to monitor the application for any unexpected behaviors. Utilize the monitoring tools available in Azure Synapse to keep track of your Spark applications' performance.
101101

102-
---
103-
104-
**Question:** What steps should be taken in migrating from 2.4 to 3.X?
102+
**Question:** What steps should be taken in migrating from 2.4 to 3.X? <br>
105103
**Answer:** Refer to the [Apache Spark migration guide](https://spark.apache.org/docs/latest/sql-migration-guide.html).
106104

107-
---
108-
109-
**Question:** I got an error when I tried to upgrade Spark pool runtime using PowerShell cmdlet when they have attached libraries.
110-
**Answer:** Don't use PowerShell cmdlet if you have custom libraries installed in your Synapse workspace. Instead follow these steps:
111-
1. Recreate Spark Pool 3.3 from the ground up.
112-
1. Downgrade the current Spark Pool 3.3 to 3.1, remove any packages attached, and then upgrade again to 3.3.
113-
114-
---
105+
**Question:** I got an error when I tried to upgrade Spark pool runtime using PowerShell cmdlet when they have attached libraries. <br>
106+
**Answer:** Don't use PowerShell cmdlet if you have custom libraries installed in your Synapse workspace. Instead follow these steps: <br>
107+
1. Recreate Spark Pool 3.3 from the ground up.
108+
1. Downgrade the current Spark Pool 3.3 to 3.1, remove any packages attached, and then upgrade again to 3.3.
115109

116110
## Related content
117111

118112
- [Manage libraries for Apache Spark in Azure Synapse Analytics](apache-spark-azure-portal-add-libraries.md)
119-
- [Synapse runtime for Apache Spark lifecycle and supportability](runtime-for-apache-spark-lifecycle-and-supportability.md)
113+
- [Synapse runtime for Apache Spark lifecycle and supportability](runtime-for-apache-spark-lifecycle-and-supportability.md)

0 commit comments

Comments
 (0)