Skip to content

Commit 05ecfdd

Browse files
authored
Update spark-dotnet.md
adding the note on removal of .NET Core support in Spark 3.3 and above based on note from this https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-33-runtime
1 parent 82c126b commit 05ecfdd

File tree

1 file changed

+7
-0
lines changed

1 file changed

+7
-0
lines changed

articles/synapse-analytics/spark/spark-dotnet.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,13 @@ It provides .NET bindings for Spark, which allows you to access Spark APIs throu
1919

2020
You can analyze data with .NET for Apache Spark through Spark batch job definitions or with interactive Azure Synapse Analytics notebooks. In this article, you learn how to use .NET for Apache Spark with Azure Synapse using both techniques.
2121

22+
>[!IMPORTANT]
23+
> The [.NET for Apache Spark](https://github.com/dotnet/spark) is an open-source project under the .NET Foundation that currently requires the .NET 3.1 library, which has reached the out-of-support status. We would like to inform users of Azure Synapse Spark of the removal of the .NET for Apache Spark library in the Azure Synapse Runtime for Apache Spark version 3.3. Users may refer to the [.NET Support Policy](https://dotnet.microsoft.com/platform/support/policy/dotnet-core) for more details on this matter.
24+
>
25+
> As a result, it will no longer be possible for users to utilize Apache Spark APIs via C# and F#, or execute C# code in notebooks within Synapse or through Apache Spark Job definitions in Synapse. It is important to note that this change affects only Azure Synapse Runtime for Apache Spark 3.3 and above.
26+
>
27+
> We will continue to support .NET for Apache Spark in all previous versions of the Azure Synapse Runtime according to [their lifecycle stages](runtime-for-apache-spark-lifecycle-and-supportability.md). However, we do not have plans to support .NET for Apache Spark in Azure Synapse Runtime for Apache Spark 3.3 and future versions. We recommend that users with existing workloads written in C# or F# migrate to Python or Scala. Users are advised to take note of this information and plan accordingly.
28+
2229
## Submit batch jobs using the Spark job definition
2330

2431
Visit the tutorial to learn how to use Azure Synapse Analytics to [create Apache Spark job definitions for Synapse Spark pools](apache-spark-job-definitions.md). If you haven't packaged your app to submit to Azure Synapse, complete the following steps.

0 commit comments

Comments
 (0)