You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/how-to-submit-spark-jobs.md
+11-2Lines changed: 11 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -174,6 +174,9 @@ To create a job, a standalone Spark job can be defined as a YAML specification f
174
174
- `runtime_version` - defines the Spark runtime version. The following Spark runtime versions are currently supported:
175
175
- `3.1`
176
176
- `3.2`
177
+
> [!IMPORTANT]
178
+
>
179
+
> End of life announcement (EOLA) for Azure Synapse Runtime for Apache Spark 3.1 was made on January 26, 2023. In accordance, Apache Spark 3.1 will not be supported after July 31, 2023. We recommend that you use Apache Spark 3.2.
177
180
178
181
An example is shown here:
179
182
```yaml
@@ -313,7 +316,10 @@ To create a standalone Spark job, use the `azure.ai.ml.spark` function, with the
313
316
- `Standard_E64S_V3`
314
317
- `runtime_version` - a key that defines the Spark runtime version. The following Spark runtime versions are currently supported:
315
318
- `3.1.0`
316
-
- `3.2.0`
319
+
- `3.2.0`
320
+
> [!IMPORTANT]
321
+
>
322
+
> End of life announcement (EOLA) for Azure Synapse Runtime for Apache Spark 3.1 was made on January 26, 2023. In accordance, Apache Spark 3.1 will not be supported after July 31, 2023. We recommend that you use Apache Spark 3.2.
317
323
- `compute`- the name of an attached Synapse Spark pool.
318
324
- `inputs` - the inputs for the Spark job. This parameter should pass a dictionary with mappings of the input data bindings used in the job. This dictionary has these values:
319
325
- a dictionary key defines the input name
@@ -412,6 +418,9 @@ To submit a standalone Spark job using the Azure Machine Learning studio UI:
412
418
1. If you selected **Spark automatic compute (Preview)**:
413
419
1. Select **Virtual machine size**.
414
420
1. Select **Spark runtime version**.
421
+
> [!IMPORTANT]
422
+
>
423
+
> End of life announcement (EOLA) for Azure Synapse Runtime for Apache Spark 3.1 was made on January 26, 2023. In accordance, Apache Spark 3.1 will not be supported after July 31, 2023. We recommend that you use Apache Spark 3.2.
415
424
1. If you selected **Attached compute**:
416
425
1. Select an attached Synapse Spark pool from the **Select Azure Machine Learning attached compute** menu.
417
426
1. Select **Next**.
@@ -696,4 +705,4 @@ This functionality isn't available in the Studio UI. The Studio UI doesn't suppo
696
705
## Next steps
697
706
698
707
- [Code samples for Spark jobs using Azure Machine Learning CLI](https://github.com/Azure/azureml-examples/tree/main/cli/jobs/spark)
699
-
- [Code samples for Spark jobs using Azure Machine Learning Python SDK](https://github.com/Azure/azureml-examples/tree/main/sdk/python/jobs/spark)
708
+
- [Code samples for Spark jobs using Azure Machine Learning Python SDK](https://github.com/Azure/azureml-examples/tree/main/sdk/python/jobs/spark)
0 commit comments