Skip to content

Commit cf28127

Browse files
author
Estera Kot
committed
3.3 EOSA updates
1 parent 89a0727 commit cf28127

File tree

1 file changed

+20
-19
lines changed

1 file changed

+20
-19
lines changed

articles/machine-learning/how-to-submit-spark-jobs.md

Lines changed: 20 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -169,20 +169,20 @@ To create a job, a standalone Spark job can be defined as a YAML specification f
169169
- `standard_e32s_v3`
170170
- `standard_e64s_v3`
171171
- `runtime_version` - defines the Spark runtime version. The following Spark runtime versions are currently supported:
172-
- `3.2`
173172
- `3.3`
173+
- `3.4`
174174
> [!IMPORTANT]
175175
> Azure Synapse Runtime for Apache Spark: Announcements
176-
> * Azure Synapse Runtime for Apache Spark 3.2:
177-
> * EOLA Announcement Date: July 8, 2023
178-
> * End of Support Date: July 8, 2024. After this date, the runtime will be disabled.
179-
> * For continued support and optimal performance, we advise migrating to Apache Spark 3.3.
176+
> * Azure Synapse Runtime for Apache Spark 3.3:
177+
> * EOLA Announcement Date: July 12, 2024
178+
> * End of Support Date: March 31, 2025. After this date, the runtime will be disabled.
179+
> * For continued support and optimal performance, we advise migrating to Apache Spark 3.4.
180180

181181
This is an example:
182182
```yaml
183183
resources:
184184
instance_type: standard_e8s_v3
185-
runtime_version: "3.3"
185+
runtime_version: "3.4"
186186
```
187187
- `compute` - this property defines the name of an attached Synapse Spark pool, as shown in this example:
188188
```yaml
@@ -263,7 +263,7 @@ identity:
263263
264264
resources:
265265
instance_type: standard_e4s_v3
266-
runtime_version: "3.3"
266+
runtime_version: "3.4"
267267
```
268268

269269
> [!NOTE]
@@ -315,14 +315,14 @@ To create a standalone Spark job, use the `azure.ai.ml.spark` function, with the
315315
- `Standard_E32S_V3`
316316
- `Standard_E64S_V3`
317317
- `runtime_version` - a key that defines the Spark runtime version. The following Spark runtime versions are currently supported:
318-
- `3.2.0`
319318
- `3.3.0`
319+
- `3.4.0`
320320
> [!IMPORTANT]
321321
> Azure Synapse Runtime for Apache Spark: Announcements
322-
> * Azure Synapse Runtime for Apache Spark 3.2:
323-
> * EOLA Announcement Date: July 8, 2023
324-
> * End of Support Date: July 8, 2024. After this date, the runtime will be disabled.
325-
> * For continued support and optimal performance, we advise migrating to Apache Spark 3.3.
322+
> * Azure Synapse Runtime for Apache Spark 3.3:
323+
> * EOLA Announcement Date: July 12, 2024
324+
> * End of Support Date: March 31, 2025. After this date, the runtime will be disabled.
325+
> * For continued support and optimal performance, we advise migrating to Apache Spark 3.4.
326326

327327
- `compute` - the name of an attached Synapse Spark pool.
328328
- `inputs` - the inputs for the Spark job. This parameter should pass a dictionary with mappings of the input data bindings used in the job. This dictionary has these values:
@@ -375,7 +375,7 @@ spark_job = spark(
375375
executor_instances=2,
376376
resources={
377377
"instance_type": "Standard_E8S_V3",
378-
"runtime_version": "3.3.0",
378+
"runtime_version": "3.4.0",
379379
},
380380
inputs={
381381
"titanic_data": Input(
@@ -426,10 +426,11 @@ To submit a standalone Spark job using the Azure Machine Learning studio UI:
426426
2. Select **Spark runtime version**.
427427
> [!IMPORTANT]
428428
> Azure Synapse Runtime for Apache Spark: Announcements
429-
> * Azure Synapse Runtime for Apache Spark 3.2:
430-
> * EOLA Announcement Date: July 8, 2023
431-
> * End of Support Date: July 8, 2024. After this date, the runtime will be disabled.
432-
> * For continued support and optimal performance, we advise migrating to Apache Spark 3.3.
429+
> * Azure Synapse Runtime for Apache Spark 3.3:
430+
> * EOLA Announcement Date: July 12, 2024
431+
> * End of Support Date: March 31, 2025. After this date, the runtime will be disabled.
432+
> * For continued support and optimal performance, we advise migrating to Apache Spark 3.4.
433+
433434
3. If you selected **Attached compute**:
434435
1. Select an attached Synapse Spark pool from the **Select Azure Machine Learning attached compute** menu.
435436
4. Select **Next**.
@@ -601,7 +602,7 @@ jobs:
601602
602603
resources:
603604
instance_type: standard_e8s_v3
604-
runtime_version: "3.3"
605+
runtime_version: "3.4"
605606
```
606607
> [!NOTE]
607608
> To use an attached Synapse Spark pool, define the `compute` property in the sample YAML specification file shown above, instead of `resources` property.
@@ -684,7 +685,7 @@ def spark_pipeline(spark_input_data):
684685
spark_step.identity = ManagedIdentityConfiguration()
685686
spark_step.resources = {
686687
"instance_type": "Standard_E8S_V3",
687-
"runtime_version": "3.3.0",
688+
"runtime_version": "3.4.0",
688689
}
689690
690691
pipeline = spark_pipeline(

0 commit comments

Comments
 (0)