Skip to content

Commit 78810ba

Browse files
authored
Merge pull request #195215 from v-lanjli/updatepiplineactivity
update synapse notebook activity
2 parents c122817 + 2ad154f commit 78810ba

File tree

4 files changed

+16
-2
lines changed

4 files changed

+16
-2
lines changed
Loading
-15.3 KB
Loading

articles/synapse-analytics/quickstart-transform-data-using-spark-job-definition.md

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -78,8 +78,20 @@ On this panel, you can reference to the Spark job definition to run.
7878

7979
* Expand the Spark job definition list, you can choose an existing Apache Spark job definition. You can also create a new Apache Spark job definition by selecting the **New** button to reference the Spark job definition to be run.
8080

81-
* You can add command-line arguments by clicking the **New** button. It should be noted that adding command-line arguments will override the command-line arguments defined by the Spark job definition. <br> *Sample: `abfss://…/path/to/shakespeare.txt` `abfss://…/path/to/result`* <br>
82-
81+
* (Optional) You can fill in information for Apache Spark job definition. If the following settings are empty, the settings of the spark job definition itself will be used to run; if the following settings are not empty, these settings will replace the settings of the spark job definition itself.
82+
83+
| Property | Description |
84+
| ----- | ----- |
85+
|Main definition file| The main file used for the job. Select a PY/JAR/ZIP file from your storage. You can select **Upload file** to upload the file to a storage account. <br> Sample: `abfss://…/path/to/wordcount.jar`|
86+
|Main class name| The fully qualified identifier or the main class that is in the main definition file. <br> Sample: `WordCount`|
87+
|Command-line arguments| You can add command-line arguments by clicking the **New** button. It should be noted that adding command-line arguments will override the command-line arguments defined by the Spark job definition. <br> *Sample: `abfss://…/path/to/shakespeare.txt` `abfss://…/path/to/result`* <br> |
88+
|Apache Spark pool| You can select Apache Spark pool from the list.|
89+
|Dynamically allocate executors| This setting maps to the dynamic allocation property in Spark configuration for Spark Application executors allocation.|
90+
|Min executors| Min number of executors to be allocated in the specified Spark pool for the job.|
91+
|Max executors| Max number of executors to be allocated in the specified Spark pool for the job.|
92+
|Driver size| Number of cores and memory to be used for driver given in the specified Apache Spark pool for the job.|
93+
94+
8395
![spark job definition pipline settings](media/quickstart-transform-data-using-spark-job-definition/spark-job-definition-pipline-settings.png)
8496

8597
* You can add dynamic content by clicking the **Add Dynamic Content** button or by pressing the shortcut key <kbd>Alt</kbd>+<kbd>Shift</kbd>+<kbd>D</kbd>. In the **Add Dynamic Content** page, you can use any combination of expressions, functions, and system variables to add to dynamic content.

articles/synapse-analytics/synapse-notebook-activity.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,8 @@ You can create a Synapse notebook activity directly from the Synapse pipeline ca
2727

2828
Drag and drop **Synapse notebook** under **Activities** onto the Synapse pipeline canvas. Select on the Synapse notebook activity box and config the notebook content for current activity in the **settings**. You can select an existing notebook from the current workspace or add a new one.
2929

30+
You can also select an Apache Spark pool in the settings. It should be noted that the Apache spark pool set here will replace the Apache spark pool used in the notebook. If Apache spark pool is not selected in the settings of notebook content for current activity, the Apache spark pool selected in that notebook will be used to run.
31+
3032
![screenshot-showing-create-notebook-activity](./media/synapse-notebook-activity/create-synapse-notebook-activity.png)
3133

3234
> [!NOTE]

0 commit comments

Comments
 (0)