You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/synapse-analytics/spark/apache-spark-development-using-notebooks.md
+19-21Lines changed: 19 additions & 21 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -285,13 +285,13 @@ A step-by-step status of a cell run appears beneath the cell to help you see its
285
285
286
286
A Synapse notebook is purely Spark based. Code cells run on the serverless Apache Spark pool remotely. A Spark job progress indicator with a real-time progress bar appears to help you understand the job run status.
287
287
288
-
The number of tasks for each job or stage help you identify the parallel level of your Spark job. You can also drill deeper to the Spark UI of a specific job (or stage) by selecting the link on the job (or stage) name.
288
+
The number of tasks for each job or stage helps you identify the parallel level of your Spark job. You can also drill deeper to the Spark UI of a specific job (or stage) by selecting the link on the job (or stage) name.
289
289
290
290
291
291
292
292
### Configure a Spark session
293
293
294
-
You can specify the timeout duration, the number, and the size of executors to give to the current Spark session on the **Configure session** pane. Restart the Spark session for configuration changes to take effect. All cached notebook variables are cleared.
294
+
On the **Configure session** pane, you can specify the timeout duration, the number, and the size of executors to give to the current Spark session. Restart the Spark session for configuration changes to take effect. All cached notebook variables are cleared.
295
295
296
296
You can also create a configuration from the Apache Spark configuration or select an existing configuration. For details, refer to [Manage Apache Spark configuration](../../synapse-analytics/spark/apache-spark-azure-create-spark-configuration.md).
297
297
@@ -369,7 +369,7 @@ During the pipeline run mode, you can use the **Settings** tab to configure sett
369
369
370
370
371
371
372
-
If you want to change the session configuration, the name of the pipeline notebook activity parameter should be same as `activityParameterName` in the notebook. In this example, during a pipeline run, `driverCores` in `%%configure` will be replaced by `8` and`livy.rsc.sql.num-rows` will be replaced by `4000`.
372
+
If you want to change the session configuration, the name of the pipeline notebook activity parameter should be the same as `activityParameterName` in the notebook. In this example, during a pipeline run, `8` replaces `driverCores` in `%%configure`, and `4000` replaces`livy.rsc.sql.num-rows`.
373
373
374
374
If a pipeline run fails after you use the `%%configure` magic command, you can get more error information by running the `%%configure` magic cell in the interactive mode of the notebook.
@@ -531,7 +531,7 @@ Available magic commands for cells:
531
531
532
532
## <aname = "reference-unpublished-notebook"></a>Reference an unpublished notebook
533
533
534
-
Referencing an unpublished notebook is helpful when you want to debug locally. When you enable this feature, a notebook run fetches the current content in web cache. If you run a cell that includes a reference notebook statement, you reference the presenting notebooks in the current notebook browser instead of a saved versions in cluster. Other notebooks can reference the changes in your notebook editor without your having to publish (Live mode) or commit (Git mode) the changes. By using this approach, you can prevent the pollution of common libraries during the developing or debugging process.
534
+
Referencing an unpublished notebook is helpful when you want to debug locally. When you enable this feature, a notebook run fetches the current content in web cache. If you run a cell that includes a reference notebook statement, you reference the presenting notebooks in the current notebook browser instead of a saved version in a cluster. Other notebooks can reference the changes in your notebook editor without your having to publish (Live mode) or commit (Git mode) the changes. By using this approach, you can prevent the pollution of common libraries during the developing or debugging process.
535
535
536
536
You can enable referencing an unpublished notebook by selecting the appropriate checkbox on the **Properties** pane.
Synapse notebooks support the magic command `%history` to print the input command history that ran in the current session. The `%history` magic command is similar to the standard Jupyter IPython command and works for multiple language contexts in notebook.
610
+
Synapse notebooks support the magic command `%history` to print the input command history for the current session. The `%history` magic command is similar to the standard Jupyter IPython command and works for multiple language contexts in a notebook.
611
611
612
612
`%history [-n] [range [range ...]]`
613
613
@@ -616,9 +616,7 @@ In the preceding code, `-n` is the print execution number. The `range` value can
616
616
*`N`: Print code of the `Nth` executed cell.
617
617
*`M-N`: Print code from the `Mth` to `Nth` executed cell.
618
618
619
-
Example:
620
-
621
-
* Print input history from the first to the second executed cell: `%history -n 1-2`
619
+
For example, to print input history from the first to the second executed cell, use `%history -n 1-2`.
622
620
623
621
## Integrate a notebook
624
622
@@ -634,15 +632,15 @@ To parameterize your notebook, select the ellipsis (**...**) to access more comm
634
632
635
633
636
634
637
-
Azure Data Factory looks for the parameter cell and treats this cell as defaults for the parameters passed in at execution time. The execution engine adds a new cell beneath the parameter cell with input parameters to overwrite the default values.
635
+
Azure Data Factory looks for the parameter cell and treats this cell as the default for the parameters passed in at execution time. The execution engine adds a new cell beneath the parameter cell with input parameters to overwrite the default values.
638
636
639
637
### Assign parameters values from a pipeline
640
638
641
-
After you create a notebook with parameters, you can run it from a pipeline by using a Synapse notebook activity. After you add the activity to your pipeline canvas, you can set the parameter values in the **Base parameters** section on the **Settings** tab.
639
+
After you create a notebook with parameters, you can run it from a pipeline by using a Synapse notebook activity. After you add the activity to your pipeline canvas, you can set the parameter values in the **Base parameters** section of the **Settings** tab.
642
640
643
641
644
642
645
-
When assigning parameter values, you can use the [pipeline expression language](../../data-factory/control-flow-expression-language-functions.md) or [system variables](../../data-factory/control-flow-system-variables.md).
643
+
When you're assigning parameter values, you can use the [pipeline expression language](../../data-factory/control-flow-expression-language-functions.md) or [system variables](../../data-factory/control-flow-system-variables.md).
646
644
647
645
## Use shortcut keys
648
646
@@ -658,7 +656,7 @@ Similar to Jupyter Notebooks, Synapse notebooks have a modal user interface. The
658
656
659
657
### Shortcut keys in command mode
660
658
661
-
| Action |Synapse notebook shortcuts |
659
+
| Action |Synapse notebook shortcut|
662
660
|--|--|
663
661
|Run the current cell and select below | Shift+Enter |
664
662
|Run the current cell and insert below | Alt+Enter |
@@ -674,7 +672,7 @@ Similar to Jupyter Notebooks, Synapse notebooks have a modal user interface. The
1. Choose a name for your SQL script by selecting the **Properties** button and replacing the default name assigned to the SQL script.
32
+
33
+
31
34
32
-
2. Choose the specific dedicated SQL pool or serverless SQL pool from the **Connect to** drop-down menu. Or if necessary, choose the database from **Use database**.
2. On the **Connect to** dropdown menu, select the specific dedicated SQL pool or serverless SQL pool. Or if necessary, choose the database from **Use database**.
34
36
35
-
3. Start authoring your SQL script using the intellisense feature.
37
+
38
+
39
+
3. Start authoring your SQL script by using the IntelliSense feature.
36
40
37
41
## Run your SQL script
38
42
39
-
Select the **Run** button to execute your SQL script. The results are displayed by default in a table.
43
+
To run your SQL script, select the **Run** button. The results appear in a table by default.
42
46
43
-
Synapse Studio creates a new session for each SQL script execution. Once a SQL script execution completes, the session is automatically closed.
47
+
Synapse Studio creates a new session for each SQL script execution. After a SQL script execution finishes, the session is automatically closed.
44
48
45
-
Temporary tables are only visible to the session in which they were created and are automatically dropped when the session closes.
49
+
Temporary tables are visible only in the session where you created them. They're automatically dropped when the session closes.
46
50
47
51
## Export your results
48
52
49
-
You can export the results to your local storage in different formats (including CSV, Excel, JSON, XML) by selecting "Export results" and choosing the extension.
53
+
You can export the results to your local storage in various formats (including CSV, Excel, JSON, and XML) by selecting **Export results** and choosing the extension.
50
54
51
-
You can also visualize the SQL script results in a chart by selecting the **Chart** button. Select the "Chart type" and **Category column**. You can export the chart into a picture by selecting **Save as image**.
55
+
You can also visualize the SQL script results in a chart by selecting the **Chart** button. Then select the **Chart type** and **Category column** values. You can export the chart into a picture by selecting **Save as image**.
54
58
55
59
## Explore data from a Parquet file
56
60
57
-
You can explore Parquet files in a storage account using SQL script to preview the file contents.
61
+
You can explore Parquet files in a storage account by using a SQL script to preview the file contents.
62
+
63
+
By using a shortcut menu on the **Data** pane, you can select actions for resources like SQL tables, external tables, and views. Explore the available commands by right-clicking the nodes of SQL databases. The commands for **New SQL script** include:
62
68
63
-
By selecting the **Actions** menu under data, you can select several actions such as:
69
+
-**Select TOP 100 rows**
70
+
-**CREATE**
71
+
-**DROP**
72
+
-**DROP and CREATE**
64
73
65
-
- New SQL script
66
-
- Select TOP 1000 rows
67
-
- CREATE
68
-
- DROP and CREATE
69
-
70
-
Explore the available gesture by right-clicking the nodes of SQL databases.
73
75
74
76
## Create folders and move SQL scripts into a folder
75
77
76
-
From the Actions menu under Develop SQL scripts Choose "New folder" from the "Actions" menu under Develop SQL scripts. And type in the name of the new folder in the pop-up window.
78
+
To create a folder:
79
+
80
+
1. On the **Develop** pane, right-click in the **SQL scripts** area and then select **New folder**.
81
+
82
+
> [!div class="mx-imgBorder"]
83
+
> 
84
+
85
+
1. On the **New folder** pane, enter the name of the new folder.
86
+
87
+
To move a SQL script into a folder:
77
88
78
-
> [!div class="mx-imgBorder"]
79
-
> 
89
+
1. Right-click the SQL script, and then select **Move to**.
80
90
81
-
To move a SQL script into a folder, you can select the sql script and choose "Move To" from the Actions menu. Then find the destination folder in the new window and move the sql script into selected folder.You can also quickly drag the sql script and drop it into a folder.
91
+
1. On the **Move to** pane, choose a destination folder, and then select **Move here**. You can also quickly drag the SQL script and drop it into a folder.
0 commit comments