You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/synapse-analytics/spark/apache-spark-development-using-notebooks.md
+16-11Lines changed: 16 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -52,6 +52,11 @@ We provide rich operations to develop notebooks:
52
52
+[Collapse a cell output](#collapse-a-cell-output)
53
53
+[Notebook outline](#notebook-outline)
54
54
55
+
> [!NOTE]
56
+
>
57
+
> In the notebooks, there is a SparkSession automatically created for you, stored in a variable called `spark`. Also there is a variable for SparkContext which is called `sc`. Users can access these variables directly and should not change the values of these variables.
58
+
59
+
55
60
<h3id="add-a-cell">Add a cell</h3>
56
61
57
62
There are multiple ways to add a new cell to your notebook.
@@ -161,7 +166,7 @@ Select the **Undo** / **Redo** button or press **Z** / **Shift+Z** to revoke the
161
166

162
167
163
168
Supported undo cell operations:
164
-
+ Insert/Delete cell: You could revoke the delete operations by selecting **Undo**, the text content will be kept along with the cell.
169
+
+ Insert/Delete cell: You could revoke the delete operations by selecting **Undo**, the text content is kept along with the cell.
165
170
+ Reorder cell.
166
171
+ Toggle parameter.
167
172
+ Convert between Code cell and Markdown cell.
@@ -271,7 +276,7 @@ Select the **Cancel All** button to cancel the running cells or cells waiting in
271
276
272
277
### Notebook reference
273
278
274
-
You can use ```%run <notebook path>``` magic command to reference another notebook within current notebook's context. All the variables defined in the reference notebook are available in the current notebook. ```%run``` magic command supports nested calls but not support recursive calls. You will receive an exception if the statement depth is larger than **five**.
279
+
You can use ```%run <notebook path>``` magic command to reference another notebook within current notebook's context. All the variables defined in the reference notebook are available in the current notebook. ```%run``` magic command supports nested calls but not support recursive calls. You receive an exception if the statement depth is larger than **five**.
@@ -289,7 +294,7 @@ Notebook reference works in both interactive mode and Synapse pipeline.
289
294
290
295
### Variable explorer
291
296
292
-
Synapse notebook provides a built-in variables explorer for you to see the list of the variables name, type, length, and value in the current Spark session for PySpark (Python) cells. More variables will show up automatically as they are defined in the code cells. Clicking on each column header will sort the variables in the table.
297
+
Synapse notebook provides a built-in variables explorer for you to see the list of the variables name, type, length, and value in the current Spark session for PySpark (Python) cells. More variables show up automatically as they are defined in the code cells. Clicking on each column header sorts the variables in the table.
293
298
294
299
You can select the **Variables** button on the notebook command bar to open or hide the variable explorer.
295
300
@@ -382,7 +387,7 @@ Parameterized session configuration allows you to replace the value in %%configu
382
387
}
383
388
```
384
389
385
-
Notebook will use default value if run a notebook in interactive mode directly or no parameter that match "activityParameterName" is given from Pipeline Notebook activity.
390
+
Notebook uses default value if run a notebook in interactive mode directly or no parameter that match "activityParameterName" is given from Pipeline Notebook activity.
386
391
387
392
During the pipeline run mode, you can configure pipeline Notebook activity settings as below:
388
393

@@ -450,7 +455,7 @@ You can access data in the primary storage account directly. There's no need to
450
455
451
456
## IPython Widgets
452
457
453
-
Widgets are eventful Python objects that have a representation in the browser, often as a control like a slider, textbox etc. IPython Widgets only works in Python environment, it's not supported in other languages (e.g. Scala, SQL, C#) yet.
458
+
Widgets are eventful Python objects that have a representation in the browser, often as a control like a slider, textbox etc. IPython Widgets only works in Python environment, it's not supported in other languages (for example, Scala, SQL, C#) yet.
454
459
455
460
### To use IPython Widget
456
461
1. You need to import `ipywidgets` module first to use the Jupyter Widget framework.
@@ -468,11 +473,11 @@ Widgets are eventful Python objects that have a representation in the browser, o
468
473
slider
469
474
```
470
475
471
-
3. Run the cell, the widget will display at the output area.
476
+
3. Run the cell, the widget displays at the output area.
472
477
473
478

474
479
475
-
4. You can use multiple `display()` calls to render the same widget instance multiple times, but they will remain in sync with each other.
480
+
4. You can use multiple `display()` calls to render the same widget instance multiple times, but they remain in sync with each other.
476
481
477
482
```python
478
483
slider = widgets.IntSlider()
@@ -514,7 +519,7 @@ Widgets are eventful Python objects that have a representation in the browser, o
514
519
|`widgets.jslink()`|You can use `widgets.link()` function to link two similar widgets.|
515
520
|`FileUpload` widget| Not support yet.|
516
521
517
-
2. Global `display` function provided by Synapse does not support displaying multiple widgets in 1 call (i.e.`display(a, b)`), which is different from IPython `display` function.
522
+
2. Global `display` function provided by Synapse does not support displaying multiple widgets in one call (that is,`display(a, b)`), which is different from IPython `display` function.
518
523
3. If you close a notebook that contains IPython Widget, you will not be able to see or interact with it until you execute the corresponding cell again.
Reference unpublished notebook is helpful when you want to debug "locally", when enabling this feature, notebook run will fetch the current content in web cache, if you run a cell including a reference notebooks statement, you will reference the presenting notebooks in the current notebook browser instead of a saved versions in cluster, that means the changes in your notebook editor can be referenced immediately by other notebooks without having to be published(Live mode) or committed(Git mode), by leveraging this approach you can easily avoid common libraries getting polluted during developing or debugging process.
576
+
Reference unpublished notebook is helpful when you want to debug "locally", when enabling this feature, notebook run fetches the current content in web cache, if you run a cell including a reference notebooks statement, you reference the presenting notebooks in the current notebook browser instead of a saved versions in cluster, that means the changes in your notebook editor can be referenced immediately by other notebooks without having to be published(Live mode) or committed(Git mode), by leveraging this approach you can easily avoid common libraries getting polluted during developing or debugging process.
572
577
573
578
You can enable Reference unpublished notebook from Properties panel:
574
579
@@ -607,7 +612,7 @@ You can reuse your notebook sessions conveniently now without having to start ne
607
612
608
613

609
614
610
-
In the **Active sessions** list you can see the session information and the corresponding notebook that is currently attached to the session. You can operate Detach with notebook, Stop the session, and View in monitoring from here. Moreover, you can easily connect your selected notebook to an active session in the list started from another notebook, the session will be detached from the previous notebook (if it's not idle) then attach to the current one.
615
+
In the **Active sessions**, list you can see the session information and the corresponding notebook that is currently attached to the session. You can operate Detach with notebook, Stop the session, and View in monitoring from here. Moreover, you can easily connect your selected notebook to an active session in the list started from another notebook, the session is detached from the previous notebook (if it's not idle) then attach to the current one.
611
616
612
617

613
618
@@ -664,7 +669,7 @@ To parameterize your notebook, select the ellipses (...) to access the **more co
664
669
665
670
---
666
671
667
-
Azure Data Factory looks for the parameters cell and treats this cell as defaults for the parameters passed in at execution time. The execution engine will add a new cell beneath the parameters cell with input parameters in order to overwrite the default values.
672
+
Azure Data Factory looks for the parameters cell and treats this cell as defaults for the parameters passed in at execution time. The execution engine adds a new cell beneath the parameters cell with input parameters in order to overwrite the default values.
0 commit comments