You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/spark/apache-spark-jupyter-notebook-kernels.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,14 +6,14 @@ author: hrasheed-msft
6
6
ms.author: hrasheed
7
7
ms.reviewer: jasonh
8
8
ms.service: hdinsight
9
-
ms.custom: hdinsightactive,hdiseo17may2017
10
9
ms.topic: conceptual
11
-
ms.date: 05/27/2019
10
+
ms.custom: hdinsightactive,hdiseo17may2017
11
+
ms.date: 03/20/2020
12
12
---
13
13
14
14
# Kernels for Jupyter notebook on Apache Spark clusters in Azure HDInsight
15
15
16
-
HDInsight Spark clusters provide kernels that you can use with the Jupyter notebook on [Apache Spark](https://spark.apache.org/) for testing your applications. A kernel is a program that runs and interprets your code. The three kernels are:
16
+
HDInsight Spark clusters provide kernels that you can use with the Jupyter notebook on [Apache Spark](./apache-spark-overview.md) for testing your applications. A kernel is a program that runs and interprets your code. The three kernels are:
17
17
18
18
-**PySpark** - for applications written in Python2.
19
19
-**PySpark3** - for applications written in Python3.
@@ -53,7 +53,7 @@ Here are a few benefits of using the new kernels with Jupyter notebook on Spark
53
53
-**sc** - for Spark context
54
54
-**sqlContext** - for Hive context
55
55
56
-
So, you don't have to run statements like the following to set the contexts:
56
+
So, you **don't** have to run statements like the following to set the contexts:
57
57
58
58
sc = SparkContext('yarn-client')
59
59
sqlContext = HiveContext(sc)
@@ -119,7 +119,7 @@ The way notebooks are saved to the storage account is compatible with [Apache Ha
119
119
120
120
hdfs dfs -ls /HdiNotebooks # List everything at the root directory – everything in this directory is visible to Jupyter from the home page
121
121
hdfs dfs –copyToLocal /HdiNotebooks # Download the contents of the HdiNotebooks folder
122
-
hdfs dfs –copyFromLocal example.ipynb /HdiNotebooks # Upload a notebook example.ipynb to the root folder so it’s visible from Jupyter
122
+
hdfs dfs –copyFromLocal example.ipynb /HdiNotebooks # Upload a notebook example.ipynb to the root folder so it's visible from Jupyter
123
123
124
124
Irrespective of whether the cluster uses Azure Storage or Azure Data Lake Storage as the default storage account, the notebooks are also saved on the cluster headnode at `/var/lib/jupyter`.
125
125
@@ -131,7 +131,7 @@ Jupyter notebooks on Spark HDInsight clusters are supported only on Google Chrom
131
131
132
132
The new kernels are in evolving stage and will mature over time. This could also mean that APIs could change as these kernels mature. We would appreciate any feedback that you have while using these new kernels. This is useful in shaping the final release of these kernels. You can leave your comments/feedback under the **Feedback** section at the bottom of this article.
133
133
134
-
## <aname="seealso"></a>See also
134
+
## See also
135
135
136
136
-[Overview: Apache Spark on Azure HDInsight](apache-spark-overview.md)
0 commit comments