You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/spark/apache-spark-zeppelin-notebook.md
+26-3Lines changed: 26 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ description: Step-by-step instructions on how to use Zeppelin notebooks with Apa
4
4
ms.service: hdinsight
5
5
ms.topic: how-to
6
6
ms.custom: hdinsightactive,seoapr2020
7
-
ms.date: 05/26/2023
7
+
ms.date: 12/14/2023
8
8
---
9
9
10
10
# Use Apache Zeppelin notebooks with Apache Spark cluster on Azure HDInsight
@@ -81,7 +81,7 @@ HDInsight Spark clusters include [Apache Zeppelin](https://zeppelin.apache.org/)
81
81
82
82
The**%sql** statement at the beginning tells the notebook to use the LivyScala interpreter.
83
83
84
-
6. Select the **BarChart** icon to change the display. **settings**, appear after you have selected **BarChart**, allows you to choose **Keys**, and **Values**. The following screenshot shows the output.
84
+
6. Select the **BarChart** icon to change the display. **settings** appear after you have selected **BarChart**, allows you to choose **Keys**, and **Values**. The following screenshot shows the output.
85
85
86
86
:::image type="content" source="./media/apache-spark-zeppelin-notebook/hdinsight-zeppelin-spark-query-1.png " alt-text="Run a Spark SQL statement using the notebook1" border="true":::
87
87
@@ -145,6 +145,29 @@ The Zeppelin notebooks are saved to the cluster headnodes. So, if you delete the
145
145
146
146
This action saves the notebook asaJSON file in your download location.
147
147
148
+
> [!NOTE]
149
+
>*InHDI4.0, the zeppelin notebook directory path is, `/usr/hdp/<version>/zeppelin/notebook/<notebook_session_id>/`
150
+
>
151
+
>Eg. /usr/hdp/4.1.17.10/zeppelin/2JMC9BZ8X/
152
+
>
153
+
>WhereasinHDI5.0 and above this path is different `/usr/hdp/<version>/zeppelin/notebook/<Kernel_name>/`
154
+
>
155
+
>Eg. /usr/hdp/5.1.4.5/zeppelin/notebook/Scala/
156
+
>*The file name stored is different in HDI5.0.
157
+
>It is stored as `<notebook_name>_<sessionid>.zpln`
158
+
>
159
+
>Eg. testzeppelin_2JJK53XQA.zpln
160
+
>
161
+
>InHDI4.0, the file name is just note.json stored under session_id directory.
162
+
>
163
+
>Eg. /2JMC9BZ8X/note.json
164
+
>
165
+
>*HDIZeppelin always saves the notebook in the path `/usr/hdp/<version>/zeppelin/notebook/` in hn0 local disk.
166
+
>
167
+
>If you want the notebook to be available even after cluster deletion , you can try to use azure file storage (UsingSMB protocol ) and link it to local path. For more details, see [MountSMBAzure file share on Linux](/azure/storage/files/storage-how-to-use-files-linux)
168
+
>
169
+
>After mounting it, you can modify the zeppelin configuration zeppelin.notebook.dir to the mounted path in ambari UI.
170
+
148
171
##Use `Shiro` to ConfigureAccess to ZeppelinInterpreters in EnterpriseSecurityPackage (ESP) Clusters
149
172
150
173
As noted above, the `%sh` interpreter isn't supported from HDInsight4.0 onwards. Furthermore, since `%sh` interpreter introduces potential security issues, such asaccess keytabs using shell commands, it has been removed from HDInsight3.6ESP clusters aswell. It means `%sh` interpreter isn't available when clicking **Createnew note** or in the InterpreterUI by default.
@@ -216,7 +239,7 @@ To validate the service from a command line, SSH to the head node. Switch user t
0 commit comments