Skip to content

Commit 7a44f18

Browse files
authored
Merge pull request #104738 from dagiro/ts_zeppelin2
ts_zeppelin2
2 parents 03736e9 + 9f764aa commit 7a44f18

File tree

1 file changed

+41
-7
lines changed

1 file changed

+41
-7
lines changed

articles/hdinsight/spark/apache-spark-zeppelin-notebook.md

Lines changed: 41 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,9 @@ author: hrasheed-msft
55
ms.author: hrasheed
66
ms.reviewer: jasonh
77
ms.service: hdinsight
8-
ms.custom: hdinsightactive
98
ms.topic: conceptual
10-
ms.date: 11/05/2019
9+
ms.custom: hdinsightactive
10+
ms.date: 02/18/2020
1111
---
1212

1313
# Use Apache Zeppelin notebooks with Apache Spark cluster on Azure HDInsight
@@ -16,9 +16,8 @@ HDInsight Spark clusters include [Apache Zeppelin](https://zeppelin.apache.org/)
1616

1717
## Prerequisites
1818

19-
* An Azure subscription. See [Get Azure free trial](https://azure.microsoft.com/documentation/videos/get-azure-free-trial-for-testing-hadoop-in-hdinsight/).
2019
* An Apache Spark cluster on HDInsight. For instructions, see [Create Apache Spark clusters in Azure HDInsight](apache-spark-jupyter-spark-sql.md).
21-
* The URI scheme for your clusters primary storage. This would be `wasb://` for Azure Blob Storage, `abfs://` for Azure Data Lake Storage Gen2 or `adl://` for Azure Data Lake Storage Gen1. If secure transfer is enabled for Blob Storage, the URI would be `wasbs://`. See also, [Require secure transfer in Azure Storage](../../storage/common/storage-require-secure-transfer.md) for more information.
20+
* The URI scheme for your clusters primary storage. This would be `wasb://` for Azure Blob Storage, `abfs://` for Azure Data Lake Storage Gen2 or `adl://` for Azure Data Lake Storage Gen1. If secure transfer is enabled for Blob Storage, the URI would be `wasbs://`. For more information, see [Require secure transfer in Azure Storage](../../storage/common/storage-require-secure-transfer.md) .
2221

2322
## Launch an Apache Zeppelin notebook
2423

@@ -149,7 +148,7 @@ This saves the notebook as a JSON file in your download location.
149148

150149
## Livy session management
151150

152-
When you run the first code paragraph in your Zeppelin notebook, a new Livy session is created in your HDInsight Spark cluster. This session is shared across all Zeppelin notebooks that you subsequently create. If for some reason the Livy session is killed (cluster reboot, etc.), you will not be able to run jobs from the Zeppelin notebook.
151+
When you run the first code paragraph in your Zeppelin notebook, a new Livy session is created in your HDInsight Spark cluster. This session is shared across all Zeppelin notebooks that you subsequently create. If for some reason the Livy session is killed (cluster reboot, and so on), you won't be able to run jobs from the Zeppelin notebook.
153152

154153
In such a case, you must perform the following steps before you can start running jobs from a Zeppelin notebook.
155154

@@ -163,9 +162,44 @@ In such a case, you must perform the following steps before you can start runnin
163162

164163
3. Run a code cell from an existing Zeppelin notebook. This creates a new Livy session in the HDInsight cluster.
165164

166-
## <a name="seealso"></a>See also
165+
## General information
166+
167+
### Validate service
168+
169+
To validate the service from Ambari, navigate to `https://CLUSTERNAME.azurehdinsight.net/#/main/services/ZEPPELIN/summary` where CLUSTERNAME is the name of your cluster.
170+
171+
To validate the service from a command line, SSH to the head node. Switch user to zeppelin using command `sudo su zeppelin`. Status commands:
172+
173+
|Command |Description |
174+
|---|---|
175+
|`/usr/hdp/current/zeppelin-server/bin/zeppelin-daemon.sh status`|Service status.|
176+
|`/usr/hdp/current/zeppelin-server/bin/zeppelin-daemon.sh --version`|Service version.|
177+
|`ps -aux | grep zeppelin`|Identify PID.|
178+
179+
### Log locations
180+
181+
|Service |Path |
182+
|---|---|
183+
|zeppelin-server|/usr/hdp/current/zeppelin-server/|
184+
|Server Logs|/var/log/zeppelin|
185+
|Configuration Interpreter, Shiro, site.xml, log4j|/usr/hdp/current/zeppelin-server/conf or /etc/zeppelin/conf|
186+
|PID directory|/var/run/zeppelin|
187+
188+
### Enable debug logging
189+
190+
1. Navigate to `https://CLUSTERNAME.azurehdinsight.net/#/main/services/ZEPPELIN/summary` where CLUSTERNAME is the name of your cluster.
191+
192+
1. Navigate to **CONFIGS** > **Advanced zeppelin-log4j-properties** > **log4j_properties_content**.
193+
194+
1. Modify `log4j.appender.dailyfile.Threshold = INFO` to `log4j.appender.dailyfile.Threshold = DEBUG`.
195+
196+
1. Add `log4j.logger.org.apache.zeppelin.realm=DEBUG`.
197+
198+
1. Save changes and restart service.
199+
200+
## Next steps
167201

168-
* [Overview: Apache Spark on Azure HDInsight](apache-spark-overview.md)
202+
[Overview: Apache Spark on Azure HDInsight](apache-spark-overview.md)
169203

170204
### Scenarios
171205

0 commit comments

Comments
 (0)