You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-apache-spark-with-kafka.md
+7-6Lines changed: 7 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,6 +11,7 @@ ms.topic: conceptual
11
11
ms.date: 11/06/2018
12
12
ms.author: hrasheed
13
13
---
14
+
14
15
# Apache Spark streaming (DStream) example with Apache Kafka on HDInsight
15
16
16
17
Learn how to use [Apache Spark](https://spark.apache.org/) to stream data into or out of [Apache Kafka](https://kafka.apache.org/) on HDInsight using [DStreams](https://spark.apache.org/docs/latest/api/java/org/apache/spark/streaming/dstream/DStream.html). This example uses a [Jupyter Notebook](https://jupyter.org/) that runs on the Spark cluster.
@@ -35,9 +36,9 @@ Apache Kafka on HDInsight does not provide access to the Kafka brokers over the
35
36
While you can create an Azure virtual network, Kafka, and Spark clusters manually, it's easier to use an Azure Resource Manager template. Use the following steps to deploy an Azure virtual network, Kafka, and Spark clusters to your Azure subscription.
36
37
37
38
1. Use the following button to sign in to Azure and open the template in the Azure portal.
38
-
39
-
<ahref="https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fhditutorialdata.blob.core.windows.net%2Farmtemplates%2Fcreate-linux-based-kafka-spark-cluster-in-vnet-v4.1.json"target="_blank"><imgsrc="./media/hdinsight-apache-spark-with-kafka/hdi-deploy-to-azure1.png"alt="Deploy to Azure"></a>
40
-
39
+
40
+
<ahref="https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fhditutorialdata.blob.core.windows.net%2Farmtemplates%2Fcreate-linux-based-kafka-spark-cluster-in-vnet-v4.1.json"target="_blank"><imgsrc="./media/hdinsight-apache-spark-with-kafka/hdi-deploy-to-azure1.png"alt="Deploy to Azure button for new cluster"></a>
41
+
41
42
The Azure Resource Manager template is located at **https://hditutorialdata.blob.core.windows.net/armtemplates/create-linux-based-kafka-spark-cluster-in-vnet-v4.1.json**.
42
43
43
44
> [!WARNING]
@@ -46,9 +47,9 @@ While you can create an Azure virtual network, Kafka, and Spark clusters manuall
46
47
This template creates an HDInsight 3.6 cluster for both Kafka and Spark.
47
48
48
49
2. Use the following information to populate the entries on the **Custom deployment** section:
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-apache-storm-with-kafka.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -384,11 +384,11 @@ The following diagram shows how communication flows between Storm and Kafka:
384
384
To create an Azure Virtual Network, and then create the Kafka and Storm clusters within it, use the following steps:
385
385
386
386
1. Use the following button to sign in to Azure and open the template in the Azure portal.
387
-
388
-
<a href="https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure-Samples%2Fhdinsight-storm-java-kafka%2Fmaster%2Fcreate-kafka-storm-clusters-in-vnet.json" target="_blank"><img src="./media/hdinsight-apache-storm-with-kafka/hdi-deploy-to-azure1.png" alt="Deploy to Azure"></a>
389
-
387
+
388
+
<a href="https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure-Samples%2Fhdinsight-storm-java-kafka%2Fmaster%2Fcreate-kafka-storm-clusters-in-vnet.json" target="_blank"><img src="./media/hdinsight-apache-storm-with-kafka/hdi-deploy-to-azure1.png" alt="Deploy to Azure button for new cluster"></a>
389
+
390
390
The Azure Resource Manager template is located at **https://github.com/Azure-Samples/hdinsight-storm-java-kafka/blob/master/create-kafka-storm-clusters-in-vnet.json**. It creates the following resources:
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-apps-install-custom-applications.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,8 +9,8 @@ ms.custom: hdinsightactive
9
9
ms.topic: conceptual
10
10
ms.date: 05/14/2018
11
11
ms.author: hrasheed
12
-
13
12
---
13
+
14
14
# Install custom Apache Hadoop applications on Azure HDInsight
15
15
16
16
In this article, you will learn how to install an [Apache Hadoop](https://hadoop.apache.org/) application on Azure HDInsight, which has not been published to the Azure portal. The application you will install in this article is [Hue](https://gethue.com/).
@@ -41,7 +41,7 @@ The files needed for deploying this application (Hue):
41
41
42
42
1. Click the following image to sign in to Azure and open the Resource Manager template in the Azure portal.
43
43
44
-
<ahref="https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fhdinsight%2FIaas-Applications%2Fmaster%2FHue%2Fazuredeploy.json"target="_blank"><imgsrc="./media/hdinsight-apps-install-custom-applications/hdi-deploy-to-azure1.png"alt="Deploy to Azure"></a>
44
+
<ahref="https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fhdinsight%2FIaas-Applications%2Fmaster%2FHue%2Fazuredeploy.json"target="_blank"><imgsrc="./media/hdinsight-apps-install-custom-applications/hdi-deploy-to-azure1.png"alt="Deploy to Azure button for new cluster"></a>
45
45
46
46
This button opens a Resource Manager template on the Azure portal. The Resource Manager template is located at [https://github.com/hdinsight/Iaas-Applications/tree/master/Hue](https://github.com/hdinsight/Iaas-Applications/tree/master/Hue). To learn how to write this Resource Manager template, see [MSDN: Install an HDInsight application](https://msdn.microsoft.com/library/mt706515.aspx).
47
47
2. From the **Parameters** blade, enter the following:
@@ -56,7 +56,7 @@ The files needed for deploying this application (Hue):
56
56
57
57
1. Click the following image to sign in to Azure and open the Resource Manager template in the Azure portal.
58
58
59
-
<ahref="https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fhditutorialdata.blob.core.windows.net%2Fhdinsightapps%2Fcreate-linux-based-hadoop-cluster-in-hdinsight.json"target="_blank"><imgsrc="./media/hdinsight-apps-install-custom-applications/hdi-deploy-to-azure1.png"alt="Deploy to Azure"></a>
59
+
<ahref="https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fhditutorialdata.blob.core.windows.net%2Fhdinsightapps%2Fcreate-linux-based-hadoop-cluster-in-hdinsight.json"target="_blank"><imgsrc="./media/hdinsight-apps-install-custom-applications/hdi-deploy-to-azure1.png"alt="Deploy to Azure button for new cluster"></a>
60
60
61
61
This button opens a Resource Manager template on the Azure portal. The Resource Manager template is located at [https://hditutorialdata.blob.core.windows.net/hdinsightapps/create-linux-based-hadoop-cluster-in-hdinsight.json](https://hditutorialdata.blob.core.windows.net/hdinsightapps/create-linux-based-hadoop-cluster-in-hdinsight.json). To learn how to write this Resource Manager template, see [MSDN: Install an HDInsight application](https://msdn.microsoft.com/library/mt706515.aspx).
62
62
2. Follow the instruction to create cluster and install Hue. For more information on creating HDInsight clusters, see [Create Linux-based Hadoop clusters in HDInsight](hdinsight-hadoop-provision-linux-clusters.md).
0 commit comments