Skip to content

Commit 06ec0c6

Browse files
authored
Update apache-kafka-spark-structured-streaming-cosmosdb.md
Updated for HDInsight 4.0
1 parent 849acd2 commit 06ec0c6

File tree

1 file changed

+5
-5
lines changed

1 file changed

+5
-5
lines changed

articles/hdinsight/apache-kafka-spark-structured-streaming-cosmosdb.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ Learn how to use [Apache Spark](https://spark.apache.org/) [Structured Streaming
1616
Spark structured streaming is a stream processing engine built on Spark SQL. It allows you to express streaming computations the same as batch computation on static data. For more information on Structured Streaming, see the [Structured Streaming Programming Guide](https://spark.apache.org/docs/2.2.0/structured-streaming-programming-guide.html) at Apache.org.
1717

1818
> [!IMPORTANT]
19-
> This example used Spark 2.2 on HDInsight 3.6.
19+
> This example uses Spark 2.4 on HDInsight 4.0.
2020
>
2121
> The steps in this document create an Azure resource group that contains both a Spark on HDInsight and a Kafka on HDInsight cluster. These clusters are both located within an Azure Virtual Network, which allows the Spark cluster to directly communicate with the Kafka cluster.
2222
>
@@ -43,16 +43,16 @@ While you can create an Azure virtual network, Kafka, and Spark clusters manuall
4343

4444
This template creates the following resources:
4545

46-
* A Kafka on HDInsight 3.6 cluster.
46+
* A Kafka on HDInsight 4.0 cluster.
4747

48-
* A Spark on HDInsight 3.6 cluster.
48+
* A Spark on HDInsight 4.0 cluster.
4949

5050
* An Azure Virtual Network, which contains the HDInsight clusters. The virtual network created by the template uses the 10.0.0.0/16 address space.
5151

5252
* An Azure Cosmos DB SQL API database.
5353

5454
> [!IMPORTANT]
55-
> The structured streaming notebook used in this example requires Spark on HDInsight 3.6. If you use an earlier version of Spark on HDInsight, you receive errors when using the notebook.
55+
> The structured streaming notebook used in this example requires Spark on HDInsight 4.0. If you use an earlier version of Spark on HDInsight, you receive errors when using the notebook.
5656
5757
1. Use the following information to populate the entries on the **Custom deployment** section:
5858

@@ -62,7 +62,7 @@ While you can create an Azure virtual network, Kafka, and Spark clusters manuall
6262
|Resource group|Create a group or select an existing one. This group contains the HDInsight cluster.|
6363
|Cosmos DB Account Name|This value is used as the name for the Cosmos DB account. The name can only contain lowercase letters, numbers, and the hyphen (-) character. It must be between 3-31 characters in length.|
6464
|Base Cluster Name|This value is used as the base name for the Spark and Kafka clusters. For example, entering **myhdi** creates a Spark cluster named __spark-myhdi__ and a Kafka cluster named **kafka-myhdi**.|
65-
|Cluster Version|The HDInsight cluster version. This example is tested with HDInsight 3.6, and may not work with other cluster types.|
65+
|Cluster Version|The HDInsight cluster version. This example is tested with HDInsight 4.0, and may not work with other cluster types.|
6666
|Cluster Login User Name|The admin user name for the Spark and Kafka clusters.|
6767
|Cluster Login Password|The admin user password for the Spark and Kafka clusters.|
6868
|Ssh User Name|The SSH user to create for the Spark and Kafka clusters.|

0 commit comments

Comments
 (0)