You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight-aks/flink/assign-kafka-topic-event-message-to-azure-data-lake-storage-gen2.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -103,7 +103,7 @@ Flink provides an Apache Kafka connector for reading data from and writing data
103
103
*abfsGen2.java*
104
104
105
105
> [!Note]
106
-
> Replace [Apache Kafka on HDInsight cluster](../../hdinsight/kafka/apache-kafka-get-started.md) bootStrapServers with your own brokers for Kafka 2.4 or 3.2
106
+
> Replace [Apache Kafka on HDInsight cluster](../../hdinsight/kafka/apache-kafka-get-started.md) bootStrapServers with your own brokers for Kafka 3.2
Copy file name to clipboardExpand all lines: articles/hdinsight-aks/flink/sink-kafka-to-kibana.md
+15-14Lines changed: 15 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,9 +1,9 @@
1
1
---
2
2
title: Use Elasticsearch along with Apache Flink® on HDInsight on AKS
3
-
description: Learn how to use Elasticsearch along Apache Flink® on HDInsight on AKS
3
+
description: Learn how to use Elasticsearch along Apache Flink® on HDInsight on AKS.
4
4
ms.service: hdinsight-aks
5
5
ms.topic: how-to
6
-
ms.date: 10/27/2023
6
+
ms.date: 04/04/2024
7
7
---
8
8
9
9
# Using Elasticsearch with Apache Flink® on HDInsight on AKS
@@ -18,7 +18,8 @@ In this article, learn how to Use Elastic along Apache Flink® on HDInsight on A
18
18
19
19
## Elasticsearch and Kibana
20
20
21
-
Elasticsearch is a distributed, free and open search and analytics engine for all types of data, including
21
+
Elasticsearch is a distributed, free, and open search and analytics engine for all types of data, including.
22
+
22
23
* Textual
23
24
* Numerical
24
25
* Geospatial
@@ -27,17 +28,17 @@ Elasticsearch is a distributed, free and open search and analytics engine for al
27
28
28
29
Kibana is a free and open frontend application that sits on top of the elastic stack, providing search and data visualization capabilities for data indexed in Elasticsearch.
* IntelliJ IDEA for development on an Azure VM which in the same Vnet
42
43
43
44
@@ -90,7 +91,7 @@ sudo apt install elasticsearch
90
91
91
92
For installing and configuring Kibana Dashboard, we don’t need to add any other repository because the packages are available through the already added ElasticSearch.
92
93
93
-
We use the following command to install Kibana
94
+
We use the following command to install Kibana.
94
95
95
96
```
96
97
sudo apt install kibana
@@ -111,9 +112,9 @@ sudo apt install kibana
111
112
```
112
113
### Access the Kibana Dashboard web interface
113
114
114
-
In order to make Kibana accessible from output, need to set network.host to 0.0.0.0
115
+
In order to make Kibana accessible from output, need to set network.host to 0.0.0.0.
115
116
116
-
configure /etc/kibana/kibana.yml on Ubuntu VM
117
+
Configure `/etc/kibana/kibana.yml` on Ubuntu VM
117
118
118
119
> [!NOTE]
119
120
> 10.0.1.4 is a local private IP, that we have used which can be accessed in maven project develop Windows VM. You're required to make modifications according to your network security requirements. We use the same IP later to demo for performing analytics on Kibana.
@@ -289,7 +290,7 @@ public class kafkaSinkToElastic {
289
290
290
291
**Package the jar and submit to Flink to run on WebSSH**
291
292
292
-
On [Secure Shell for Flink](./flink-web-ssh-on-portal-to-flink-sql.md), you can use the following commands
293
+
On [Secure Shell for Flink](./flink-web-ssh-on-portal-to-flink-sql.md), you can use the following commands.
293
294
294
295
```
295
296
msdata@pod-0 [ ~ ]$ ls -l FlinkElasticSearch-1.0-SNAPSHOT.jar
@@ -329,7 +330,7 @@ Job has been submitted with JobID e0eba72d5143cea53bcf072335a4b1cb
329
330
330
331
## Validation on Apache Flink Job UI
331
332
332
-
You can find the job in running state on your Flink Web UI
333
+
You can find the job in running state on your Flink Web UI.
333
334
334
335
:::image type="content" source="./media/sink-kafka-to-kibana/flink-elastic-job.png" alt-text="Screenshot showing Kibana UI to start Elasticsearch and Kibana and perform analytics on Kibana." lightbox="./media/sink-kafka-to-kibana/flink-elastic-job.png":::
Copy file name to clipboardExpand all lines: articles/hdinsight-aks/flink/use-apache-nifi-with-datastream-api.md
+2-7Lines changed: 2 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -31,12 +31,8 @@ By combining the low latency streaming features of Apache Flink and the dataflow
31
31
For purposes of this demonstration, we're using a HDInsight Kafka Cluster. Let us prepare HDInsight Kafka topic for the demo.
32
32
33
33
> [!NOTE]
34
-
> Setup a HDInsight cluster with [Apache Kafka](../../hdinsight/kafka/apache-kafka-get-started.md) and replace broker list with your own list before you get started for both Kafka 2.4 and 3.2.
34
+
> Setup a HDInsight cluster with [Apache Kafka](../../hdinsight/kafka/apache-kafka-get-started.md) and replace broker list with your own list before you get started for both Kafka 3.2.
0 commit comments