You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In this article, learn how to write messages to HBase with Apache Flink DataStream API
13
+
In this article, learn how to write messages to HBase with Apache Flink DataStream API.
14
14
15
15
## Overview
16
16
17
17
Apache Flink offers HBase connector as a sink, with this connector with Flink you can store the output of a real-time processing application in HBase. Learn how to process streaming data on HDInsight Kafka as a source, perform transformations, then sink into HDInsight HBase table.
18
18
19
-
In a real world scenario, this example is a stream analytics layer to realize value from Internet of Things (IOT) analytics, which use live sensor data. The Flink Stream can read data from Kafka topic and write it to HBase table. If there's a real time streaming IOT application, the information can be gathered, transformed, and optimized.
19
+
In a real world scenario, this example is a stream analytics layer to realize value from Internet of Things (IOT) analytics, which use live sensor data. The Flink Stream can read data from Kafka article and write it to HBase table. If there's a real time streaming IOT application, the information can be gathered, transformed, and optimized.
20
20
21
21
22
22
## Prerequisites
@@ -346,18 +346,28 @@ public class KafkaSinkToHbase {
346
346
}
347
347
348
348
```
349
+
## Submit job
349
350
350
-
### Submit job on Secure Shell
351
+
1. Upload the job Jar to Storage Account associated with the Cluster.
351
352
352
-
We use [Flink CLI](./flink-web-ssh-on-portal-to-flink-sql.md) from Azure portal to submit jobs.
353
+
:::image type="content" source="./media/use-flink-to-sink-kafka-message-into-hbase/upload-jar.png" alt-text="Screenshot showing how to upload jar." lightbox="./media/use-flink-to-sink-kafka-message-into-hbase/upload-jar.png":::
353
354
354
-
:::image type="content" source="./media/use-flink-to-sink-kafka-message-into-hbase/submit-job-on-web-ssh.png" alt-text="Screenshot showing how to submit job on web ssh." lightbox="./media/use-flink-to-sink-kafka-message-into-hbase/submit-job-on-web-ssh.png":::
> Make sure to add `Hadoop.class.enable` and `classloader.resolve-order` setting.
359
361
360
-
:::image type="content" source="./media/use-flink-to-sink-kafka-message-into-hbase/check-job-on-flink-ui.png" alt-text="Screenshot showing how to check job on Flink UI." lightbox="./media/use-flink-to-sink-kafka-message-into-hbase/check-job-on-flink-ui.png":::
362
+
1. Select **Job Log Aggregation** to store logs in ABFS.
363
+
364
+
:::image type="content" source="./media/use-flink-to-sink-kafka-message-into-hbase/deployment-type.png" alt-text="Screenshot showing how to submit job on web ssh." lightbox="./media/use-flink-to-sink-kafka-message-into-hbase/deployment-type.png":::
365
+
366
+
1. Submit the job.
367
+
368
+
1. You should be able to see the job submitted status here.
369
+
370
+
:::image type="content" source="./media/use-flink-to-sink-kafka-message-into-hbase/job-status.png" alt-text="Screenshot showing how to check job on Flink UI." lightbox="./media/use-flink-to-sink-kafka-message-into-hbase/job-status.png":::
0 commit comments