You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight-aks/flink/flink-catalog-iceberg-hive.md
+18-6Lines changed: 18 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,9 +1,9 @@
1
1
---
2
2
title: Table API and SQL - Use Iceberg Catalog type with Hive in Apache Flink® on HDInsight on AKS
3
-
description: Learn how to create Iceberg Catalog in Apache Flink® on HDInsight on AKS
3
+
description: Learn how to create Iceberg Catalog in Apache Flink® on HDInsight on AKS.
4
4
ms.service: hdinsight-aks
5
5
ms.topic: how-to
6
-
ms.date: 3/28/2024
6
+
ms.date: 04/19/2024
7
7
---
8
8
9
9
# Create Iceberg Catalog in Apache Flink® on HDInsight on AKS
@@ -12,15 +12,27 @@ ms.date: 3/28/2024
12
12
13
13
[Apache Iceberg](https://iceberg.apache.org/) is an open table format for huge analytic datasets. Iceberg adds tables to compute engines like Apache Flink, using a high-performance table format that works just like a SQL table. Apache Iceberg [supports](https://iceberg.apache.org/multi-engine-support/#apache-flink) both Apache Flink’s DataStream API and Table API.
14
14
15
-
In this article, we learn how to use Iceberg Table managed in Hive catalog, with Apache Flink on HDInsight on AKS cluster
15
+
In this article, we learn how to use Iceberg Table managed in Hive catalog, with Apache Flink on HDInsight on AKS cluster.
16
16
17
17
## Prerequisites
18
18
- You're required to have an operational Flink cluster with secure shell, learn how to [create a cluster](../flink/flink-create-cluster-portal.md)
19
19
- Refer this article on how to use CLI from [Secure Shell](./flink-web-ssh-on-portal-to-flink-sql.md) on Azure portal.
20
20
21
21
### Add dependencies
22
22
23
-
Once you launch the Secure Shell (SSH), let us start downloading the dependencies required to the SSH node, to illustrate the Iceberg table managed in Hive catalog.
23
+
**Script actions**
24
+
25
+
1. Upload hadoop-hdfs-client and iceberg-flink connector jar into Flink cluster Job Manager and Task Manager.
1. Once you launch the Secure Shell (SSH), let us start downloading the dependencies required to the SSH node, to illustrate the Iceberg table managed in Hive catalog.
@@ -36,7 +48,7 @@ A detailed explanation is given on how to get started with Flink SQL Client usin
36
48
```
37
49
### Create Iceberg Table managed in Hive catalog
38
50
39
-
With the following steps, we illustrate how you can create Flink-Iceberg Catalog using Hive catalog
51
+
With the following steps, we illustrate how you can create Flink-Iceberg catalog using Hive catalog.
40
52
41
53
```sql
42
54
CREATE CATALOG hive_catalog WITH (
@@ -85,7 +97,7 @@ ADD JAR '/opt/flink-webssh/lib/parquet-column-1.12.2.jar';
85
97
86
98
#### Output of the Iceberg Table
87
99
88
-
You can view the Iceberg Table output on the ABFS container
100
+
You can view the Iceberg Table output on the ABFS container.
89
101
90
102
:::image type="content" source="./media/flink-catalog-iceberg-hive/flink-catalog-iceberg-hive-output.png" alt-text="Screenshot showing output of the Iceberg table in ABFS.":::
0 commit comments