Skip to content

Commit 312ed0f

Browse files
authored
Merge pull request #272562 from sreekzz/patch-34
Added script action section.
2 parents 90ebbe4 + 9fc3d6d commit 312ed0f

File tree

3 files changed

+18
-6
lines changed

3 files changed

+18
-6
lines changed

articles/hdinsight-aks/flink/flink-catalog-iceberg-hive.md

Lines changed: 18 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
---
22
title: Table API and SQL - Use Iceberg Catalog type with Hive in Apache Flink® on HDInsight on AKS
3-
description: Learn how to create Iceberg Catalog in Apache Flink® on HDInsight on AKS
3+
description: Learn how to create Iceberg Catalog in Apache Flink® on HDInsight on AKS.
44
ms.service: hdinsight-aks
55
ms.topic: how-to
6-
ms.date: 3/28/2024
6+
ms.date: 04/19/2024
77
---
88

99
# Create Iceberg Catalog in Apache Flink® on HDInsight on AKS
@@ -12,15 +12,27 @@ ms.date: 3/28/2024
1212

1313
[Apache Iceberg](https://iceberg.apache.org/) is an open table format for huge analytic datasets. Iceberg adds tables to compute engines like Apache Flink, using a high-performance table format that works just like a SQL table. Apache Iceberg [supports](https://iceberg.apache.org/multi-engine-support/#apache-flink) both Apache Flink’s DataStream API and Table API.
1414

15-
In this article, we learn how to use Iceberg Table managed in Hive catalog, with Apache Flink on HDInsight on AKS cluster
15+
In this article, we learn how to use Iceberg Table managed in Hive catalog, with Apache Flink on HDInsight on AKS cluster.
1616

1717
## Prerequisites
1818
- You're required to have an operational Flink cluster with secure shell, learn how to [create a cluster](../flink/flink-create-cluster-portal.md)
1919
- Refer this article on how to use CLI from [Secure Shell](./flink-web-ssh-on-portal-to-flink-sql.md) on Azure portal.
2020

2121
### Add dependencies
2222

23-
Once you launch the Secure Shell (SSH), let us start downloading the dependencies required to the SSH node, to illustrate the Iceberg table managed in Hive catalog.
23+
**Script actions**
24+
25+
1. Upload hadoop-hdfs-client and iceberg-flink connector jar into Flink cluster Job Manager and Task Manager.
26+
27+
1. Go to Script actions on Cluster Azure portal.
28+
29+
1. Upload [hadoop-hdfs-client_jar](https://hdiconfigactions2.blob.core.windows.net/flink-script-action/hudi-sa-test.sh)
30+
31+
:::image type="content" source="./media/flink-catalog-iceberg-hive/add-script-action.png" alt-text="Screenshot showing how to add script action.":::
32+
33+
:::image type="content" source="./media/flink-catalog-iceberg-hive/script-action-successful.png" alt-text="Screenshot showing script action added successfully.":::
34+
35+
1. Once you launch the Secure Shell (SSH), let us start downloading the dependencies required to the SSH node, to illustrate the Iceberg table managed in Hive catalog.
2436

2537
```
2638
wget https://repo1.maven.org/maven2/org/apache/iceberg/iceberg-flink-runtime-1.17/1.4.0/iceberg-flink-runtime-1.17-1.4.0.jar -P $FLINK_HOME/lib
@@ -36,7 +48,7 @@ A detailed explanation is given on how to get started with Flink SQL Client usin
3648
```
3749
### Create Iceberg Table managed in Hive catalog
3850

39-
With the following steps, we illustrate how you can create Flink-Iceberg Catalog using Hive catalog
51+
With the following steps, we illustrate how you can create Flink-Iceberg catalog using Hive catalog.
4052

4153
```sql
4254
CREATE CATALOG hive_catalog WITH (
@@ -85,7 +97,7 @@ ADD JAR '/opt/flink-webssh/lib/parquet-column-1.12.2.jar';
8597

8698
#### Output of the Iceberg Table
8799

88-
You can view the Iceberg Table output on the ABFS container
100+
You can view the Iceberg Table output on the ABFS container.
89101

90102
:::image type="content" source="./media/flink-catalog-iceberg-hive/flink-catalog-iceberg-hive-output.png" alt-text="Screenshot showing output of the Iceberg table in ABFS.":::
91103

156 KB
Loading
145 KB
Loading

0 commit comments

Comments
 (0)