You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -15,59 +15,59 @@ HDInsight on AKS now offers a Flink Application mode cluster. This cluster lets
15
15
16
16
This deployment mode enables you to assign dedicated resources for specific Flink applications, ensuring that they have enough computing power and memory to handle large workloads efficiently.
:::image type="content" source="./media/application-mode-cluster-on-hdinsight-on-aks/job-manager.png" alt-text="Screenshot showing the job manager window." lightbox="./media/application-mode-cluster-on-hdinsight-on-aks/job-manager.png":::
19
19
20
20
## Advantages
21
21
22
-
1. Simplified cluster deployment with Job jar.
22
+
* Simplified cluster deployment with Job jar.
23
23
24
-
1. User friendly REST API: HDInsight on AKS provides user friendly ARM Rest APIs to manage app mode job operation like Update, Savepoint, Cancel, Delete.
24
+
* User friendly REST API: HDInsight on AKS provides user friendly ARM Rest APIs to manage app mode job operation like Update, Savepoint, Cancel, Delete.
25
25
26
-
1. Easy to manage Job Updates and State Management: The native Azure portal integration provides a hassle-free experience for updating jobs and restoring them to their last saved state (savepoint). This functionality ensures continuity and data integrity throughout the job lifecycle.
26
+
* Easy to manage Job Updates and State Management: The native Azure portal integration provides a hassle-free experience for updating jobs and restoring them to their last saved state (savepoint). This functionality ensures continuity and data integrity throughout the job lifecycle.
27
27
28
-
1. Automate Flink Job(s) using Azure Pipelines or other CI/CD tools: Using HDInsight on AKS, Flink users have access to user-friendly ARM Rest API, you can seamlessly integrate Flink job operations into your Azure Pipeline or other CI/CD tools.
28
+
* Automate Flink Job(s) using Azure Pipelines or other CI/CD tools: Using HDInsight on AKS, Flink users have access to user-friendly ARM Rest API, you can seamlessly integrate Flink job operations into your Azure Pipeline or other CI/CD tools.
29
29
30
30
## Key Features
31
31
32
-
1.**Stop and Start Jobs with Savepoints**: Users can gracefully stop and start their Flink AppMode jobs from their previous state (Savepoint). Savepoints ensure that job progress is preserved, enabling seamless resumptions.
32
+
***Stop and Start Jobs with Savepoints**: Users can gracefully stop and start their Flink AppMode jobs from their previous state (Savepoint). Savepoints ensure that job progress is preserved, enabling seamless resumptions.
33
33
34
-
1.**Job Updates**: User can update the running AppMode job after updating the jar on storage account. This update automatically takes the savepoint and starts the AppMode job with a new jar.
34
+
***Job Updates**: User can update the running AppMode job after updating the jar on storage account. This update automatically takes the savepoint and starts the AppMode job with a new jar.
35
35
36
-
1.**Stateless Updates**: Performing a fresh restart for a AppMode job is simplified through stateless updates. This feature allows users to initiate a clean restart using updated job jar.
36
+
***Stateless Updates**: Performing a fresh restart for a AppMode job is simplified through stateless updates. This feature allows users to initiate a clean restart using updated job jar.
37
37
38
-
1.**Savepoint Management**: At any given moment, users can create savepoints for their running jobs. These savepoints can be listed and used to restart the job from a specific checkpoint as needed.
38
+
***Savepoint Management**: At any given moment, users can create savepoints for their running jobs. These savepoints can be listed and used to restart the job from a specific checkpoint as needed.
39
39
40
-
1.**Cancel**: Cancels the job permanently.
40
+
***Cancel**: Cancels the job permanently.
41
41
42
-
1.**Delete**: Delete AppMode cluster.
42
+
***Delete**: Delete AppMode cluster.
43
43
44
44
## How to create Flink Application Cluster
45
45
46
46
### Prerequisites
47
47
48
-
**Complete the prerequisites in the following sections:
48
+
**Complete the prerequisites in the following sections:**
1.[Create a cluster pool].(../prerequisites-resources.md)
54
+
*[Create a cluster pool](../prerequisites-resources.md)
55
55
56
-
1. Add job jar in Storage Account.
56
+
* Add job jar in Storage Account.
57
57
58
-
Before setting up a Flink App Mode Cluster, several preparatory steps required. One of these steps involves placing the App Mode job JAR in the cluster's storage account.
58
+
Before setting up a Flink App Mode Cluster, several preparatory steps required. One of these steps involves placing the App Mode job JAR in the cluster's storage account.
59
59
60
-
1. Create a Directory for App Mode Job JAR:
60
+
* Create a Directory for App Mode Job JAR:
61
61
62
62
Inside the dedicated containers, create a directory where you upload the App Mode job JAR file. This directory serves as the location for storing JAR files that you want to include in the classpath of the Flink cluster or job.
63
63
64
-
1. Savepoints Directory (Optional):
64
+
* Savepoints Directory (Optional):
65
65
66
66
If users intend to take savepoints during job execution, create a separate directory within the storage account for storing these savepoints. This directory used to store checkpoint data and metadata for savepoints.
67
67
68
-
1. Example directory structure:
68
+
* Example directory structure:
69
69
70
-
:::image type="content" source="./media/application-mode-cluster-on-hdinsight-on-aks/save-points-directory-structure.png" alt-text="Screenshot showing save points directory structure." lightbox="./media/application-mode-cluster-on-hdinsight-on-aks/save-points-directory-structure.png":::
70
+
:::image type="content" source="./media/application-mode-cluster-on-hdinsight-on-aks/save-points-directory-structure.png" alt-text="Screenshot showing the save points directory structure." lightbox="./media/application-mode-cluster-on-hdinsight-on-aks/save-points-directory-structure.png":::
71
71
72
72
73
73
### Create Flink App Mode Cluster
@@ -183,14 +183,14 @@ Portal > HDInsight on AKS Cluster Pool > Flink Cluster > Settings > Flink Jobs.
183
183
184
184
:::image type="content" source="./media/application-mode-cluster-on-hdinsight-on-aks/flink-job.png" alt-text="Screenshot showing Apache Flink jobs that are running." lightbox="./media/application-mode-cluster-on-hdinsight-on-aks/flink-job.png":::
185
185
186
-
1. Stop: Stop job didn't require any parameters. User can stop the job by selecting the action. Once the job is stopped, the job status on the portal to be STOPPED.
187
-
1. Start: Starts the job from savepoint. To start the job, select the stopped job and start it.
188
-
1. Update: Update helps to restart jobs with updated job code. Users need to update the latest job jar in storage location and update the job from portal. This action stops the job with savepoint and start again with latest jar.
189
-
1. Stateless Update: Stateless is like an update, but it involves a fresh restart of the job with the latest code. Once the job is updated, the job status on the portal a shows as **Running.**
190
-
1. Savepoint: Take the savepoint for the Flink Job.
191
-
1. Cancel: Terminate the job.
192
-
1. Delete: Delete AppMode cluster.
193
-
1. View Job Details: To view the job detail user can click on job name, it gives the details about the job and last action result.
186
+
* Stop: Stop job didn't require any parameters. User can stop the job by selecting the action. Once the job is stopped, the job status on the portal to be STOPPED.
187
+
* Start: Starts the job from savepoint. To start the job, select the stopped job and start it.
188
+
* Update: Update helps to restart jobs with updated job code. Users need to update the latest job jar in storage location and update the job from portal. This action stops the job with savepoint and start again with latest jar.
189
+
* Stateless Update: Stateless is like an update, but it involves a fresh restart of the job with the latest code. Once the job is updated, the job status on the portal a shows as **Running.**
190
+
* Savepoint: Take the savepoint for the Flink Job.
191
+
* Cancel: Terminate the job.
192
+
* Delete: Delete AppMode cluster.
193
+
* View Job Details: To view the job detail user can click on job name, it gives the details about the job and last action result.
0 commit comments