Skip to content

Commit 0f0b2c2

Browse files
authored
[FLINK-38124][docs] Add README for Autoscaling example (#1024)
1 parent a57579c commit 0f0b2c2

File tree

1 file changed

+100
-0
lines changed

1 file changed

+100
-0
lines changed

examples/autoscaling/README.md

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
<!--
2+
Licensed to the Apache Software Foundation (ASF) under one
3+
or more contributor license agreements. See the NOTICE file
4+
distributed with this work for additional information
5+
regarding copyright ownership. The ASF licenses this file
6+
to you under the Apache License, Version 2.0 (the
7+
"License"); you may not use this file except in compliance
8+
with the License. You may obtain a copy of the License at
9+
10+
http://www.apache.org/licenses/LICENSE-2.0
11+
12+
Unless required by applicable law or agreed to in writing,
13+
software distributed under the License is distributed on an
14+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
KIND, either express or implied. See the License for the
16+
specific language governing permissions and limitations
17+
under the License.
18+
-->
19+
20+
# Flink Kubernetes Autoscaling Example
21+
22+
## Overview
23+
24+
This example contains two Flink applications showcasing the Flink Autoscaler capabilities:
25+
26+
- `AutoscalingExample.java` with its accompanying `autoscaling.yaml` containing the `FlinkDeployment` definition
27+
- `LoadSimulationPipeline.java` with its accompanying `autoscaling-dynamic.yaml` containing the `FlinkDeployment` definition
28+
29+
Both applications are packaged into a single fat jar, which is then included in a Docker image
30+
built from the provided `Dockerfile`.
31+
32+
### AutoscalingExample
33+
34+
This application contains a source that emits long values, a map function with an emulated
35+
processing load, and a discard sink. The processing load per record can be configured by
36+
adjusting the job argument in `autoscaling.yaml`:
37+
38+
```
39+
job:
40+
args: ["10"]
41+
```
42+
43+
The argument value specifies how many synthetic iterations are performed for each record.
44+
45+
### LoadSimulationPipeline
46+
47+
This application simulates fluctuating load that could be configured via the job arguments in
48+
`autoscaling-dynamic.yaml`:
49+
50+
```
51+
job:
52+
args:
53+
- --maxLoadPerTask "1;2;4;8;16;\n16;8;4;2;1\n8;4;16;1;2" --repeatsAfterMinutes "60"
54+
```
55+
56+
Refer to `LoadSimulationPipeline.java`'s JavaDoc and comments for the details concerning the argument
57+
notation and simulated load pattern.
58+
59+
## Usage
60+
61+
The following steps assume that you have the Flink Kubernetes Operator installed and running in
62+
your environment. If not, please follow the Flink Kubernetes Operator [quickstart](https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/try-flink-kubernetes-operator/quick-start/) to start with.
63+
64+
**Step 1**: Build Autoscaling example maven project
65+
```bash
66+
cd examples/autoscaling
67+
mvn clean package
68+
```
69+
70+
**Step 2**: Build docker image
71+
```bash
72+
# Uncomment when building for local minikube env:
73+
# eval $(minikube docker-env)
74+
75+
DOCKER_BUILDKIT=1 docker build . -t autoscaling-example:latest
76+
```
77+
78+
This step will create an image based on an official Flink base image including the Autoscaling application jar.
79+
80+
**Step 3**: Only for AutoscalingExample: Mount volume to keep savepoints and checkpoints
81+
82+
```bash
83+
# Assuming minikube is used for local testing or alternatively ensure any other k8s cluster setup with access to a persistent volume
84+
mkdir /tmp/flink # or any other local directory
85+
minikube mount --uid=9999 --gid=9999 /tmp/flink:/tmp/flink
86+
```
87+
88+
**Step 4**: Submit FlinkDeployment Yaml
89+
90+
For *AutoscalingExample*:
91+
92+
```bash
93+
kubectl apply -f autoscaling.yaml
94+
```
95+
96+
or for *LoadSimulationPipeline*:
97+
98+
```bash
99+
kubectl apply -f autoscaling-dynamic.yaml
100+
```

0 commit comments

Comments
 (0)