Skip to content

Commit 44143ff

Browse files
schonglooschongloo
committed
[FLINK-37515] FLIP-503: Basic support for Blue/Green deployments
1 parent 5b3856b commit 44143ff

File tree

23 files changed

+12904
-5
lines changed

23 files changed

+12904
-5
lines changed

‎docs/content/docs/custom-resource/reference.md

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,23 @@ This serves as a full reference for FlinkDeployment and FlinkSessionJob custom r
5757
| Parameter | Type | Docs |
5858
| ----------| ---- | ---- |
5959

60+
### FlinkBlueGreenDeploymentConfigOptions
61+
**Class**: org.apache.flink.kubernetes.operator.api.spec.FlinkBlueGreenDeploymentConfigOptions
62+
63+
**Description**: Configuration options to be used by the Flink Blue/Green Deployments.
64+
65+
| Parameter | Type | Docs |
66+
| ----------| ---- | ---- |
67+
68+
### FlinkBlueGreenDeploymentSpec
69+
**Class**: org.apache.flink.kubernetes.operator.api.spec.FlinkBlueGreenDeploymentSpec
70+
71+
**Description**: Spec that describes a Flink application with blue/green deployment capabilities.
72+
73+
| Parameter | Type | Docs |
74+
| ----------| ---- | ---- |
75+
| template | org.apache.flink.kubernetes.operator.api.spec.FlinkDeploymentTemplateSpec | |
76+
6077
### FlinkDeploymentSpec
6178
**Class**: org.apache.flink.kubernetes.operator.api.spec.FlinkDeploymentSpec
6279

@@ -78,6 +95,17 @@ This serves as a full reference for FlinkDeployment and FlinkSessionJob custom r
7895
| logConfiguration | java.util.Map<java.lang.String,java.lang.String> | Log configuration overrides for the Flink deployment. Format logConfigFileName -> configContent. |
7996
| mode | org.apache.flink.kubernetes.operator.api.spec.KubernetesDeploymentMode | Deployment mode of the Flink cluster, native or standalone. |
8097

98+
### FlinkDeploymentTemplateSpec
99+
**Class**: org.apache.flink.kubernetes.operator.api.spec.FlinkDeploymentTemplateSpec
100+
101+
**Description**: Template Spec that describes a Flink application managed by the blue/green controller.
102+
103+
| Parameter | Type | Docs |
104+
| ----------| ---- | ---- |
105+
| metadata | io.fabric8.kubernetes.api.model.ObjectMeta | |
106+
| configuration | java.util.Map<java.lang.String,java.lang.String> | |
107+
| spec | org.apache.flink.kubernetes.operator.api.spec.FlinkDeploymentSpec | |
108+
81109
### FlinkSessionJobSpec
82110
**Class**: org.apache.flink.kubernetes.operator.api.spec.FlinkSessionJobSpec
83111

@@ -290,6 +318,33 @@ This serves as a full reference for FlinkDeployment and FlinkSessionJob custom r
290318
| UNKNOWN | Checkpoint format unknown, if the checkpoint was not triggered by the operator. |
291319
| description | org.apache.flink.configuration.description.InlineElement | |
292320

321+
### FlinkBlueGreenDeploymentState
322+
**Class**: org.apache.flink.kubernetes.operator.api.status.FlinkBlueGreenDeploymentState
323+
324+
**Description**: Enumeration of the possible states of the blue/green transition.
325+
326+
| Value | Docs |
327+
| ----- | ---- |
328+
| INITIALIZING_BLUE | We use this state while initializing for the first time, always with a "Blue" deployment type. |
329+
| ACTIVE_BLUE | Identifies the system is running normally with a "Blue" deployment type. |
330+
| ACTIVE_GREEN | Identifies the system is running normally with a "Green" deployment type. |
331+
| TRANSITIONING_TO_BLUE | Identifies the system is transitioning from "Green" to "Blue". |
332+
| TRANSITIONING_TO_GREEN | Identifies the system is transitioning from "Blue" to "Green". |
333+
334+
### FlinkBlueGreenDeploymentStatus
335+
**Class**: org.apache.flink.kubernetes.operator.api.status.FlinkBlueGreenDeploymentStatus
336+
337+
**Description**: Last observed status of the Flink Blue/Green deployment.
338+
339+
| Parameter | Type | Docs |
340+
| ----------| ---- | ---- |
341+
| jobStatus | org.apache.flink.kubernetes.operator.api.status.JobStatus | |
342+
| blueGreenState | org.apache.flink.kubernetes.operator.api.status.FlinkBlueGreenDeploymentState | The state of the blue/green transition. |
343+
| lastReconciledSpec | java.lang.String | Last reconciled (serialized) deployment spec. |
344+
| lastReconciledTimestamp | java.lang.String | Timestamp of last reconciliation. |
345+
| abortTimestamp | java.lang.String | Computed from abortGracePeriodMs, timestamp after which the deployment should be aborted. |
346+
| deploymentReadyTimestamp | java.lang.String | Timestamp when the deployment became READY/STABLE. Used to determine when to delete it. |
347+
293348
### FlinkDeploymentReconciliationStatus
294349
**Class**: org.apache.flink.kubernetes.operator.api.status.FlinkDeploymentReconciliationStatus
295350

‎flink-kubernetes-operator-api/pom.xml

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -212,6 +212,7 @@ under the License.
212212
<include>flinkdeployments.flink.apache.org-v1.yml</include>
213213
<include>flinksessionjobs.flink.apache.org-v1.yml</include>
214214
<include>flinkstatesnapshots.flink.apache.org-v1.yml</include>
215+
<include>flinkbluegreendeployments.flink.apache.org-v1.yml</include>
215216
</includes>
216217
<filtering>false</filtering>
217218
</resource>
@@ -236,6 +237,8 @@ under the License.
236237
<classpath refid="maven.compile.classpath"/>
237238
<arg value="file://${rootDir}/helm/flink-kubernetes-operator/crds/flinkdeployments.flink.apache.org-v1.yml"/>
238239
<arg value="https://raw.githubusercontent.com/apache/flink-kubernetes-operator/release-1.9.0/helm/flink-kubernetes-operator/crds/flinkdeployments.flink.apache.org-v1.yml"/>
240+
<arg value="https://raw.githubusercontent.com/apache/flink-kubernetes-operator/release-1.10.0/helm/flink-kubernetes-operator/crds/flinkdeployments.flink.apache.org-v1.yml"/>
241+
<arg value="https://raw.githubusercontent.com/apache/flink-kubernetes-operator/release-1.11.0/helm/flink-kubernetes-operator/crds/flinkdeployments.flink.apache.org-v1.yml"/>
239242
</java>
240243
</target>
241244
</configuration>
@@ -253,6 +256,24 @@ under the License.
253256
<classpath refid="maven.compile.classpath"/>
254257
<arg value="file://${rootDir}/helm/flink-kubernetes-operator/crds/flinksessionjobs.flink.apache.org-v1.yml"/>
255258
<arg value="https://raw.githubusercontent.com/apache/flink-kubernetes-operator/release-1.9.0/helm/flink-kubernetes-operator/crds/flinksessionjobs.flink.apache.org-v1.yml"/>
259+
<arg value="https://raw.githubusercontent.com/apache/flink-kubernetes-operator/release-1.10.0/helm/flink-kubernetes-operator/crds/flinksessionjobs.flink.apache.org-v1.yml"/>
260+
<arg value="https://raw.githubusercontent.com/apache/flink-kubernetes-operator/release-1.11.0/helm/flink-kubernetes-operator/crds/flinksessionjobs.flink.apache.org-v1.yml"/>
261+
</java>
262+
</target>
263+
</configuration>
264+
</execution>
265+
<execution>
266+
<id>flinkbgdeployments-remove-scale-subresource</id>
267+
<phase>package</phase>
268+
<goals>
269+
<goal>run</goal>
270+
</goals>
271+
<configuration>
272+
<target>
273+
<java classname="org.apache.flink.kubernetes.operator.api.utils.RemoveScaleSubResource"
274+
fork="true" failonerror="true">
275+
<classpath refid="maven.compile.classpath"/>
276+
<arg value="${rootDir}/helm/flink-kubernetes-operator/crds/flinkbluegreendeployments.flink.apache.org-v1.yml"/>
256277
</java>
257278
</target>
258279
</configuration>

‎flink-kubernetes-operator-api/src/main/java/org/apache/flink/kubernetes/operator/api/FlinkBlueGreenDeployment.java

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
package org.apache.flink.kubernetes.operator.api;
19+
20+
import org.apache.flink.annotation.Experimental;
21+
import org.apache.flink.kubernetes.operator.api.spec.FlinkBlueGreenDeploymentSpec;
22+
import org.apache.flink.kubernetes.operator.api.status.FlinkBlueGreenDeploymentStatus;
23+
24+
import com.fasterxml.jackson.annotation.JsonInclude;
25+
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
26+
import io.fabric8.kubernetes.api.model.Namespaced;
27+
import io.fabric8.kubernetes.client.CustomResource;
28+
import io.fabric8.kubernetes.model.annotation.Group;
29+
import io.fabric8.kubernetes.model.annotation.ShortNames;
30+
import io.fabric8.kubernetes.model.annotation.Version;
31+
32+
/** Custom resource definition that represents a deployments with Blue/Green rollout capability. */
33+
@Experimental
34+
@JsonInclude(JsonInclude.Include.NON_NULL)
35+
@JsonDeserialize()
36+
@Group(CrdConstants.API_GROUP)
37+
@Version(CrdConstants.API_VERSION)
38+
@ShortNames({"flinkbgdep"})
39+
public class FlinkBlueGreenDeployment
40+
extends CustomResource<FlinkBlueGreenDeploymentSpec, FlinkBlueGreenDeploymentStatus>
41+
implements Namespaced {}

‎flink-kubernetes-operator-api/src/main/java/org/apache/flink/kubernetes/operator/api/bluegreen/DeploymentType.java

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
package org.apache.flink.kubernetes.operator.api.bluegreen;
19+
20+
import org.apache.flink.kubernetes.operator.api.FlinkDeployment;
21+
22+
/**
23+
* Enumeration of the two possible Flink Blue/Green deployment types. Only one of each type will be
24+
* present at all times for a particular job.
25+
*/
26+
public enum DeploymentType {
27+
/** Identifier for the first or "Blue" deployment type. */
28+
BLUE,
29+
30+
/** Identifier for the second or "Green" deployment type. */
31+
GREEN;
32+
33+
public static final String LABEL_KEY = "flink/blue-green-deployment-type";
34+
35+
public static DeploymentType fromDeployment(FlinkDeployment flinkDeployment) {
36+
String typeAnnotation = flinkDeployment.getMetadata().getLabels().get(LABEL_KEY);
37+
return DeploymentType.valueOf(typeAnnotation);
38+
}
39+
}

‎flink-kubernetes-operator-api/src/main/java/org/apache/flink/kubernetes/operator/api/spec/FlinkBlueGreenDeploymentConfigOptions.java

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
package org.apache.flink.kubernetes.operator.api.spec;
19+
20+
import org.apache.flink.configuration.ConfigOption;
21+
import org.apache.flink.configuration.ConfigOptions;
22+
23+
/** Configuration options to be used by the Flink Blue/Green Deployments. */
24+
public class FlinkBlueGreenDeploymentConfigOptions {
25+
26+
public static final String BLUE_GREEN_CONF_PREFIX = "bluegreen.";
27+
28+
public static final int MIN_ABORT_GRACE_PERIOD_MS = 120000; // 2 mins
29+
30+
public static ConfigOptions.OptionBuilder operatorConfig(String key) {
31+
return ConfigOptions.key(BLUE_GREEN_CONF_PREFIX + key);
32+
}
33+
34+
public static final ConfigOption<Integer> ABORT_GRACE_PERIOD_MS =
35+
operatorConfig("abortGracePeriodMs")
36+
.intType()
37+
.defaultValue(0)
38+
.withDescription(
39+
"The max time to wait for a deployment to become ready before aborting it, in milliseconds. Cannot be smaller than 2 minutes.");
40+
41+
public static final ConfigOption<Integer> RECONCILIATION_RESCHEDULING_INTERVAL_MS =
42+
operatorConfig("reconciliationReschedulingIntervalMs")
43+
.intType()
44+
.defaultValue(15000) // 15 seconds
45+
.withDescription(
46+
"Configurable delay in milliseconds to use when the operator reschedules a reconciliation.");
47+
48+
public static final ConfigOption<Integer> DEPLOYMENT_DELETION_DELAY_MS =
49+
operatorConfig("deploymentDeletionDelayMs")
50+
.intType()
51+
.defaultValue(0)
52+
.withDescription(
53+
"Configurable delay before deleting a deployment after being marked done.");
54+
}

‎flink-kubernetes-operator-api/src/main/java/org/apache/flink/kubernetes/operator/api/spec/FlinkBlueGreenDeploymentSpec.java

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
package org.apache.flink.kubernetes.operator.api.spec;
19+
20+
import org.apache.flink.annotation.Experimental;
21+
22+
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
23+
import lombok.AllArgsConstructor;
24+
import lombok.Data;
25+
import lombok.NoArgsConstructor;
26+
27+
/** Spec that describes a Flink application with blue/green deployment capabilities. */
28+
@Experimental
29+
@Data
30+
@NoArgsConstructor
31+
@AllArgsConstructor
32+
@JsonIgnoreProperties(ignoreUnknown = true)
33+
public class FlinkBlueGreenDeploymentSpec {
34+
35+
private FlinkDeploymentTemplateSpec template;
36+
}

‎flink-kubernetes-operator-api/src/main/java/org/apache/flink/kubernetes/operator/api/spec/FlinkDeploymentTemplateSpec.java

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
package org.apache.flink.kubernetes.operator.api.spec;
19+
20+
import com.fasterxml.jackson.annotation.JsonIgnore;
21+
import com.fasterxml.jackson.annotation.JsonProperty;
22+
import io.fabric8.kubernetes.api.model.ObjectMeta;
23+
import lombok.AllArgsConstructor;
24+
import lombok.Data;
25+
import lombok.NoArgsConstructor;
26+
import lombok.experimental.SuperBuilder;
27+
28+
import java.util.LinkedHashMap;
29+
import java.util.Map;
30+
31+
/** Template Spec that describes a Flink application managed by the blue/green controller. */
32+
@AllArgsConstructor
33+
@NoArgsConstructor
34+
@Data
35+
@SuperBuilder
36+
public class FlinkDeploymentTemplateSpec {
37+
38+
@JsonProperty("metadata")
39+
private ObjectMeta metadata;
40+
41+
@JsonProperty("configuration")
42+
private Map<String, String> configuration;
43+
44+
@JsonProperty("spec")
45+
private FlinkDeploymentSpec spec;
46+
47+
@JsonIgnore
48+
private Map<String, Object> additionalProperties = new LinkedHashMap<String, Object>();
49+
}

‎flink-kubernetes-operator-api/src/main/java/org/apache/flink/kubernetes/operator/api/status/FlinkBlueGreenDeploymentState.java

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
package org.apache.flink.kubernetes.operator.api.status;
19+
20+
/** Enumeration of the possible states of the blue/green transition. */
21+
public enum FlinkBlueGreenDeploymentState {
22+
23+
/**
24+
* We use this state while initializing for the first time, always with a "Blue" deployment
25+
* type.
26+
*/
27+
INITIALIZING_BLUE,
28+
29+
/** Identifies the system is running normally with a "Blue" deployment type. */
30+
ACTIVE_BLUE,
31+
32+
/** Identifies the system is running normally with a "Green" deployment type. */
33+
ACTIVE_GREEN,
34+
35+
/** Identifies the system is transitioning from "Green" to "Blue". */
36+
TRANSITIONING_TO_BLUE,
37+
38+
/** Identifies the system is transitioning from "Blue" to "Green". */
39+
TRANSITIONING_TO_GREEN,
40+
}

0 commit comments

Comments
 (0)