Skip to content

[FLINK-33634] Add Conditions to Flink CRD's Status field #957

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -19,16 +19,20 @@

import org.apache.flink.annotation.Experimental;
import org.apache.flink.kubernetes.operator.api.spec.FlinkDeploymentSpec;
import org.apache.flink.kubernetes.operator.api.utils.ConditionUtils;

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import io.fabric8.kubernetes.api.model.Condition;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.EqualsAndHashCode;
import lombok.NoArgsConstructor;
import lombok.ToString;
import lombok.experimental.SuperBuilder;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

/** Last observed status of the Flink deployment. */
Expand All @@ -55,4 +59,13 @@ public class FlinkDeploymentStatus extends CommonStatus<FlinkDeploymentSpec> {

/** Information about the TaskManagers for the scale subresource. */
private TaskManagerInfo taskManager;

/** Condition of the CR . */
private List<Condition> conditions = new ArrayList<>();

public List<Condition> getConditions() {
Condition condition = ConditionUtils.getCondition(this);
ConditionUtils.updateLastTransitionTime(conditions, condition);
return condition == null ? List.of() : List.of(condition);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
package org.apache.flink.kubernetes.operator.api.status;

/** Condition Status of the Flink JobManager Kubernetes deployment. */
public enum JobManagerDeploymentConditionStatus {
READY("True", "JobManagerReady", "JobManager is running and ready to receive REST API calls"),
MISSING("False", "JobManagerDeploymentMissing", "JobManager deployment not found"),
DEPLOYING("False", "JobManagerIsDeploying", "JobManager process is starting up"),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is deploying a starting up the same thing?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK that makes sense. Maybe link to that class in a comment to say where you got the descriptions from.

DEPLOYED_NOT_READY(
"False",
"DeployedNotReady",
"JobManager is running but not ready yet to receive REST API calls"),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit ready yet -> yet ready

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I will change that .

ERROR("False", "Error", "JobManager deployment failed");

private String status;
private String reason;
private String message;

JobManagerDeploymentConditionStatus(String status, String reason, String message) {
this.status = status;
this.reason = reason;
this.message = message;
}

public String getReason() {
return reason;
}

public String getMessage() {
return message;
}

public String getStatus() {
return status;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.flink.kubernetes.operator.api.utils;

import org.apache.flink.api.common.JobStatus;
import org.apache.flink.kubernetes.operator.api.status.FlinkDeploymentStatus;
import org.apache.flink.kubernetes.operator.api.status.JobManagerDeploymentConditionStatus;
import org.apache.flink.kubernetes.operator.api.status.JobManagerDeploymentStatus;

import io.fabric8.kubernetes.api.model.Condition;
import io.fabric8.kubernetes.api.model.ConditionBuilder;

import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.List;
import java.util.Map;

import static org.apache.flink.api.common.JobStatus.RUNNING;

/** Creates a condition object with the type, status, message and reason. */
public class ConditionUtils {
public static final String CONDITION_TYPE_RUNNING = "Running";
private static final Map<String, Condition> SESSION_MODE_CONDITION =

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gyfora are you happy with this map in the utils class - it might be cleaner to be in its own appropriately named class, same for the application mode map

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be completely honest, I am not very happy about it... :)

This is again a copy paste, logic that could be simply replaced with:

public Condition from(JobManagerDeploymentStatus jmStatus) {
   return new ConditionBuilder()
                            .withType(jmStatus  == DEPLOYED ? CONDITION_TYPE_RUNNING : NOT_RUNNING)
                            .withStatus(jmStatus.getStatus())
                            .withReason(jmStatus.getReason())
                            .withMessage(jmStatus.getMessage())
                            .build()
}

I have already given the same exact feedback to the JobStatus, not sure why it wasn't followed here too.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has been moved to method and moved reason and messages to enum and getting from there.

Map.of(
JobManagerDeploymentStatus.READY.name(),
new ConditionBuilder()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: you could refactor to a method and pass JobManagerDeploymentConditionStatus.READY, as the rest of the condition builder is the same apart from the status .
So

JobManagerDeploymentStatus.READY.name(),
createRunningConditionBuilder(JobManagerDeploymentConditionStatus.READY),
 JobManagerDeploymentStatus.MISSING.name(),
createRunningConditionBuilder(JobManagerDeploymentConditionStatus.MISSING),
...

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has been moved to method and reason and messages are getting from enum.

.withType(CONDITION_TYPE_RUNNING)
.withStatus(JobManagerDeploymentConditionStatus.READY.getStatus())
.withReason(JobManagerDeploymentConditionStatus.READY.getReason())
.withMessage(JobManagerDeploymentConditionStatus.READY.getMessage())
.build(),
JobManagerDeploymentStatus.MISSING.name(),
new ConditionBuilder()
.withType(CONDITION_TYPE_RUNNING)
.withStatus(JobManagerDeploymentConditionStatus.MISSING.getStatus())
.withReason(JobManagerDeploymentConditionStatus.MISSING.getReason())
.withMessage(JobManagerDeploymentConditionStatus.MISSING.getMessage())
.build(),
JobManagerDeploymentStatus.DEPLOYING.name(),
new ConditionBuilder()
.withType(CONDITION_TYPE_RUNNING)
.withStatus(JobManagerDeploymentConditionStatus.DEPLOYING.getStatus())
.withReason(JobManagerDeploymentConditionStatus.DEPLOYING.getReason())
.withMessage(JobManagerDeploymentConditionStatus.DEPLOYING.getMessage())
.build(),
JobManagerDeploymentStatus.DEPLOYED_NOT_READY.name(),
new ConditionBuilder()
.withType(CONDITION_TYPE_RUNNING)
.withStatus(
JobManagerDeploymentConditionStatus.DEPLOYED_NOT_READY
.getStatus())
.withReason(
JobManagerDeploymentConditionStatus.DEPLOYED_NOT_READY
.getReason())
.withMessage(
JobManagerDeploymentConditionStatus.DEPLOYED_NOT_READY
.getMessage())
.build(),
JobManagerDeploymentStatus.ERROR.name(),
new ConditionBuilder()
.withType(CONDITION_TYPE_RUNNING)
.withStatus(JobManagerDeploymentConditionStatus.ERROR.getStatus())
.withReason(JobManagerDeploymentConditionStatus.ERROR.getReason())
.withMessage(JobManagerDeploymentConditionStatus.ERROR.getMessage())
.build());

public static Condition getCondition(FlinkDeploymentStatus flinkDeploymentStatus) {
Copy link

@davidradl davidradl May 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only the parameters that are required should be passed not the complete FlinkDeploymentStatus.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think from here @gyfora meant by operate on status on that util method .

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think from here @gyfora meant by operate on status on that util method .

OK I see

org.apache.flink.kubernetes.operator.api.status.JobStatus status =
flinkDeploymentStatus.getJobStatus();
Condition conditionToAdd = null;
if (status != null) {

JobStatus jobStatus = status.getState();

conditionToAdd =
jobStatus == null
? SESSION_MODE_CONDITION.get(
flinkDeploymentStatus.getJobManagerDeploymentStatus().name())
: getApplicationModeCondition(jobStatus);
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we not update the last transition time here?

return conditionToAdd;
}

public static void updateLastTransitionTime(List<Condition> conditions, Condition condition) {
if (condition == null) {
return;
}
if (isLastTransactionTimeStampUpdateRequired(conditions, condition)) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it not be simpler to have an existing condition that is null or a value, then we do not need to get the first element of the list twice.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated,

condition.setLastTransitionTime(

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Above is from flink-autoscaler , but flink-kubernetes-operator-api , doesn't have dependency to flink-autoscaler , so we can't utilise it.

new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'").format(new Date()));
} else {
condition.setLastTransitionTime(conditions.get(0).getLastTransitionTime());
}
}

private static Condition getApplicationModeCondition(JobStatus jobStatus) {
return new ConditionBuilder()
.withType(CONDITION_TYPE_RUNNING)
.withStatus(jobStatus == RUNNING ? "True" : "False")
.withReason(toCameCase(jobStatus.name()))
.withMessage("Job state " + jobStatus.name())

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should the message say Job status - as that is what we are reading?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes,

.build();
}

private static String toCameCase(String reason) {
Copy link

@davidradl davidradl May 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you mean toCamelCase

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, my bad.

reason = reason.toLowerCase();
return reason.substring(0, 1).toUpperCase() + reason.substring(1);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Camel case is not just upper casing the first letter. We may need to upper case in the string as well. I suggest mapping the lower case to the appropriate camel cased reason.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, but not as of now as per here. If we go with mapping, we end up with new method, do we required them?.

}

private static boolean isLastTransactionTimeStampUpdateRequired(
List<Condition> conditions, Condition newCondition) {
return conditions.isEmpty()
|| !conditions.get(0).getStatus().equals(newCondition.getStatus());
}
}
Loading