-
Notifications
You must be signed in to change notification settings - Fork 484
[FLINK-33634] Add Conditions to Flink CRD's Status field #957
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 12 commits
afdf40b
1d84164
727bb4b
c66fd1f
c623f33
196c494
32294f4
7b6d6a1
4f6679b
93be98a
9faab16
b40dcf3
70030db
9311031
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
package org.apache.flink.kubernetes.operator.api.status; | ||
|
||
/** Condition Status of the Flink JobManager Kubernetes deployment. */ | ||
public enum JobManagerDeploymentConditionStatus { | ||
READY("True", "JobManagerReady", "JobManager is running and ready to receive REST API calls"), | ||
MISSING("False", "JobManagerDeploymentMissing", "JobManager deployment not found"), | ||
DEPLOYING("False", "JobManagerIsDeploying", "JobManager process is starting up"), | ||
DEPLOYED_NOT_READY( | ||
"False", | ||
"DeployedNotReady", | ||
"JobManager is running but not ready yet to receive REST API calls"), | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit ready yet -> yet ready There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks, I will change that . |
||
ERROR("False", "Error", "JobManager deployment failed"); | ||
|
||
private String status; | ||
private String reason; | ||
private String message; | ||
|
||
JobManagerDeploymentConditionStatus(String status, String reason, String message) { | ||
this.status = status; | ||
this.reason = reason; | ||
this.message = message; | ||
} | ||
|
||
public String getReason() { | ||
return reason; | ||
} | ||
|
||
public String getMessage() { | ||
return message; | ||
} | ||
|
||
public String getStatus() { | ||
return status; | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,131 @@ | ||
/* | ||
* Licensed to the Apache Software Foundation (ASF) under one or more | ||
* contributor license agreements. See the NOTICE file distributed with | ||
* this work for additional information regarding copyright ownership. | ||
* The ASF licenses this file to You under the Apache License, Version 2.0 | ||
* (the "License"); you may not use this file except in compliance with | ||
* the License. You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
|
||
package org.apache.flink.kubernetes.operator.api.utils; | ||
|
||
import org.apache.flink.api.common.JobStatus; | ||
import org.apache.flink.kubernetes.operator.api.status.FlinkDeploymentStatus; | ||
import org.apache.flink.kubernetes.operator.api.status.JobManagerDeploymentConditionStatus; | ||
import org.apache.flink.kubernetes.operator.api.status.JobManagerDeploymentStatus; | ||
|
||
import io.fabric8.kubernetes.api.model.Condition; | ||
import io.fabric8.kubernetes.api.model.ConditionBuilder; | ||
|
||
import java.text.SimpleDateFormat; | ||
import java.util.Date; | ||
import java.util.List; | ||
import java.util.Map; | ||
|
||
import static org.apache.flink.api.common.JobStatus.RUNNING; | ||
|
||
/** Creates a condition object with the type, status, message and reason. */ | ||
public class ConditionUtils { | ||
public static final String CONDITION_TYPE_RUNNING = "Running"; | ||
private static final Map<String, Condition> SESSION_MODE_CONDITION = | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @gyfora are you happy with this map in the utils class - it might be cleaner to be in its own appropriately named class, same for the application mode map There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. To be completely honest, I am not very happy about it... :) This is again a copy paste, logic that could be simply replaced with:
I have already given the same exact feedback to the JobStatus, not sure why it wasn't followed here too. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It has been moved to method and moved reason and messages to enum and getting from there. |
||
Map.of( | ||
JobManagerDeploymentStatus.READY.name(), | ||
new ConditionBuilder() | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit: you could refactor to a method and pass JobManagerDeploymentConditionStatus.READY, as the rest of the condition builder is the same apart from the status .
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It has been moved to method and reason and messages are getting from enum. |
||
.withType(CONDITION_TYPE_RUNNING) | ||
.withStatus(JobManagerDeploymentConditionStatus.READY.getStatus()) | ||
.withReason(JobManagerDeploymentConditionStatus.READY.getReason()) | ||
.withMessage(JobManagerDeploymentConditionStatus.READY.getMessage()) | ||
.build(), | ||
JobManagerDeploymentStatus.MISSING.name(), | ||
new ConditionBuilder() | ||
.withType(CONDITION_TYPE_RUNNING) | ||
.withStatus(JobManagerDeploymentConditionStatus.MISSING.getStatus()) | ||
.withReason(JobManagerDeploymentConditionStatus.MISSING.getReason()) | ||
.withMessage(JobManagerDeploymentConditionStatus.MISSING.getMessage()) | ||
.build(), | ||
JobManagerDeploymentStatus.DEPLOYING.name(), | ||
new ConditionBuilder() | ||
.withType(CONDITION_TYPE_RUNNING) | ||
.withStatus(JobManagerDeploymentConditionStatus.DEPLOYING.getStatus()) | ||
.withReason(JobManagerDeploymentConditionStatus.DEPLOYING.getReason()) | ||
.withMessage(JobManagerDeploymentConditionStatus.DEPLOYING.getMessage()) | ||
.build(), | ||
JobManagerDeploymentStatus.DEPLOYED_NOT_READY.name(), | ||
new ConditionBuilder() | ||
.withType(CONDITION_TYPE_RUNNING) | ||
.withStatus( | ||
JobManagerDeploymentConditionStatus.DEPLOYED_NOT_READY | ||
.getStatus()) | ||
.withReason( | ||
JobManagerDeploymentConditionStatus.DEPLOYED_NOT_READY | ||
.getReason()) | ||
.withMessage( | ||
JobManagerDeploymentConditionStatus.DEPLOYED_NOT_READY | ||
.getMessage()) | ||
.build(), | ||
JobManagerDeploymentStatus.ERROR.name(), | ||
new ConditionBuilder() | ||
.withType(CONDITION_TYPE_RUNNING) | ||
.withStatus(JobManagerDeploymentConditionStatus.ERROR.getStatus()) | ||
.withReason(JobManagerDeploymentConditionStatus.ERROR.getReason()) | ||
.withMessage(JobManagerDeploymentConditionStatus.ERROR.getMessage()) | ||
.build()); | ||
|
||
public static Condition getCondition(FlinkDeploymentStatus flinkDeploymentStatus) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Only the parameters that are required should be passed not the complete FlinkDeploymentStatus. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
org.apache.flink.kubernetes.operator.api.status.JobStatus status = | ||
flinkDeploymentStatus.getJobStatus(); | ||
Condition conditionToAdd = null; | ||
if (status != null) { | ||
|
||
JobStatus jobStatus = status.getState(); | ||
|
||
conditionToAdd = | ||
jobStatus == null | ||
? SESSION_MODE_CONDITION.get( | ||
flinkDeploymentStatus.getJobManagerDeploymentStatus().name()) | ||
: getApplicationModeCondition(jobStatus); | ||
} | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. can we not update the last transition time here? |
||
return conditionToAdd; | ||
} | ||
|
||
public static void updateLastTransitionTime(List<Condition> conditions, Condition condition) { | ||
if (condition == null) { | ||
return; | ||
} | ||
if (isLastTransactionTimeStampUpdateRequired(conditions, condition)) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. would it not be simpler to have an existing condition that is null or a value, then we do not need to get the first element of the list twice. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. updated, |
||
condition.setLastTransitionTime( | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Above is from |
||
new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'").format(new Date())); | ||
} else { | ||
condition.setLastTransitionTime(conditions.get(0).getLastTransitionTime()); | ||
} | ||
} | ||
|
||
private static Condition getApplicationModeCondition(JobStatus jobStatus) { | ||
return new ConditionBuilder() | ||
.withType(CONDITION_TYPE_RUNNING) | ||
.withStatus(jobStatus == RUNNING ? "True" : "False") | ||
.withReason(toCameCase(jobStatus.name())) | ||
.withMessage("Job state " + jobStatus.name()) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. should the message say Job status - as that is what we are reading? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, |
||
.build(); | ||
} | ||
|
||
private static String toCameCase(String reason) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think you mean toCamelCase There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thank you, my bad. |
||
reason = reason.toLowerCase(); | ||
return reason.substring(0, 1).toUpperCase() + reason.substring(1); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Camel case is not just upper casing the first letter. We may need to upper case in the string as well. I suggest mapping the lower case to the appropriate camel cased reason. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Right, but not as of now as per here. If we go with mapping, we end up with new method, do we required them?. |
||
} | ||
|
||
private static boolean isLastTransactionTimeStampUpdateRequired( | ||
List<Condition> conditions, Condition newCondition) { | ||
return conditions.isEmpty() | ||
|| !conditions.get(0).getStatus().equals(newCondition.getStatus()); | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is deploying a starting up the same thing?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have referenced from here https://github.com/apache/flink-kubernetes-operator/blob/main/flink-kubernetes-operator-api/src/main/java/org/apache/flink/kubernetes/operator/api/status/JobManagerDeploymentStatus.java#L29. I would have as
JobManager deployment is in progress
. Happy to make change , WDYT?.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK that makes sense. Maybe link to that class in a comment to say where you got the descriptions from.