Skip to content

Commit 83ec184

Browse files
committed
Consolidate workload controllers status: add new conditions + analysis
1 parent 2bca996 commit 83ec184

File tree

1 file changed

+119
-33
lines changed
  • keps/sig-apps/2804-consolidate-workload-controllers-status

1 file changed

+119
-33
lines changed

keps/sig-apps/2804-consolidate-workload-controllers-status/README.md

Lines changed: 119 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -86,11 +86,24 @@ tags, and then generate with `hack/update-toc.sh`.
8686
- [User Stories (Optional)](#user-stories-optional)
8787
- [Story 1](#story-1)
8888
- [Story 2](#story-2)
89+
- [Overview of all conditions](#overview-of-all-conditions)
8990
- [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats-optional)
91+
- [Progressing](#progressing)
92+
- [Complete](#complete)
93+
- [Failed](#failed)
94+
- [Available](#available)
95+
- [Batch Workloads Conditions: Waiting & Running](#batch-workloads-conditions-waiting--running)
9096
- [Risks and Mitigations](#risks-and-mitigations)
9197
- [Design Details](#design-details)
9298
- [Test Plan](#test-plan)
9399
- [Graduation Criteria](#graduation-criteria)
100+
- [Alpha](#alpha)
101+
- [Alpha -> Beta Graduation](#alpha---beta-graduation)
102+
- [Beta -> GA Graduation](#beta---ga-graduation)
103+
- [Alpha](#alpha-1)
104+
- [Beta](#beta)
105+
- [GA](#ga)
106+
- [Deprecation](#deprecation)
94107
- [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy)
95108
- [Version Skew Strategy](#version-skew-strategy)
96109
- [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire)
@@ -150,10 +163,14 @@ Items marked with (R) are required *prior to targeting to a milestone / release*
150163

151164
## Summary
152165

153-
The main goal of this enhancement is to consolidate the workload controller lifecycle
154-
state. This includes defining Progressing, Complete or Failed conditions for:
166+
The main goal of this enhancement is to compare all the workload conditions and consolidate the workload controller lifecycle
167+
state. The secondary goal is to identify and expose other conditions that could bring benefit to the users.
168+
This includes defining conditions (Waiting, Running, Progressing, Available) for:
169+
- Deployment
155170
- DaemonSet
171+
- ReplicaSet & ReplicationController
156172
- StatefulSet
173+
- Job
157174

158175
<!--
159176
This section is incredibly important for producing high-quality, user-focused
@@ -185,17 +202,22 @@ demonstrate the interest in a KEP within the wider Kubernetes community.
185202
[experience reports]: https://github.com/golang/go/wiki/ExperienceReports
186203
-->
187204

188-
Today only deployment controller has [status](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#deployment-status) to reflect the state during it's lifecycle. This enhancement extends the scope of
189-
those of conditions to other controllers(DaemonSet and StatefulSet).
205+
Today only deployment controller has a [status](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#deployment-status) to fully reflect the state during its lifecycle.
206+
This enhancement extends the scope of those and other conditions to other controllers (DaemonSet, Job, ReplicaSet & ReplicationController, StatefulSet).
190207

191208
### Goals
192209

193-
The current status of a workload can be depicted via its conditions. It can be
210+
The current status of a workload can be depicted via its conditions. It can be a subset of:
194211
- Progressing
212+
- Available
213+
- ReplicaFailure
214+
- Suspended
195215
- Complete
196-
- Failed to Progress
216+
- Failed (to progress)
217+
- Waiting (Job only)
218+
- Running (Job only)
197219

198-
All workload controllers should have above conditions to reflect their states
220+
Workload controllers should have above conditions (when applicable) to reflect their states.
199221

200222
### Non-Goals
201223

@@ -239,6 +261,24 @@ As an end-user of Kubernetes, I'd like all my workload controllers to have consi
239261
As an developer building Kubernetes Operators, I'd like all my operators deployed to have
240262
consistent states.
241263

264+
265+
### Overview of all conditions
266+
267+
The following table gives an overview on what conditions each of the workload resources support.
268+
269+
| | Progressing | Available | ReplicaFailure | Suspended | Failed | Complete |
270+
| --------------------------------- | ----------- | --------- | --------------- | --------- | ------ | -------- |
271+
| ReplicaSet & ReplicationController | - | - | failed to create / delete pod (FailedCreate, FailedDelete) | - | - | - |
272+
| Deployment | True when scaling replicas / creating-updating new ReplicaSet / successfully finished progressing (Pods ready or available for MinReadySeconds). False when failed creating ReplicaSet / reached progressDeadlineSeconds. Unknown when rollout paused | True if if required number of replicas is available (takes MaxSurge and MaxUnavailable into account) | failure propagated from new or old ReplicaSet | - | -* | -* |
273+
| StatefulSet | - | - | - | - | - | - |
274+
| DaemonSet | - | - | - | - | - | - |
275+
| Job | - | - | - | True / False when suspended / resumed | failed execution (BackoffLimitExceeded, DeadlineExceeded)| completed execution |
276+
| CronJob** | - | - | - | - | - | - |
277+
278+
**\* Success of the rollout is instead represented by a Progressing condition (status and reason)**
279+
280+
**\*\* CronJob does not even have Conditions field in its Status**
281+
242282
### Notes/Constraints/Caveats (Optional)
243283

244284
<!--
@@ -247,11 +287,25 @@ What are some important details that didn't come across above?
247287
Go in to as much detail as necessary here.
248288
This might be a good place to talk about core concepts and how they relate.
249289
-->
250-
We are proposing 3 new conditions called
251-
- Progressing
252-
- Complete
253-
- Failed
254-
to be added to DaemonSet and Stateful controllers. The definitions are similar to what we have for [Deployment controller](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#deployment-status).
290+
We are proposing 4 new conditions to be added to the following controllers:
291+
- Available (DaemonSet, ReplicaSet & ReplicationController, StatefulSet)
292+
- Progressing (DaemonSet, StatefulSet)
293+
- Waiting (Job)
294+
- Running (Job)
295+
296+
The definitions for Progressing condition (including Failed/Complete) are similar to what we have for [Deployment controller](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#deployment-status).
297+
298+
299+
The following table is indicating what conditions are currently available (`X`) and what conditions will be added (`A`).
300+
301+
| | Waiting | Running | Progressing | Available | ReplicaFailure | Suspended | Failed | Complete |
302+
| --------------------------------- | --------| ------- | ----------- | --------- | --------------- | --------- | ------ | -------- |
303+
| ReplicaSet & ReplicationController | - | - | - | A | X | - | - | - |
304+
| Deployment | - | - | X | X | X | - | - | - |
305+
| StatefulSet | - | - | A | A | - | - | - | - |
306+
| DaemonSet | - | - | A | A | - | - | - | - |
307+
| Job | A | A | - | - | - | X | X | X |
308+
| CronJob | - | - | - | - | - | - | - | - |
255309

256310
#### Progressing
257311
Kubernetes marks a DaemonSet or Stateful as `progressing` when:
@@ -260,11 +314,11 @@ Kubernetes marks a DaemonSet or Stateful as `progressing` when:
260314
- New DaemonSet or StatefulSet pods become Ready or available
261315

262316
#### Complete
263-
Kubernetes marks a DaemonSet or Stateful as complete when it has the following characteristics:
317+
Kubernetes marks a DaemonSet, ReplicaSet or Stateful as `complete` when it has the following characteristics:
264318

265-
- All of the replicas associated with the DaemonSet or StatefulSet have been updated to the latest version you've specified, meaning any updates you've requested have been completed.
266-
- All of the replicas associated with the DaemonSet or StatefulSet are available.
267-
- No old replicas for the DaemonSet or Stateful are running.
319+
- All of the replicas/pods associated with the DaemonSet or StatefulSet have been updated to the latest version you've specified, meaning any updates you've requested have been completed.
320+
- All of the replicas/pods associated with the DaemonSet, ReplicaSet or StatefulSet are available.
321+
- No old or mischeduled replicas/pods for the DaemonSet, ReplicaSet or Stateful are running.
268322

269323
#### Failed
270324
In order to introduce this condition we need to come up with a new field called `.spec.progressDeadlineSeconds` which denotes the number of seconds the controller waits before indicating(in the workload controller status) that the controller progress has stalled.
@@ -273,11 +327,37 @@ There are many factors that can cause failure to happen like:
273327
- Insufficient quota
274328
- Readiness probe failures
275329
- Image pull errors
276-
330+
- Failed to create/delete pod
277331

278332
See the [Kubernetes API Conventions](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#typical-status-properties) for more information on status conditions
279333

280-
Because of the number of changes that are involved as part of this effort, we are thinking of a phased approach where we introduce these conditions to DaemonSet controller behind a featuregate flag first in one release and then make similar changes to StatefulSet controller. This also needs some code refactoring of existing conditions for Deployment controller.
334+
Because of the number of changes that are involved as part of this effort, we are thinking of a phased approach where we introduce these conditions to DaemonSet controller behind a featuregate flag first in one release and then make similar changes to ReplicaSet and StatefulSet controller.
335+
This also needs some code refactoring of existing conditions for Deployment controller.
336+
337+
#### Available
338+
Kubernetes marks a ReplicaSet, StatefulSet as `available` when number of available replicas reaches number of replicas.
339+
- This could be confusing in ReplicaSets a bit since Deployment could get available sooner than its ReplicaSet due `maxUnavailable`.
340+
- Available replicas is alpha feature guarded by StatefulSetMinReadySeconds gate in StatefulSets, but the value defaults to ReadyReplicas when the feature gate is disabled so using it shouldn't be an issue.
341+
342+
Kubernetes marks DaemonSet as `available` when `numberUnavailable` or `desiredNumberScheduled - numberAvailable` is zero.
343+
344+
#### Batch Workloads Conditions: Waiting & Running
345+
346+
Batch workloads behaviour does not properly map to the other workloads that are expected to be always running (eg. `Progressing` condition and its behaviour).
347+
- Jobs are indicating a `Failed`/`Complete` state in a standalone condition compared to `Progressing` condition in other workloads.
348+
- `.spec.activeDeadlineSeconds` variable, is similar to `progressDeadlineSeconds`, but does not have a default value.
349+
It also resets on suspension, so its behaviour is a bit different.
350+
351+
352+
Kubernetes marks a Job as `waiting` if one of the following conditions is true:
353+
354+
- There are no Pods with phase `Running` and there is at least one Pod with phase `Pending`.
355+
- The Job is suspended.
356+
357+
Kubernetes marks a Job as `running` if there is at least one Pod with phase `Running`.
358+
359+
This KEP does not introduce CronJob conditions as it is difficult to define conditions that would describe CronJob behaviour in useful manner.
360+
In case the user is interested if there are any running Jobs, `.status.active` field should be sufficient.
281361

282362
### Risks and Mitigations
283363

@@ -292,14 +372,20 @@ How will UX be reviewed, and by whom?
292372
293373
Consider including folks who also work outside the SIG or subproject.
294374
-->
295-
We are proposing a new field called `.spec.progressDeadlineSeconds` to both DaemonSet and StatefulSet whose default value will be set to the max value of int32 (i.e. 2147483647) by default, which means "no deadline". In this mode, DaemonSet and StatefulSet controllers will behave exactly like its current behavior but with no `Failed` mode as they're either `Progressing` or `Complete`. This is to ensure backward compatibility with current behavior. We will default to reasonable values for both the controllers in a future release. Since a DaemonSet can make progress no faster than "healthy but not ready nodes", the default value for `progressDeadlineSeconds` can be set to 30 minutes but this value can vary depending on the node count in the cluster. The value for StatefulSet can be longer than 10 minutes since it also involves provisioning storage and binding. This default value can be set to 15 minutes in case of StatefulSet.
375+
We are proposing a new field called `.spec.progressDeadlineSeconds` to DaemonSet and StatefulSet whose default value will be set to the max value of int32 (i.e. 2147483647) by default, which means "no deadline".
376+
In this mode, DaemonSet and StatefulSet controllers will behave exactly like its current behavior but with no `Failed` mode as they're either `Progressing` or `Complete`.
377+
This is to ensure backward compatibility with current behavior. We will default to reasonable values for the controllers in a future release.
378+
Since a DaemonSet can make progress no faster than "healthy but not ready nodes", the default value for `progressDeadlineSeconds` can be set to 30 minutes but this value can vary depending on the node count in the cluster.
379+
The value for StatefulSet can be longer than 10 minutes since it also involves provisioning storage and binding. This default value can be set to 15 minutes in case of StatefulSet.
296380

297381
It is possible that we introduce a bug in the implementation. The bug can cause:
298-
- DaemonSet and StatefulSet controllers can be marked `Failed` eventhough rollout is progress
382+
- DaemonSet and StatefulSet controllers can be marked `Failed` even though rollout is in progress
299383
- The states could be misrepresented, for example a DaemonSet or StatefulSet can be marked `Progressing` when actually it is complete
300384

301-
The mitigation currently is that these features will be in Alpha stage behind featuregates for people to try out and give feedback. In Beta phase when
302-
these are enabled by default, people will only see issues or bugs when `progressDeadlineSeconds` is set to something greater than 0 and we choose default values for that field. Since people would have tried this feature in Alpha, we would have had time to fix issues. The featuregates we use are `DaemonSetConditions` for DaemonSet controller and `StatefulSetConditions` for StatefulSet controller.
385+
The mitigation currently is that these features will be in Alpha stage behind featuregates for people to try out and give feedback. In Beta phase when
386+
these are enabled by default, people will only see issues or bugs when `progressDeadlineSeconds` is set to something greater than 0 and we choose default values for that field.
387+
Since people would have tried this feature in Alpha, we would have had time to fix issues.
388+
The featuregates we use are `DaemonSetConditions` for DaemonSet controller, `StatefulSetConditions` for StatefulSet controller.
303389

304390

305391

@@ -454,7 +540,7 @@ enhancement:
454540
CRI or CNI may require updating that component before the kubelet.
455541
-->
456542

457-
TBD
543+
None identified yet.
458544

459545
## Production Readiness Review Questionnaire
460546

@@ -523,7 +609,7 @@ NOTE: Also set `disable-supported` to `true` or `false` in `kep.yaml`.
523609
The DaemonSet, StatefulSet controller starts respecting the `progressDeadlineSeconds` again.
524610

525611
###### Are there any tests for feature enablement/disablement?
526-
612+
527613
Yes, unit, integration and e2e tests for feature enabled, disabled
528614

529615
<!--
@@ -679,8 +765,8 @@ previous answers based on experience in the field.
679765

680766
###### Will enabling / using this feature result in any new API calls?
681767
Yes
682-
- Update of DaemonSet, StatefulSet status
683-
- Controllers could make additional update calls when syncing the resources
768+
- Update of DaemonSet, Job, ReplicaSet, ReplicationController, StatefulSet status
769+
- Controllers could potentially make additional update calls when syncing the resources
684770
<!--
685771
Describe them, providing:
686772
- API call type (e.g. PATCH pods)
@@ -715,10 +801,10 @@ Describe them, providing:
715801

716802
###### Will enabling / using this feature result in increasing size or count of the existing API objects?
717803
Yes.
718-
API type(s): DaemonSet, StatefulSet
719-
- Estimated increase in size:
720-
- New field in DaemonSet, StatefulSet spec about 4 bytes
721-
- Add new condition in DaemonSet, StatefulSet status
804+
API type(s): DaemonSet, Deployment, Job, ReplicaSet, ReplicationController, StatefulSet
805+
- Estimated increase in size:
806+
- New field in DaemonSet and StatefulSet spec about 4 bytes
807+
- Add new conditions in Deployment, DaemonSet, Job, ReplicaSet, ReplicationController, StatefulSet status
722808
<!--
723809
Describe them, providing:
724810
- API type(s):
@@ -728,7 +814,7 @@ Describe them, providing:
728814

729815
###### Will enabling / using this feature result in increasing time taken by any operations covered by existing SLIs/SLOs?
730816

731-
No.
817+
TBD.
732818
<!--
733819
Look at the [existing SLIs/SLOs].
734820
@@ -794,14 +880,14 @@ Major milestones might include:
794880
-->
795881

796882
## Drawbacks
797-
Adds more complexity to DaemonSet, StatefulSet controllers in terms of checking conditions and updating the conditions continuously.
883+
Adds more complexity to Deployment, DaemonSet, Job, ReplicaSet, ReplicationController, StatefulSet controllers in terms of checking conditions and updating the conditions continuously.
798884

799885
<!--
800886
Why should this KEP _not_ be implemented?
801887
-->
802888

803889
## Alternatives
804-
Continue to check AvailableReplicas, Replicas and other fields instead of having explict conditions. This is not always fool proof and can cause problems.
890+
Continue to check AvailableReplicas, Replicas and other fields instead of having explicit conditions. This is not always foolproof and can cause problems.
805891
<!--
806892
What other approaches did you consider, and why did you rule them out? These do
807893
not need to be as detailed as the proposal, but should include enough

0 commit comments

Comments
 (0)