KEP-1287: PreferNoRestart

Author: tallclair

keps/sig-node/1287-in-place-update-pod-resources/README.md

@@ -274,25 +274,25 @@ the `/resize` subresource:
 To provide fine-grained user control, PodSpec.Containers is extended with
 ResizeRestartPolicy - a list of named subobjects (new object) that supports
 'cpu' and 'memory' as names. It supports the following restart policy values:
-* NotRequired - default value; resize the Container without restart, if possible.
-* RestartContainer - the container requires a restart to apply new resource values.
+* `PreferNoRestart` - default value; resize the Container without restart, if possible.
+  * `NotRequired` - Equivalent to `PreferNoRestart`, deprecated with v1.33.
+* `RestartContainer` - the container requires a restart to apply new resource values.
   (e.g.  Java process needs to change its Xmx flag) By using ResizePolicy, user
   can mark Containers as safe (or unsafe) for in-place resource update. Kubelet
   uses it to determine the required action.
 
-Note: `NotRequired` restart policy for resize does not *guarantee* that a container
-won't be restarted. The runtime may choose to stop the container if it is unable to
-apply the new resources without restarts.
+Note: `PreferNoRestart` restart policy for resize does not *guarantee* that a container
+won't be restarted.
 
 Setting the flag to separately control CPU & memory is due to an observation
 that usually CPU can be added/removed without much problem whereas changes to
 available memory are more probable to require restarts.
 
 If more than one resource type with different policies are updated at the same
-time, then `RestartContainer` policy takes precedence over `NotRequired` policy.
+time, then `RestartContainer` policy takes precedence over `PreferNoRestart` policy.
 
 If a pod's RestartPolicy is `Never`, the ResizePolicy fields must be set to
-`NotRequired` to pass validation.  That said, any in-place resize may result
+`PreferNoRestart` to pass validation.  That said, any in-place resize may result
 in the container being stopped *and not restarted*, if the system can not
 perform the resize in place.
 
@@ -340,6 +340,12 @@ As of Kubernetes v1.20, the CRI has included support for in-place resizing of co
 `ContainerStatus` message includes a `ContainerResources` field, which reports the current resource
 configuration of the container.
 
+Starting with Kubernetes v1.33, the contract on the `UpdateContainerResources` call will be updated
+to specify that runtimes should not deliberately restart the container to adjust the resources. If a
+restart is required to resize, the runtime should return an error instead. There may be edge-cases
+where a restart can still be triggered (see [Memory Limit Decreases](#memory-limit-decreases)), so
+this is a best-effort requirement. There is no enforcement of this behavior.
+
 Even though pod-level cgroups are currently managed by the Kubelet, runtimes may rely need to be
 notified when the resource configuration changes. For example, this information should be passed
 through to NRI plugins. To this end, we will add a new `UpdatePodSandboxResources` API:
@@ -470,7 +476,7 @@ the larger of the Pod's `Spec...Resources.Requests` and
 ### Flow Control
 
 The following steps denote the flow of a series of in-place resize operations
-for a Pod with ResizePolicy set to NotRequired for all its Containers.
+for a Pod with ResizePolicy set to PreferNoRestart for all its Containers.
 This is intentionally hitting various edge-cases to demonstrate.
 
 ```
@@ -1039,15 +1045,15 @@ Setup a namespace with min and max LimitRange and create a single, valid Pod.
 #### Resize Policy Tests
 
 Setup a guaranteed class Pod with two containers (c1 & c2).
-1. No resize policy specified, defaults to NotRequired. Verify that CPU and
+1. No resize policy specified, defaults to PreferNoRestart. Verify that CPU and
    memory are resized without restarting containers.
-1. NotRequired (cpu, memory) policy for c1, RestartContainer (cpu, memory) for c2.
+1. PreferNoRestart (cpu, memory) policy for c1, RestartContainer (cpu, memory) for c2.
    Verify that c1 is resized without restart, c2 is restarted on resize.
-1. NotRequired cpu, RestartContainer memory policy for c1. Resize c1 CPU only,
+1. PreferNoRestart cpu, RestartContainer memory policy for c1. Resize c1 CPU only,
    verify container is resized without restart.
-1. NotRequired cpu, RestartContainer memory policy for c1. Resize c1 memory only,
+1. PreferNoRestart cpu, RestartContainer memory policy for c1. Resize c1 memory only,
    verify container is resized with restart.
-1. NotRequired cpu, RestartContainer memory policy for c1. Resize c1 CPU & memory,
+1. PreferNoRestart cpu, RestartContainer memory policy for c1. Resize c1 CPU & memory,
    verify container is resized with restart.
 
 #### Backward Compatibility and Negative Tests