|
| 1 | +--- |
| 2 | +layout: blog |
| 3 | +title: "Kubernetes v1.36: In-Place Vertical Scaling for Pod-Level Resources Graduates to Beta" |
| 4 | +draft: true |
| 5 | +slug: kubernetes-v1-36-inplace-pod-level-resources-beta |
| 6 | +author: Narang Dixita Sohanlal (Google) |
| 7 | +--- |
| 8 | + |
| 9 | +Following the graduation of Pod-Level Resources to Beta in v1.34 and the General Availability (GA) of In-Place Pod Vertical Scaling in v1.35, the Kubernetes community is thrilled to announce that **In-Place Pod-Level Resources Vertical Scaling has graduated to Beta in v1.36!** |
| 10 | + |
| 11 | +This feature is now enabled by default via the `InPlacePodLevelResourcesVerticalScaling` feature gate. It allows users to update the aggregate Pod resource budget (`.spec.resources`) for a running Pod, **often without requiring a container restart.** |
| 12 | + |
| 13 | +## Why Pod-level in-place resize? |
| 14 | + |
| 15 | +The Pod-level resource model simplified management for complex Pods (such as those with sidecars) by allowing containers to share a collective pool of resources. In v1.36, you can now adjust this aggregate boundary on-the-fly. |
| 16 | + |
| 17 | +This is particularly useful for Pods where containers do not have individual limits defined. These containers automatically scale their effective boundaries to fit the newly resized Pod-level dimensions, allowing you to expand the shared pool during peak demand without manual per-container recalculations. |
| 18 | + |
| 19 | +## Resource inheritance and the `resizePolicy` |
| 20 | + |
| 21 | +When a Pod-level resize is initiated, the Kubelet treats the change as a resize event for every container that inherits its limits from the Pod-level budget. To determine whether a restart is required, the Kubelet consults the `resizePolicy` defined within individual containers: |
| 22 | + |
| 23 | +* **Non-disruptive Updates:** If a container's `restartPolicy` is set to `NotRequired`, the Kubelet attempts to update the cgroup limits dynamically via the Container Runtime Interface (CRI). |
| 24 | +* **Disruptive Updates:** If set to `RestartContainer`, the container will be restarted to apply the new aggregate boundary safely. |
| 25 | + |
| 26 | +> **Note:** Currently, `resizePolicy` is not supported at the Pod level. The Kubelet always defers to individual container settings to decide if an update can be applied in-place or requires a restart. |
| 27 | +
|
| 28 | +## Example: ccaling a shared resource pool |
| 29 | + |
| 30 | +In this scenario, a Pod is defined with a 2 CPU pod-level limit. Because the individual containers do not have their own limits defined, they share this total pool. |
| 31 | + |
| 32 | +### 1. Initial Pod specification |
| 33 | + |
| 34 | +```yaml |
| 35 | +apiVersion: v1 |
| 36 | +kind: Pod |
| 37 | +metadata: |
| 38 | + name: shared-pool-app |
| 39 | +spec: |
| 40 | + resources: # Pod-level limits |
| 41 | + limits: |
| 42 | + cpu: "2" |
| 43 | + memory: "4Gi" |
| 44 | + containers: |
| 45 | + - name: main-app |
| 46 | + image: my-app:v1 |
| 47 | + resizePolicy: [{resourceName: "cpu", restartPolicy: "NotRequired"}] |
| 48 | + - name: sidecar |
| 49 | + image: logger:v1 |
| 50 | + resizePolicy: [{resourceName: "cpu", restartPolicy: "NotRequired"}] |
| 51 | +``` |
| 52 | +
|
| 53 | +### 2. The resize operation |
| 54 | +
|
| 55 | +To double the CPU capacity to 4 CPUs, apply a patch using the `resize` subresource: |
| 56 | + |
| 57 | +```bash |
| 58 | +kubectl patch pod shared-pool-app --subresource resize --patch \ |
| 59 | + '{"spec":{"resources":{"limits":{"cpu":"4"}}}}' |
| 60 | +``` |
| 61 | + |
| 62 | +## Node-Level reality: feasibility and safety |
| 63 | + |
| 64 | +Applying a resize patch is only the first step. The Kubelet performs several checks and follows a specific sequence to ensure node stability: |
| 65 | + |
| 66 | +### 1. The feasibility check |
| 67 | + |
| 68 | +Before admitting a resize, the Kubelet verifies if the new aggregate request fits within the Node's allocatable capacity. If the Node is overcommitted, the resize is not ignored; instead, the `PodResizePending` condition will reflect a `Deferred` or `Infeasible` status, providing immediate feedback on why the "envelope" hasn't grown. |
| 69 | + |
| 70 | +### 2. Update sequencing |
| 71 | + |
| 72 | +To prevent resource "overshoot," the Kubelet coordinates the cgroup updates in a specific order: |
| 73 | +* **When Increasing:** The Pod-level cgroup is expanded first, creating the "room" before the individual container cgroups are enlarged. |
| 74 | +* **When Decreasing:** The container cgroups are throttled first, and only then is the aggregate Pod-level cgroup shrunken. |
| 75 | + |
| 76 | +## Observability: tracking resize status |
| 77 | + |
| 78 | +With the move to Beta, Kubernetes uses **Pod Conditions** to track the lifecycle of a resize: |
| 79 | + |
| 80 | +* **`PodResizePending`**: The spec is updated, but the Node hasn't admitted the change (e.g., due to capacity). |
| 81 | +* **`PodResizeInProgress`**: The Node has admitted the resize (`status.allocatedResources`) but the changes aren't yet fully applied to the cgroups (`status.resources`). |
| 82 | + |
| 83 | +```yaml |
| 84 | +status: |
| 85 | + allocatedResources: |
| 86 | + cpu: "4" |
| 87 | + resources: |
| 88 | + limits: |
| 89 | + cpu: "4" |
| 90 | + conditions: |
| 91 | + - type: PodResizeInProgress |
| 92 | + status: "True" |
| 93 | +``` |
| 94 | + |
| 95 | +## Constraints and requirements |
| 96 | + |
| 97 | +* **cgroup v2 Only:** Required for accurate aggregate enforcement. |
| 98 | +* **CRI Support:** Requires a container runtime that supports the `UpdateContainerResources` CRI call (e.g., containerd v2.0+ or CRI-O). |
| 99 | +* **Feature Gates:** Requires `PodLevelResources`, `InPlacePodVerticalScaling`, `InPlacePodLevelResourcesVerticalScaling`, and `NodeDeclaredFeatures`. |
| 100 | +* **Linux Only:** Currently exclusive to Linux-based nodes. |
| 101 | + |
| 102 | +## What's next? |
| 103 | + |
| 104 | +As we move toward General Availability (GA), the community is focusing on **Vertical Pod Autoscaler (VPA) Integration**, enabling VPA to issue Pod-level resource recommendations and trigger in-place actuation automatically. |
| 105 | + |
| 106 | +## Getting started and providing feedback |
| 107 | + |
| 108 | +We encourage you to test this feature and provide feedback via the standard Kubernetes communication channels: |
| 109 | + |
| 110 | +* Slack: [#sig-node](https://kubernetes.slack.com/messages/sig-node) |
| 111 | +* [Mailing list](https://groups.google.com/forum/#!forum/kubernetes-sig-node) |
| 112 | +* [Open Community Issues/PRs](https://github.com/kubernetes/community/labels/sig%2Fnode) |
0 commit comments