|
| 1 | +--- |
| 2 | +layout: blog |
| 3 | +title: "Kubernetes v1.34: Mutable CSI Node Allocatable Graduates to Beta" |
| 4 | +date: 2025-XX-XX |
| 5 | +slug: kubernetes-v1-34-mutable-csi-node-allocatable-count |
| 6 | +draft: true |
| 7 | +author: Eddie Torres (Amazon Web Services) |
| 8 | +--- |
| 9 | + |
| 10 | +The [functionality for CSI drivers to update information about attachable volume count on the nodes](https://kep.k8s.io/4876), first introduced as Alpha in Kubernetes v1.33, has graduated to **Beta** in the Kubernetes v1.34 release! This marks a significant milestone in enhancing the accuracy of stateful pod scheduling by reducing failures due to outdated attachable volume capacity information. |
| 11 | + |
| 12 | +## Background |
| 13 | + |
| 14 | +Traditionally, Kubernetes [CSI drivers](https://kubernetes-csi.github.io/docs/introduction.html) report a static maximum volume attachment limit when initializing. However, actual attachment capacities can change during a node's lifecycle for various reasons, such as: |
| 15 | + |
| 16 | +- Manual or external operations attaching/detaching volumes outside of Kubernetes control. |
| 17 | +- Dynamically attached network interfaces or specialized hardware (GPUs, NICs, etc.) consuming available slots. |
| 18 | +- Multi-driver scenarios, where one CSI driver’s operations affect available capacity reported by another. |
| 19 | + |
| 20 | +Static reporting can cause Kubernetes to schedule pods onto nodes that appear to have capacity but don't, leading to pods stuck in a `ContainerCreating` state. |
| 21 | + |
| 22 | +## Dynamically adapting CSI volume limits |
| 23 | + |
| 24 | +With this new feature, Kubernetes enables CSI drivers to dynamically adjust and report node attachment capacities at runtime. This ensures that the scheduler, as well as other components relying on this information, have the most accurate, up-to-date view of node capacity. |
| 25 | + |
| 26 | +### How it works |
| 27 | + |
| 28 | +Kubernetes supports two mechanisms for updating the reported node volume limits: |
| 29 | + |
| 30 | +- **Periodic Updates:** CSI drivers specify an interval to periodically refresh the node's allocatable capacity. |
| 31 | +- **Reactive Updates:** An immediate update triggered when a volume attachment fails due to exhausted resources (`ResourceExhausted` error). |
| 32 | + |
| 33 | +### Enabling the feature |
| 34 | + |
| 35 | +To use this beta feature, the `MutableCSINodeAllocatableCount` feature gate must be enabled in these components: |
| 36 | + |
| 37 | +- `kube-apiserver` |
| 38 | +- `kubelet` |
| 39 | + |
| 40 | +### Example CSI driver configuration |
| 41 | + |
| 42 | +Below is an example of configuring a CSI driver to enable periodic updates every 60 seconds: |
| 43 | + |
| 44 | +``` |
| 45 | +apiVersion: storage.k8s.io/v1 |
| 46 | +kind: CSIDriver |
| 47 | +metadata: |
| 48 | + name: example.csi.k8s.io |
| 49 | +spec: |
| 50 | + nodeAllocatableUpdatePeriodSeconds: 60 |
| 51 | +``` |
| 52 | + |
| 53 | +This configuration directs kubelet to periodically call the CSI driver's `NodeGetInfo` method every 60 seconds, updating the node’s allocatable volume count. Kubernetes enforces a minimum update interval of 10 seconds to balance accuracy and resource usage. |
| 54 | + |
| 55 | +### Immediate updates on attachment failures |
| 56 | + |
| 57 | +When a volume attachment operation fails due to a `ResourceExhausted` error (gRPC code `8`), Kubernetes immediately updates the allocatable count instead of waiting for the next periodic update. The Kubelet then marks the affected pods as Failed, enabling their controllers to recreate them. This prevents pods from getting permanently stuck in the `ContainerCreating` state. |
| 58 | + |
| 59 | +## Getting started |
| 60 | + |
| 61 | +To enable this feature in your Kubernetes v1.34 cluster: |
| 62 | + |
| 63 | +1. Enable the feature gate `MutableCSINodeAllocatableCount` on the `kube-apiserver` and `kubelet` components. |
| 64 | +2. Update your CSI driver configuration by setting `nodeAllocatableUpdatePeriodSeconds`. |
| 65 | +3. Monitor and observe improvements in scheduling accuracy and pod placement reliability. |
| 66 | + |
| 67 | +## Next steps |
| 68 | + |
| 69 | +This feature is currently in beta and the Kubernetes community welcomes your feedback. Test it, share your experiences, and help guide its evolution to GA stability. |
| 70 | + |
| 71 | +Join discussions in the [Kubernetes Storage Special Interest Group (SIG-Storage)](https://github.com/kubernetes/community/tree/master/sig-storage) to shape the future of Kubernetes storage capabilities. |
0 commit comments