|
| 1 | +--- |
| 2 | +layout: blog |
| 3 | +title: 'Kubernetes v1.26: CPUManager goes GA' |
| 4 | +date: 2022-12-27 |
| 5 | +slug: cpumanager-ga |
| 6 | +--- |
| 7 | + |
| 8 | +**Author:** |
| 9 | +Francesco Romani (Red Hat) |
| 10 | + |
| 11 | +The CPU Manager is a part of the kubelet, the Kubernetes node agent, which enables the user to allocate exclusive CPUs to containers. |
| 12 | +Since Kubernetes v1.10, where it [graduated to Beta](/blog/2018/07/24/feature-highlight-cpu-manager/), the CPU Manager proved itself reliable and |
| 13 | +fulfilled its role of allocating exclusive CPUs to containers, so adoption has steadily grown making it a staple component of performance-critical |
| 14 | +and low-latency setups. Over time, most changes were about bugfixes or internal refactoring, with the following noteworthy user-visible changes: |
| 15 | + |
| 16 | +- [support explicit reservation of CPUs](https://github.com/Kubernetes/Kubernetes/pull/83592): it was already possible to request to reserve a given |
| 17 | + number of CPUs for system resources, including the kubelet itself, which will not be used for exclusive CPU allocation. Now it is possible to also |
| 18 | + explicitly select which CPUs to reserve instead of letting the kubelet pick them up automatically. |
| 19 | +- [report the exclusively allocated CPUs](https://github.com/Kubernetes/Kubernetes/pull/97415) to containers, much like is already done for devices, |
| 20 | + using the kubelet-local [PodResources API](/docs/concepts/extend-Kubernetes/compute-storage-net/device-plugins/#monitoring-device-plugin-resources). |
| 21 | +- [optimize the usage of system resources](https://github.com/Kubernetes/Kubernetes/pull/101771), eliminating unnecessary sysfs changes. |
| 22 | + |
| 23 | +The CPU Manager reached the point on which it "just works", so in Kubernetes v1.26 it has graduated to generally available (GA). |
| 24 | + |
| 25 | +## Customization options for CPU Manager {#cpu-managed-customization} |
| 26 | + |
| 27 | +The CPU Manager supports two operation modes, configured using its _policies_. With the `none` policy, the CPU Manager allocates CPUs to containers |
| 28 | +without any specific constraint except the (optional) quota set in the Pod spec. |
| 29 | +With the `static` policy, then provided that the pod is in the Guaranteed QoS class and every container in that Pod requests an integer amount of vCPU cores, |
| 30 | +then the CPU Manager allocates CPUs exclusively. Exclusive assignment means that other containers (whether from the same Pod, or from a different Pod) do not |
| 31 | +get scheduled onto that CPU. |
| 32 | + |
| 33 | +This simple operational model served the user base pretty well, but as the CPU Manager matured more and more, users started to look at more elaborate use |
| 34 | +cases and how to better support them. |
| 35 | + |
| 36 | +Rather than add more policies, the community realized that pretty much all the novel use cases are some variation of the behavior enabled by the `static` |
| 37 | +CPU Manager policy. Hence, it was decided to add [options to tune the behavior of the static policy](https://github.com/Kubernetes/enhancements/tree/master/keps/sig-node/2625-cpumanager-policies-thread-placement#proposed-change). |
| 38 | +The options have a varying degree of maturity, like any other Kubernetes feature, and in order to be accepted, each new option provides a backward |
| 39 | +compatible behavior when disabled, and to document how to interact with each other, should they interact at all. |
| 40 | + |
| 41 | +This enabled the Kubernetes project to graduate to GA the CPU Manager core component and core CPU allocation algorithms to GA, |
| 42 | +while also enabling a new age of experimentation in this area. |
| 43 | +In Kubernetes v1.26, the CPU Manager supports [three different policy options](/docs/tasks/administer-cluster/cpu-management-policies.md#static-policy-options): |
| 44 | + |
| 45 | +`full-pcpus-only` |
| 46 | +: restrict the CPU Manager core allocation algorithm to full physical cores only, reducing noisy neighbor issues from hardware technologies that allow sharing cores. |
| 47 | + |
| 48 | +`distribute-cpus-across-numa` |
| 49 | +: drive the CPU Manager to evenly distribute CPUs across NUMA nodes, for cases where more than one NUMA node is required to satisfy the allocation. |
| 50 | + |
| 51 | +`align-by-socket` |
| 52 | +: change how the CPU Manager allocates CPUs to a container: consider CPUs to be aligned at the socket boundary, instead of NUMA node boundary. |
| 53 | + |
| 54 | +## Further development |
| 55 | + |
| 56 | +After graduating the main CPU Manager feature, each existing policy option will follow their graduation process, independent from CPU Manager and from each other option. |
| 57 | +There is room for new options to be added, but there's also a growing demand for even more flexibility than what the CPU Manager, and its policy options, currently grant. |
| 58 | + |
| 59 | +Conversations are in progress in the community about splitting the CPU Manager and the other resource managers currently part of the kubelet executable |
| 60 | +into pluggable, independent kubelet plugins. If you are interested in this effort, please join the conversation on SIG Node communication channels (Slack, mailing list, weekly meeting). |
| 61 | + |
| 62 | +## Further reading |
| 63 | + |
| 64 | +Please check out the [Control CPU Management Policies on the Node](/docs/tasks/administer-cluster/cpu-management-policies/) |
| 65 | +task page to learn more about the CPU Manager, and how it fits in relation to the other node-level resource managers. |
| 66 | + |
| 67 | +## Getting involved |
| 68 | + |
| 69 | +This feature is driven by the [SIG Node](https://github.com/Kubernetes/community/blob/master/sig-node/README.md) community. |
| 70 | +Please join us to connect with the community and share your ideas and feedback around the above feature and |
| 71 | +beyond. We look forward to hearing from you! |
0 commit comments