Skip to content

Commit 2a5045a

Browse files
authored
Merge pull request #37435 from fromanirh/cpumanager-ga-blog-post
blog: node: cpumanager goes GA
2 parents b5666b3 + a39876c commit 2a5045a

File tree

1 file changed

+71
-0
lines changed

1 file changed

+71
-0
lines changed
Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
---
2+
layout: blog
3+
title: 'Kubernetes v1.26: CPUManager goes GA'
4+
date: 2022-12-27
5+
slug: cpumanager-ga
6+
---
7+
8+
**Author:**
9+
Francesco Romani (Red Hat)
10+
11+
The CPU Manager is a part of the kubelet, the Kubernetes node agent, which enables the user to allocate exclusive CPUs to containers.
12+
Since Kubernetes v1.10, where it [graduated to Beta](/blog/2018/07/24/feature-highlight-cpu-manager/), the CPU Manager proved itself reliable and
13+
fulfilled its role of allocating exclusive CPUs to containers, so adoption has steadily grown making it a staple component of performance-critical
14+
and low-latency setups. Over time, most changes were about bugfixes or internal refactoring, with the following noteworthy user-visible changes:
15+
16+
- [support explicit reservation of CPUs](https://github.com/Kubernetes/Kubernetes/pull/83592): it was already possible to request to reserve a given
17+
number of CPUs for system resources, including the kubelet itself, which will not be used for exclusive CPU allocation. Now it is possible to also
18+
explicitly select which CPUs to reserve instead of letting the kubelet pick them up automatically.
19+
- [report the exclusively allocated CPUs](https://github.com/Kubernetes/Kubernetes/pull/97415) to containers, much like is already done for devices,
20+
using the kubelet-local [PodResources API](/docs/concepts/extend-Kubernetes/compute-storage-net/device-plugins/#monitoring-device-plugin-resources).
21+
- [optimize the usage of system resources](https://github.com/Kubernetes/Kubernetes/pull/101771), eliminating unnecessary sysfs changes.
22+
23+
The CPU Manager reached the point on which it "just works", so in Kubernetes v1.26 it has graduated to generally available (GA).
24+
25+
## Customization options for CPU Manager {#cpu-managed-customization}
26+
27+
The CPU Manager supports two operation modes, configured using its _policies_. With the `none` policy, the CPU Manager allocates CPUs to containers
28+
without any specific constraint except the (optional) quota set in the Pod spec.
29+
With the `static` policy, then provided that the pod is in the Guaranteed QoS class and every container in that Pod requests an integer amount of vCPU cores,
30+
then the CPU Manager allocates CPUs exclusively. Exclusive assignment means that other containers (whether from the same Pod, or from a different Pod) do not
31+
get scheduled onto that CPU.
32+
33+
This simple operational model served the user base pretty well, but as the CPU Manager matured more and more, users started to look at more elaborate use
34+
cases and how to better support them.
35+
36+
Rather than add more policies, the community realized that pretty much all the novel use cases are some variation of the behavior enabled by the `static`
37+
CPU Manager policy. Hence, it was decided to add [options to tune the behavior of the static policy](https://github.com/Kubernetes/enhancements/tree/master/keps/sig-node/2625-cpumanager-policies-thread-placement#proposed-change).
38+
The options have a varying degree of maturity, like any other Kubernetes feature, and in order to be accepted, each new option provides a backward
39+
compatible behavior when disabled, and to document how to interact with each other, should they interact at all.
40+
41+
This enabled the Kubernetes project to graduate to GA the CPU Manager core component and core CPU allocation algorithms to GA,
42+
while also enabling a new age of experimentation in this area.
43+
In Kubernetes v1.26, the CPU Manager supports [three different policy options](/docs/tasks/administer-cluster/cpu-management-policies.md#static-policy-options):
44+
45+
`full-pcpus-only`
46+
: restrict the CPU Manager core allocation algorithm to full physical cores only, reducing noisy neighbor issues from hardware technologies that allow sharing cores.
47+
48+
`distribute-cpus-across-numa`
49+
: drive the CPU Manager to evenly distribute CPUs across NUMA nodes, for cases where more than one NUMA node is required to satisfy the allocation.
50+
51+
`align-by-socket`
52+
: change how the CPU Manager allocates CPUs to a container: consider CPUs to be aligned at the socket boundary, instead of NUMA node boundary.
53+
54+
## Further development
55+
56+
After graduating the main CPU Manager feature, each existing policy option will follow their graduation process, independent from CPU Manager and from each other option.
57+
There is room for new options to be added, but there's also a growing demand for even more flexibility than what the CPU Manager, and its policy options, currently grant.
58+
59+
Conversations are in progress in the community about splitting the CPU Manager and the other resource managers currently part of the kubelet executable
60+
into pluggable, independent kubelet plugins. If you are interested in this effort, please join the conversation on SIG Node communication channels (Slack, mailing list, weekly meeting).
61+
62+
## Further reading
63+
64+
Please check out the [Control CPU Management Policies on the Node](/docs/tasks/administer-cluster/cpu-management-policies/)
65+
task page to learn more about the CPU Manager, and how it fits in relation to the other node-level resource managers.
66+
67+
## Getting involved
68+
69+
This feature is driven by the [SIG Node](https://github.com/Kubernetes/community/blob/master/sig-node/README.md) community.
70+
Please join us to connect with the community and share your ideas and feedback around the above feature and
71+
beyond. We look forward to hearing from you!

0 commit comments

Comments
 (0)