@@ -6,12 +6,12 @@ weight: 20
6
6
7
7
{{% capture overview %}}
8
8
9
- {{< feature-state for_k8s_version="v1.16 " state="alpha " >}}
9
+ {{< feature-state for_k8s_version="v1.18 " state="beta " >}}
10
10
11
11
<!--
12
12
When you run a Pod on a Node, the Pod itself takes an amount of system resources. These
13
13
resources are additional to the resources needed to run the container(s) inside the Pod.
14
- _Pod Overhead_ is a feature for accounting for the resources consumed by the pod infrastructure
14
+ _Pod Overhead_ is a feature for accounting for the resources consumed by the Pod infrastructure
15
15
on top of the container requests & limits.
16
16
-->
17
17
@@ -30,63 +30,270 @@ _POD 开销_ 是一个特性,用于计算 Pod 基础设施在容器请求和
30
30
## Pod 开销
31
31
32
32
<!--
33
- In Kubernetes, the pod 's overhead is set at
33
+ In Kubernetes, the Pod 's overhead is set at
34
34
[admission](/docs/reference/access-authn-authz/extensible-admission-controllers/#what-are-admission-webhooks)
35
- time according to the overhead associated with the pod 's
35
+ time according to the overhead associated with the Pod 's
36
36
[RuntimeClass](/docs/concepts/containers/runtime-class/).
37
37
-->
38
38
39
- 在 Kubernetes 中,Pod 的开销是根据与 Pod 的 [ RuntimeClass] ( /docs/concepts/containers/runtime-class/ ) 相关联的开销在[ 准入] ( /docs/reference/access-authn-authz/extensible-admission-controllers/#what-are-admission-webhooks ) 时设置的。
39
+ 在 Kubernetes 中,Pod 的开销是根据与 Pod 的 [ RuntimeClass] ( /docs/concepts/containers/runtime-class/ ) 相关联的开销在[ 准入] ( /docs/reference/access-authn-authz/extensible-admission-controllers/#what-are-admission-webhooks ) 时设置的。
40
40
41
41
<!--
42
42
When Pod Overhead is enabled, the overhead is considered in addition to the sum of container
43
- resource requests when scheduling a pod . Similarly, Kubelet will include the pod overhead when sizing
44
- the pod cgroup, and when carrying out pod eviction ranking.
43
+ resource requests when scheduling a Pod . Similarly, Kubelet will include the Pod overhead when sizing
44
+ the Pod cgroup, and when carrying out Pod eviction ranking.
45
45
-->
46
- 当启用 Pod 开销时,在调度 Pod 时,除了考虑容器资源请求的总和外,还要考虑 Pod 开销。类似地,Kubelet 将在确定 pod cgroup 的大小和执行 Pod 驱逐排序时包含 Pod 开销。
46
+ 当启用 Pod 开销时,在调度 Pod 时,除了考虑容器资源请求的总和外,还要考虑 Pod 开销。类似地,Kubelet 将在确定 Pod cgroup 的大小和执行 Pod 驱逐排序时包含 Pod 开销。
47
47
48
48
<!--
49
- ### Set Up
49
+ ## Enabling Pod Overhead {#set-up}
50
50
-->
51
- ### 设置
51
+ ## 启用 Pod 开销 {#set-up}
52
52
53
53
<!--
54
54
You need to make sure that the `PodOverhead`
55
- [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) is enabled (it is off by default)
56
- across your cluster. This means:
55
+ [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) is enabled (it is on by default as of 1.18 )
56
+ across your cluster, and a `RuntimeClass` is utilized which defines the `overhead` field.
57
57
-->
58
- 您需要确保在集群中启用了 ` PodOverhead ` [ 特性门] ( /docs/reference/command-line-tools-reference/feature-gates/ ) (默认情况下是关闭的)。这意味着:
58
+ 您需要确保在集群中启用了 ` PodOverhead ` [ 特性门] ( /docs/reference/command-line-tools-reference/feature-gates/ ) (在 1.18 默认是开启的),以及一个用于定义 ` overhead ` 字段的 ` RuntimeClass ` 。
59
+
60
+ <!--
61
+ ## Usage example
62
+ -->
63
+ ## 使用示例
59
64
60
65
<!--
61
- - in {{< glossary_tooltip text="kube-scheduler" term_id="kube-scheduler" >}}
62
- - in {{< glossary_tooltip text="kube-apiserver" term_id="kube-apiserver" >}}
63
- - in the {{< glossary_tooltip text="kubelet" term_id="kubelet" >}} on each Node
64
- - in any custom API servers that use feature gates
66
+ To use the PodOverhead feature, you need a RuntimeClass that defines the `overhead` field. As
67
+ an example, you could use the following RuntimeClass definition with a virtualizing container runtime
68
+ that uses around 120MiB per Pod for the virtual machine and the guest OS:
69
+ -->
70
+ 要使用 PodOverhead 特性,你需要一个定义 ` overhead ` 字段的 RuntimeClass. 作为例子,你可以在虚拟机和来宾操作系统中通过一个虚拟化容器运行时来定义 RuntimeClass 如下,其中每个 Pod 大约使用 120MiB:
71
+
72
+ ``` yaml
73
+ ---
74
+ kind : RuntimeClass
75
+ apiVersion : node.k8s.io/v1beta1
76
+ metadata :
77
+ name : kata-fc
78
+ handler : kata-fc
79
+ overhead :
80
+ podFixed :
81
+ memory : " 120Mi"
82
+ cpu : " 250m"
83
+ ` ` `
84
+
85
+ <!--
86
+ Workloads which are created which specify the ` kata-fc` RuntimeClass handler will take the memory and
87
+ cpu overheads into account for resource quota calculations, node scheduling, as well as Pod cgroup sizing.
88
+
89
+ Consider running the given example workload, test-pod :
65
90
-->
66
- - 在 {{< glossary_tooltip text="kube-scheduler" term_id="kube-scheduler" >}}
67
- - 在 {{< glossary_tooltip text="kube-apiserver" term_id="kube-apiserver" >}}
68
- - 在每一个 Node 的 {{< glossary_tooltip text="kubelet" term_id="kubelet" >}}
69
- - 在任何使用特性门的自定义api服务器中
91
+ 通过指定 `kata-fc` RuntimeClass 处理程序创建的工作负载会将内存和 cpu 开销计入资源配额计算、节点调度以及 Pod cgroup 分级。
70
92
93
+ 假设我们运行下面给出的工作负载示例 test-pod :
94
+
95
+ ` ` ` yaml
96
+ apiVersion: v1
97
+ kind: Pod
98
+ metadata:
99
+ name: test-pod
100
+ spec:
101
+ runtimeClassName: kata-fc
102
+ containers:
103
+ - name: busybox-ctr
104
+ image: busybox
105
+ stdin: true
106
+ tty: true
107
+ resources:
108
+ limits:
109
+ cpu: 500m
110
+ memory: 100Mi
111
+ - name: nginx-ctr
112
+ image: nginx
113
+ resources:
114
+ limits:
115
+ cpu: 1500m
116
+ memory: 100Mi
117
+ ` ` `
118
+
119
+ <!--
120
+ At admission time the RuntimeClass [admission controller](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/)
121
+ updates the workload's PodSpec to include the `overhead` as described in the RuntimeClass. If the PodSpec already has this field defined,
122
+ the Pod will be rejected. In the given example, since only the RuntimeClass name is specified, the admission controller mutates the Pod
123
+ to include an `overhead`.
124
+ -->
125
+ 在准入阶段 RuntimeClass [准入控制器](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/) 更新工作负载的 PodSpec 以包含 RuntimeClass 中定义的 `overhead`. 如果 PodSpec 中该字段已定义,该 Pod 将会被拒绝。在这个例子中,由于只指定了 RuntimeClass 名称,所以准入控制器更新了 Pod, 包含了一个 `overhead`.
126
+
127
+ <!--
128
+ After the RuntimeClass admission controller, you can check the updated PodSpec :
129
+ -->
130
+ 在 RuntimeClass 准入控制器之后,你可以检验一下已更新的 PodSpec :
131
+
132
+ ` ` ` bash
133
+ kubectl get pod test-pod -o jsonpath='{.spec.overhead}'
134
+ ` ` `
135
+
136
+ <!--
137
+ The output is :
138
+ -->
139
+ 输出:
140
+ ```
141
+ map[ cpu:250m memory:120Mi]
142
+ ```
143
+
144
+ <!--
145
+ If a ResourceQuota is defined, the sum of container requests as well as the
146
+ `overhead` field are counted.
147
+ -->
148
+ 如果定义了 ResourceQuata, 则容器请求的总量以及 `overhead` 字段都将计算在内。
149
+
150
+ <!--
151
+ When the kube-scheduler is deciding which node should run a new Pod, the scheduler considers that Pod's
152
+ `overhead` as well as the sum of container requests for that Pod. For this example, the scheduler adds the
153
+ requests and the overhead, then looks for a node that has 2.25 CPU and 320 MiB of memory available.
154
+ -->
155
+ 当 kube-scheduler 决定在哪一个节点调度运行新的 Pod 时,调度器会兼顾该 Pod 的 `overhead` 以及该 Pod 的容器请求总量。在这个示例中,调度器将资源请求和开销相加,然后寻找具备 2.25 CPU 和 320 MiB 内存可用的节点。
156
+
157
+ <!--
158
+ Once a Pod is scheduled to a node, the kubelet on that node creates a new {{< glossary_tooltip text="cgroup" term_id="cgroup" >}}
159
+ for the Pod. It is within this pod that the underlying container runtime will create containers. -->
160
+ 一旦 Pod 调度到了某个节点, 该节点上的 kubelet 将为该 Pod 新建一个 {{< glossary_tooltip text="cgroup" term_id="cgroup" >}}. 底层容器运行时将在这个 pod 中创建容器。
161
+
162
+ <!--
163
+ If the resource has a limit defined for each container (Guaranteed QoS or Bustrable QoS with limits defined),
164
+ the kubelet will set an upper limit for the pod cgroup associated with that resource (cpu.cfs_quota_us for CPU
165
+ and memory.limit_in_bytes memory). This upper limit is based on the sum of the container limits plus the `overhead`
166
+ defined in the PodSpec.
167
+ -->
168
+ 如果该资源对每一个容器都定义了一个限制(定义了受限的 Guaranteed QoS 或者 Bustrable QoS),kubelet 会为与该资源(CPU 的 cpu.cfs_quota_us 以及内存的 memory.limit_in_bytes)相关的 pod cgroup 设定一个上限。该上限基于容器限制总量与 PodSpec 中定义的 `overhead` 之和。
71
169
72
- {{< note >}}
73
170
<!--
74
- Users who can write to RuntimeClass resources are able to have cluster-wide impact on
75
- workload performance. You can limit access to this ability using Kubernetes access controls.
76
- See [Authorization Overview](/docs/reference/access-authn-authz/authorization/) for more details.
171
+ For CPU, if the Pod is Guaranteed or Burstable QoS, the kubelet will set `cpu.shares` based on the sum of container
172
+ requests plus the `overhead` defined in the PodSpec.
77
173
-->
78
- 能够写入运行时类资源的用户能够对工作负载性能产生集群范围的影响。可以使用 Kubernetes 访问控制来限制对此功能的访问。
79
- 有关详细信息,请参见[ 授权概述] ( /docs/reference/access-authn-authz/authorization/ ) 。
80
- {{< /note >}}
174
+ 对于 CPU, 如果 Pod 的 QoS 是 Guaranteed 或者 Burstable, kubelet 会基于容器请求总量与 PodSpec 中定义的 `overhead` 之和设置 `cpu.shares`.
81
175
176
+ <!--
177
+ Looking at our example, verify the container requests for the workload:
178
+ -->
179
+ 请看这个例子,验证工作负载的容器请求:
180
+ ```bash
181
+ kubectl get pod test-pod -o jsonpath='{.spec.containers[*].resources.limits}'
182
+ ```
183
+
184
+ <!--
185
+ The total container requests are 2000m CPU and 200MiB of memory:
186
+ -->
187
+ 容器请求总计 2000m CPU 和 200MiB 内存:
188
+ ```
189
+ map[cpu: 500m memory:100Mi] map[cpu:1500m memory:100Mi]
190
+ ```
191
+
192
+ <!--
193
+ Check this against what is observed by the node:
194
+ -->
195
+ 对照从节点观察到的情况来检查一下:
196
+ ``` bash
197
+ kubectl describe node | grep test-pod -B2
198
+ ```
199
+
200
+ <!--
201
+ The output shows 2250m CPU and 320MiB of memory are requested, which includes PodOverhead:
202
+ -->
203
+ 该输出显示请求了 2250m CPU 以及 320MiB 内存,包含了 PodOverhead 在内:
204
+ ```
205
+ Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
206
+ --------- ---- ------------ ---------- --------------- ------------- ---
207
+ default test-pod 2250m (56%) 2250m (56%) 320Mi (1%) 320Mi (1%) 36m
208
+ ```
209
+
210
+ <!--
211
+ ## Verify Pod cgroup limits
212
+ -->
213
+ ## 验证 Pod cgroup 限制
214
+
215
+ <!--
216
+ Check the Pod's memory cgroups on the node where the workload is running. In the following example, [`crictl`](https://github.com/kubernetes-sigs/cri-tools/blob/master/docs/crictl.md)
217
+ is used on the node, which provides a CLI for CRI-compatible container runtimes. This is an
218
+ advanced example to show PodOverhead behavior, and it is not expected that users should need to check
219
+ cgroups directly on the node.
220
+
221
+ First, on the particular node, determine the Pod identifier:ying
222
+ -->
223
+ 在工作负载所运行的节点上检查 Pod 的内存 cgroups. 在接下来的例子中,将在该节点上使用具备 CRI 兼容的容器运行时命令行工具 [ ` crictl ` ] ( https://github.com/kubernetes-sigs/cri-tools/blob/master/docs/crictl.md ) . 这是一个展示 PodOverhead 行为的进阶示例,用户并不需要直接在该节点上检查 cgroups.
224
+
225
+ <!--
226
+ ```bash
227
+ # Run this on the node where the Pod is scheduled
228
+ -->
229
+ ``` bash
230
+ # 在该 Pod 调度的节点上执行如下命令:
231
+ POD_ID=" $( sudo crictl pods --name test-pod -q) "
232
+ ```
233
+
234
+ <!--
235
+ From this, you can determine the cgroup path for the Pod:
236
+ -->
237
+ 你可以依此判断该 Pod 的 cgroup 路径:
238
+ <!--
239
+ ```bash
240
+ # Run this on the node where the Pod is scheduled
241
+ -->
242
+ ``` bash
243
+ # 在该 Pod 调度的节点上执行如下命令:
244
+ sudo crictl inspectp -o=json $POD_ID | grep cgroupsPath
245
+ ```
246
+
247
+ <!--
248
+ The resulting cgroup path includes the Pod's `pause` container. The Pod level cgroup is one directory above.
249
+ -->
250
+ 执行结果的 cgroup 路径中包含了该 Pod 的 ` pause ` 容器。Pod 级别的 cgroup 即上面的一个目录。
251
+ ```
252
+ "cgroupsPath": "/kubepods/podd7f4b509-cf94-4951-9417-d1087c92a5b2/7ccf55aee35dd16aca4189c952d83487297f3cd760f1bbf09620e206e7d0c27a"
253
+ ```
254
+
255
+ <!--
256
+ In this specific case, the pod cgroup path is `kubepods/podd7f4b509-cf94-4951-9417-d1087c92a5b2`. Verify the Pod level cgroup setting for memory:
257
+ -->
258
+ 在这个例子中,该 pod 的 cgroup 路径是 ` kubepods/podd7f4b509-cf94-4951-9417-d1087c92a5b2 ` 。验证内存的 Pod 级别 cgroup 设置:
259
+
260
+ <!--
261
+ ```bash
262
+ # Run this on the node where the Pod is scheduled.
263
+ # Also, change the name of the cgroup to match the cgroup allocated for your pod.
264
+ -->
265
+ ``` bash
266
+ # 在该 Pod 调度的节点上执行这个命令。
267
+ # 另外,修改 cgroup 的名称以匹配为你自己 pod 分配的 cgroup。
268
+ cat /sys/fs/cgroup/memory/kubepods/podd7f4b509-cf94-4951-9417-d1087c92a5b2/memory.limit_in_bytes
269
+ ```
270
+
271
+ <!--
272
+ This is 320 MiB, as expected:
273
+ -->
274
+ 和预期的一样是 320 MiB
275
+ ```
276
+ 335544320
277
+ ```
278
+ <!--
279
+ ### Observability
280
+ -->
281
+ ### 可观察性
282
+
283
+ <!--
284
+ A `kube_pod_overhead` metric is available in [kube-state-metrics](https://github.com/kubernetes/kube-state-metrics)
285
+ to help identify when PodOverhead is being utilized and to help observe stability of workloads
286
+ running with a defined Overhead. This functionality is not available in the 1.9 release of
287
+ kube-state-metrics, but is expected in a following release. Users will need to build kube-state-metrics
288
+ from source in the meantime.
289
+ -->
290
+ 在 [ kube-state-metrics] ( https://github.com/kubernetes/kube-state-metrics ) 中可以通过 ` kube_pod_overhead ` 指标来协助确定何时使用 PodOverhead 以及协助观察以一个既定开销运行的工作负载的稳定性。该特性在 kube-state-metrics 的 1.9 发行版本中不可用,不过预计将在后续版本中发布。在此之前,用户需要从源代码构建 kube-state-metrics.
82
291
83
292
{{% /capture %}}
84
293
85
294
{{% capture whatsnext %}}
86
295
87
- <!--
88
296
* [ RuntimeClass] ( /docs/concepts/containers/runtime-class/ )
89
- * [PodOverhead Design](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/20190226-pod-overhead.md)
90
- -->
297
+ * [ PodOverhead 设计] ( https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/20190226-pod-overhead.md )
91
298
92
299
{{% /capture %}}
0 commit comments