You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
without a finalizer, to a terminal phase (`Failed` or `Succeeded` depending on
98
98
the exit statuses of the pod containers) before their deletion from the API server.
99
99
@@ -219,13 +219,13 @@ status:
219
219
...
220
220
```
221
221
222
-
The Pod conditions you add must have names that meet the Kubernetes[label key format](/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set).
223
-
222
+
The Pod conditions you add must have names that meet the Kubernetes
Startup probes are useful for Pods that have containers that take a long time to
416
410
come into service. Rather than set a long liveness interval, you can configure
417
411
a separate configuration for probing the container as it starts up, allowing
@@ -440,68 +434,77 @@ shutdown.
440
434
Typically, the container runtime sends a TERM signal to the main process in each
441
435
container. Many container runtimes respect the `STOPSIGNAL` value defined in the container
442
436
image and send this instead of TERM.
443
-
Once the grace period has expired, the KILL signal is sent to any remaining
444
-
processes, and the Pod is then deleted from the
445
-
{{< glossary_tooltip text="API Server" term_id="kube-apiserver" >}}. If the kubelet or the
446
-
container runtime's management service is restarted while waiting for processes to terminate, the
447
-
cluster retries from the start including the full original grace period.
437
+
Once the grace period has expired, the KILL signal is sent to any remaining processes, and the Pod
438
+
is then deleted from the {{< glossary_tooltip text="API Server" term_id="kube-apiserver" >}}.
439
+
If the kubelet or the container runtime's management service is restarted while waiting for
440
+
processes to terminate, the cluster retries from the start including the full original grace period.
448
441
449
442
An example flow:
450
443
451
444
1. You use the `kubectl` tool to manually delete a specific Pod, with the default grace period
452
445
(30 seconds).
446
+
453
447
1. The Pod in the API server is updated with the time beyond which the Pod is considered "dead"
454
448
along with the grace period.
455
-
If you use `kubectl describe` to check on the Pod you're deleting, that Pod shows up as
456
-
"Terminating".
449
+
If you use `kubectl describe` to check the Pod you're deleting, that Pod shows up as "Terminating".
457
450
On the node where the Pod is running: as soon as the kubelet sees that a Pod has been marked
458
451
as terminating (a graceful shutdown duration has been set), the kubelet begins the local Pod
459
452
shutdown process.
453
+
460
454
1. If one of the Pod's containers has defined a `preStop`
461
-
[hook](/docs/concepts/containers/container-lifecycle-hooks), the kubelet
462
-
runs that hook inside of the container. If the `preStop` hook is still running after the
463
-
grace period expires, the kubelet requests a small, one-off grace period extension of 2
464
-
seconds.
455
+
[hook](/docs/concepts/containers/container-lifecycle-hooks) and the `terminationGracePeriodSeconds`
456
+
in the Pod spec is not set to 0, the kubelet runs that hook inside of the container.
457
+
The default `terminationGracePeriodSeconds` setting is 30 seconds.
458
+
459
+
If the `preStop` hook is still running after the grace period expires, the kubelet requests
460
+
a small, one-off grace period extension of 2 seconds.
461
+
465
462
{{< note >}}
466
463
If the `preStop` hook needs longer to complete than the default grace period allows,
467
464
you must modify `terminationGracePeriodSeconds` to suit this.
468
465
{{< /note >}}
466
+
469
467
1. The kubelet triggers the container runtime to send a TERM signal to process 1 inside each
470
468
container.
471
469
{{< note >}}
472
470
The containers in the Pod receive the TERM signal at different times and in an arbitrary
473
471
order. If the order of shutdowns matters, consider using a `preStop` hook to synchronize.
474
472
{{< /note >}}
475
-
1. At the same time as the kubelet is starting graceful shutdown of the Pod, the control plane evaluates whether to remove that shutting-down Pod from EndpointSlice (and Endpoints) objects, where those objects represent
476
-
a {{< glossary_tooltip term_id="service" text="Service" >}} with a configured
1. At the same time as the kubelet is starting graceful shutdown of the Pod, the control plane
475
+
evaluates whether to remove that shutting-down Pod from EndpointSlice (and Endpoints) objects,
476
+
where those objects represent a {{< glossary_tooltip term_id="service" text="Service" >}}
477
+
with a configured {{< glossary_tooltip text="selector" term_id="selector" >}}.
478
478
{{< glossary_tooltip text="ReplicaSets" term_id="replica-set" >}} and other workload resources
479
-
no longer treat the shutting-down Pod as a valid, in-service replica. Pods that shut down slowly
480
-
should not continue to serve regular traffic and should start terminating and finish processing open connections.
481
-
Some applications need to go beyond finishing open connections and need more graceful termination -
482
-
for example: session draining and completion. Any endpoints that represent the terminating pods
483
-
are not immediately removed from EndpointSlices,
484
-
and a status indicating [terminating state](/docs/concepts/services-networking/endpoint-slices/#conditions)
485
-
is exposed from the EndpointSlice API (and the legacy Endpoints API). Terminating
486
-
endpoints always have their `ready` status
487
-
as `false` (for backward compatibility with versions before 1.26),
488
-
so load balancers will not use it for regular traffic.
489
-
If traffic draining on terminating pod is needed, the actual readiness can be checked as a condition `serving`.
490
-
You can find more details on how to implement connections draining
491
-
in the tutorial [Pods And Endpoints Termination Flow](/docs/tutorials/services/pods-and-endpoint-termination-flow/)
479
+
no longer treat the shutting-down Pod as a valid, in-service replica.
480
+
481
+
Pods that shut down slowly should not continue to serve regular traffic and should start
482
+
terminating and finish processing open connections. Some applications need to go beyond
483
+
finishing open connections and need more graceful termination, for example, session draining
484
+
and completion.
485
+
486
+
Any endpoints that represent the terminating Pods are not immediately removed from
487
+
EndpointSlices, and a status indicating [terminating state](/docs/concepts/services-networking/endpoint-slices/#conditions)
488
+
is exposed from the EndpointSlice API (and the legacy Endpoints API).
489
+
Terminating endpoints always have their `ready` status as `false` (for backward compatibility
490
+
with versions before 1.26), so load balancers will not use it for regular traffic.
491
+
492
+
If traffic draining on terminating Pod is needed, the actual readiness can be checked as a
493
+
condition `serving`. You can find more details on how to implement connections draining in the
494
+
tutorial [Pods And Endpoints Termination Flow](/docs/tutorials/services/pods-and-endpoint-termination-flow/)
492
495
493
496
{{<note>}}
494
497
If you don't have the `EndpointSliceTerminatingCondition` feature gate enabled
495
-
in your cluster (the gate is on by default from Kubernetes 1.22, and locked to default in 1.26), then the Kubernetes control
496
-
plane removes a Pod from any relevant EndpointSlices as soon as the Pod's
498
+
in your cluster (the gate is on by default from Kubernetes 1.22, and locked to default in 1.26),
499
+
then the Kubernetes control plane removes a Pod from any relevant EndpointSlices as soon as the Pod's
497
500
termination grace period _begins_. The behavior above is described when the
498
501
feature gate `EndpointSliceTerminatingCondition` is enabled.
499
502
{{</note>}}
500
503
501
504
1. When the grace period expires, the kubelet triggers forcible shutdown. The container runtime sends
502
505
`SIGKILL` to any processes still running in any container in the Pod.
503
506
The kubelet also cleans up a hidden `pause` container if that container runtime uses one.
504
-
1. The kubelet transitions the pod into a terminal phase (`Failed` or `Succeeded` depending on
507
+
1. The kubelet transitions the Pod into a terminal phase (`Failed` or `Succeeded` depending on
505
508
the end state of its containers). This step is guaranteed since version 1.27.
506
509
1. The kubelet triggers forcible removal of Pod object from the API server, by setting grace period
507
510
to 0 (immediate deletion).
@@ -518,11 +521,12 @@ the `--grace-period=<seconds>` option which allows you to override the default a
518
521
own value.
519
522
520
523
Setting the grace period to `0` forcibly and immediately deletes the Pod from the API
521
-
server. If the pod was still running on a node, that forcible deletion triggers the kubelet to
524
+
server. If the Pod was still running on a node, that forcible deletion triggers the kubelet to
522
525
begin immediate cleanup.
523
526
524
527
{{< note >}}
525
-
You must specify an additional flag `--force` along with `--grace-period=0` in order to perform force deletions.
528
+
You must specify an additional flag `--force` along with `--grace-period=0`
529
+
in order to perform force deletions.
526
530
{{< /note >}}
527
531
528
532
When a force deletion is performed, the API server does not wait for confirmation
@@ -532,7 +536,8 @@ name. On the node, Pods that are set to terminate immediately will still be give
532
536
a small grace period before being force killed.
533
537
534
538
{{< caution >}}
535
-
Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
539
+
Immediate deletion does not wait for confirmation that the running resource has been terminated.
540
+
The resource may continue to run on the cluster indefinitely.
536
541
{{< /caution >}}
537
542
538
543
If you need to force-delete Pods that are part of a StatefulSet, refer to the task
@@ -545,21 +550,24 @@ For failed Pods, the API objects remain in the cluster's API until a human or
545
550
{{< glossary_tooltip term_id="controller" text="controller" >}} process
546
551
explicitly removes them.
547
552
548
-
The Pod garbage collector (PodGC), which is a controller in the control plane, cleans up terminated Pods (with a phase of `Succeeded` or
549
-
`Failed`), when the number of Pods exceeds the configured threshold
550
-
(determined by `terminated-pod-gc-threshold` in the kube-controller-manager).
553
+
The Pod garbage collector (PodGC), which is a controller in the control plane, cleans up
554
+
terminated Pods (with a phase of `Succeeded` or `Failed`), when the number of Pods exceeds the
555
+
configured threshold (determined by `terminated-pod-gc-threshold` in the kube-controller-manager).
551
556
This avoids a resource leak as Pods are created and terminated over time.
552
557
553
558
Additionally, PodGC cleans up any Pods which satisfy any of the following conditions:
554
-
1. are orphan pods - bound to a node which no longer exists,
555
-
2. are unscheduled terminating pods,
556
-
3. are terminating pods, bound to a non-ready node tainted with [`node.kubernetes.io/out-of-service`](/docs/reference/labels-annotations-taints/#node-kubernetes-io-out-of-service), when the `NodeOutOfServiceVolumeDetach` feature gate is enabled.
559
+
560
+
1. are orphan Pods - bound to a node which no longer exists,
561
+
1. are unscheduled terminating Pods,
562
+
1. are terminating Pods, bound to a non-ready node tainted with
0 commit comments