Merge pull request #51849 from kubernetes/dev-1.34

Official 1.34 Release Docs

Author: michellengnx

content/en/docs/concepts/cluster-administration/node-shutdown.md

@@ -16,30 +16,66 @@ either **graceful** or **non-graceful**.
 
 ## Graceful node shutdown {#graceful-node-shutdown}
 
-{{< feature-state feature_gate_name="GracefulNodeShutdown" >}}
-
 The kubelet attempts to detect node system shutdown and terminates pods running on the node.
 
-kubelet ensures that pods follow the normal
+Kubelet ensures that pods follow the normal
 [pod termination process](/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination)
 during the node shutdown. During node shutdown, the kubelet does not accept new
 Pods (even if those Pods are already bound to the node).
 
+### Enabling graceful node shutdown
+
+{{< tabs name="graceful_shutdown_os" >}}
+{{% tab name="Linux" %}}
+{{< feature-state feature_gate_name="GracefulNodeShutdown" >}}
+
+On Linux, the graceful node shutdown feature is controlled with the `GracefulNodeShutdown`
+[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) which is
+enabled by default in 1.21.
+
+{{< note >}}
 The graceful node shutdown feature depends on systemd since it takes advantage of
 [systemd inhibitor locks](https://www.freedesktop.org/wiki/Software/systemd/inhibit/) to
 delay the node shutdown with a given duration.
+{{</ note >}}
+{{% /tab %}}
 
-Graceful node shutdown is controlled with the `GracefulNodeShutdown`
-[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) which is
-enabled by default in 1.21.
+{{% tab name="Windows" %}}
+{{< feature-state feature_gate_name="WindowsGracefulNodeShutdown" >}}
+
+On Windows, the graceful node shutdown feature is controlled with the `WindowsGracefulNodeShutdown`
+[feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
+which is introduced in 1.32 as an alpha feature. In Kubernetes 1.34 the feature is Beta
+and is enabled by default.
+
+{{< note >}}
+The Windows graceful node shutdown feature depends on kubelet running as a Windows service,
+it will then have a registered [service control handler](https://learn.microsoft.com/en-us/windows/win32/services/service-control-handler-function)
+to delay the preshutdown event with a given duration.
+{{</ note >}}
+
+Windows graceful node shutdown can not be cancelled.
+
+If kubelet is not running as a Windows service, it will not be able to set and monitor
+the [Preshutdown](https://learn.microsoft.com/en-us/windows/win32/api/winsvc/ns-winsvc-service_preshutdown_info) event,
+the node will have to go through the [Non-Graceful Node Shutdown](#non-graceful-node-shutdown) procedure mentioned above.
+
+In the case where the Windows graceful node shutdown feature is enabled, but the kubelet is not
+running as a Windows service, the kubelet will continue running instead of failing. However,
+it will log an error indicating that it needs to be run as a Windows service.
+{{% /tab %}}
+
+{{< /tabs >}}
+
+### Configuring graceful node shutdown
 
 Note that by default, both configuration options described below,
 `shutdownGracePeriod` and `shutdownGracePeriodCriticalPods`, are set to zero,
 thus not activating the graceful node shutdown functionality.
 To activate the feature, both options should be configured appropriately and
 set to non-zero values.
 
-Once systemd detects or is notified of a node shutdown, the kubelet sets a `NotReady` condition on
+Once the kubelet is notified of a node shutdown, it sets a `NotReady` condition on
 the Node, with the `reason` set to `"node is shutting down"`. The kube-scheduler honors this condition
 and does not schedule any Pods onto the affected node; other third-party schedulers are
 expected to follow the same logic. This means that new Pods won't be scheduled onto that node
@@ -273,28 +309,6 @@ via the [Non-Graceful Node Shutdown](#non-graceful-node-shutdown) procedure ment
 
 {{< /note >}}
 
-## Windows Graceful node shutdown {#windows-graceful-node-shutdown}
-
-{{< feature-state feature_gate_name="WindowsGracefulNodeShutdown" >}}
-
-The Windows graceful node shutdown feature depends on kubelet running as a Windows service,
-it will then have a registered [service control handler](https://learn.microsoft.com/en-us/windows/win32/services/service-control-handler-function)
-to delay the preshutdown event with a given duration.
-
-Windows graceful node shutdown is controlled with the `WindowsGracefulNodeShutdown`
-[feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
-which is introduced in 1.32 as an alpha feature.
-
-Windows graceful node shutdown can not be cancelled.
-
-If kubelet is not running as a Windows service, it will not be able to set and monitor
-the [Preshutdown](https://learn.microsoft.com/en-us/windows/win32/api/winsvc/ns-winsvc-service_preshutdown_info) event,
-the node will have to go through the [Non-Graceful Node Shutdown](#non-graceful-node-shutdown) procedure mentioned above.
-
-In the case where the Windows graceful node shutdown feature is enabled, but the kubelet is not
-running as a Windows service, the kubelet will continue running instead of failing. However,
-it will log an error indicating that it needs to be run as a Windows service.
-
 ## {{% heading "whatsnext" %}}
 
 Learn more about the following:

content/en/docs/concepts/cluster-administration/system-metrics.md

@@ -179,11 +179,11 @@ flag to expose these alpha stability metrics.
 
 ### kubelet Pressure Stall Information (PSI) metrics
 
-{{< feature-state for_k8s_version="v1.33" state="alpha" >}}
+{{< feature-state for_k8s_version="v1.34" state="beta" >}}
 
-As an alpha feature, Kubernetes lets you configure kubelet to collect Linux kernel
+As a beta feature, Kubernetes lets you configure kubelet to collect Linux kernel
 [Pressure Stall Information](https://docs.kernel.org/accounting/psi.html)
-(PSI) for CPU, memory and IO usage.
+(PSI) for CPU, memory and I/O usage.
 The information is collected at node, pod and container level.
 The metrics are exposed at the `/metrics/cadvisor` endpoint with the following names:
 
@@ -196,10 +196,11 @@ container_pressure_io_stalled_seconds_total
 container_pressure_io_waiting_seconds_total
 ```
 
-You must enable the `KubeletPSI` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
-to use this feature. The information is also exposed in the
+This feature is enabled by default, by setting the `KubeletPSI` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/). The information is also exposed in the
 [Summary API](/docs/reference/instrumentation/node-metrics#psi).
 
+You can learn how to interpret the PSI metrics in [Understand PSI Metrics](/docs/reference/instrumentation/understand-psi-metrics/).
+
 #### Requirements
 
 Pressure Stall Information requires:

content/en/docs/concepts/cluster-administration/system-traces.md

@@ -78,15 +78,15 @@ with `--tracing-config-file=<path-to-config>`. This is an example config that re
 spans for 1 in 10000 requests, and uses the default OpenTelemetry endpoint:
 
 ```yaml
-apiVersion: apiserver.config.k8s.io/v1beta1
+apiVersion: apiserver.config.k8s.io/v1
 kind: TracingConfiguration
 # default value
 #endpoint: localhost:4317
 samplingRatePerMillion: 100
 ```
 
 For more information about the `TracingConfiguration` struct, see
-[API server config API (v1beta1)](/docs/reference/config-api/apiserver-config.v1beta1/#apiserver-k8s-io-v1beta1-TracingConfiguration).
+[API server config API (v1)](/docs/reference/config-api/apiserver-config.v1/#apiserver-k8s-io-v1-TracingConfiguration).
 
 ### kubelet traces
 
@@ -106,8 +106,6 @@ This is an example snippet of a kubelet config that records spans for 1 in 10000
 ```yaml
 apiVersion: kubelet.config.k8s.io/v1beta1
 kind: KubeletConfiguration
-featureGates:
-  KubeletTracing: true
 tracing:
   # default value
   #endpoint: localhost:4317

content/en/docs/concepts/configuration/manage-resources-containers.md

@@ -113,21 +113,24 @@ resource requests/limits of that type for each container in the Pod.
 
 {{< feature-state feature_gate_name="PodLevelResources" >}}
 
-Starting in Kubernetes 1.32, you can also specify resource requests and limits at
+Provided your cluster has the `PodLevelResources`
+[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) enabled,
+you can specify resource requests and limits at
 the Pod level. At the Pod level, Kubernetes {{< skew currentVersion >}}
 only supports resource requests or limits for specific resource types: `cpu` and /
-or `memory`. This feature is currently in alpha and with the feature enabled,
-Kubernetes allows you to declare an overall resource budget for the Pod, which is
-especially helpful when dealing with a large number of containers where it can be
-difficult to accurately gauge individual resource needs. Additionally, it enables
-containers within a Pod to share idle resources with each other, improving resource
-utilization. 
+or `memory` and / or `hugepages`. With this feature, Kubernetes allows you to declare an overall resource
+budget for the Pod, which is especially helpful when dealing with a large number of
+containers where it can be difficult to accurately gauge individual resource needs.
+Additionally, it enables containers within a Pod to share idle resources with each
+other, improving resource utilization.
 
 For a Pod, you can specify resource limits and requests for CPU and memory by including the following:
 * `spec.resources.limits.cpu`
 * `spec.resources.limits.memory`
+* `spec.resources.limits.hugepages-<size>`
 * `spec.resources.requests.cpu`
 * `spec.resources.requests.memory`
+* `spec.resources.requests.hugepages-<size>`
 
 ## Resource units in Kubernetes
 
@@ -718,6 +721,12 @@ extender.
 }
 ```
 
+#### Extended resources allocation by DRA
+Extended resources allocation by DRA allows cluster administrators to specify an `extendedResourceName`
+in DeviceClass, then the devices matching the DeviceClass can be requested from a pod's extended
+resource requests. Read more about
+[Extended Resource allocation by DRA](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#extended-resource).
+
 ### Consuming extended resources
 
 Users can consume extended resources in Pod specs like CPU and memory.
@@ -934,3 +943,4 @@ memory limit (and possibly request) for that container.
 * Read about [project quotas](https://www.linux.org/docs/man8/xfs_quota.html) in XFS
 * Read more about the [kube-scheduler configuration reference (v1)](/docs/reference/config-api/kube-scheduler-config.v1/)
 * Read more about [Quality of Service classes for Pods](/docs/concepts/workloads/pods/pod-qos/)
+* Read more about [Extended Resource allocation by DRA](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#extended-resource)

content/en/docs/concepts/containers/container-lifecycle-hooks.md

@@ -64,13 +64,6 @@ There are three types of hook handlers that can be implemented for Containers:
 Resources consumed by the command are counted against the Container.
 * HTTP - Executes an HTTP request against a specific endpoint on the Container.
 * Sleep - Pauses the container for a specified duration. 
-  This is a beta-level feature default enabled by the `PodLifecycleSleepAction`
-  [feature gate](/docs/reference/command-line-tools-reference/feature-gates/). 
-
-{{< note >}}
-The beta level `PodLifecycleSleepActionAllowZero` feature gate which is enabled by default from v1.33.
-It allows you to set a sleep duration of zero seconds (effectively a no-op) for your Sleep lifecycle hooks.
-{{< /note >}}
 
 ### Hook handler execution
 

content/en/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md

@@ -270,11 +270,12 @@ the NUMA node where these devices are allocated. Also, for NUMA-based machines,
 information about memory and hugepages reserved for a container.
 
 Starting from Kubernetes v1.27, the `List` endpoint can provide information on resources
-of running pods allocated in `ResourceClaims` by the `DynamicResourceAllocation` API. To enable
-this feature `kubelet` must be started with the following flags:
+of running pods allocated in `ResourceClaims` by the `DynamicResourceAllocation` API.
+Starting from Kubernetes v1.34, this feature is enabled by default.
+To disable, `kubelet` must be started with the following flags:
 
 ```
---feature-gates=DynamicResourceAllocation=true,KubeletPodResourcesDynamicResources=true
+--feature-gates=KubeletPodResourcesDynamicResources=false
 ```
 
 ```gRPC
@@ -414,7 +415,7 @@ will continue working.
 
 ### `Get` gRPC endpoint {#grpc-endpoint-get}
 
-{{< feature-state state="alpha" for_k8s_version="v1.27" >}}
+{{< feature-state state="beta" for_k8s_version="v1.34" >}}
 
 The `Get` endpoint provides information on resources of a running Pod. It exposes information
 similar to those described in the `List` endpoint. The `Get` endpoint requires `PodName`
@@ -428,18 +429,19 @@ message GetPodResourcesRequest {
 }
 ```
 
-To enable this feature, you must start your kubelet services with the following flag:
+To disable this feature, you must start your kubelet services with the following flag:
 
 ```
---feature-gates=KubeletPodResourcesGet=true
+--feature-gates=KubeletPodResourcesGet=false
 ```
 
 The `Get` endpoint can provide Pod information related to dynamic resources
-allocated by the dynamic resource allocation API. To enable this feature, you must
-ensure your kubelet services are started with the following flags:
+allocated by the dynamic resource allocation API.
+Starting from Kubernetes v1.34, this feature is enabled by default.
+To disable, `kubelet` must be started with the following flags:
 
 ```
---feature-gates=KubeletPodResourcesGet=true,DynamicResourceAllocation=true,KubeletPodResourcesDynamicResources=true
+--feature-gates=KubeletPodResourcesDynamicResources=false
 ```
 
 ## Device plugin integration with the Topology Manager
@@ -509,3 +511,4 @@ Here are some examples of device plugin implementations:
 * Learn about the [Topology Manager](/docs/tasks/administer-cluster/topology-manager/)
 * Read about using [hardware acceleration for TLS ingress](/blog/2019/04/24/hardware-accelerated-ssl/tls-termination-in-ingress-controllers-using-kubernetes-device-plugins-and-runtimeclass/)
   with Kubernetes
+* Read more about [Extended Resource allocation by DRA](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#extended-resource)

content/en/docs/concepts/policy/node-resource-managers.md

@@ -207,7 +207,7 @@ listed in alphabetical order:
 : Prevent all the pods regardless of their Quality of Service class to run on reserved CPUs
   (available since Kubernetes v1.32)
 
-`prefer-align-cpus-by-uncorecache` (alpha, hidden by default)
+`prefer-align-cpus-by-uncorecache` (beta, visible by default)
 : Align CPUs by uncore (Last-Level) cache boundary on a best-effort way
   (available since Kubernetes v1.32)
 

content/en/docs/concepts/scheduling-eviction/assign-pod-node.md

@@ -641,6 +641,30 @@ spec:
 
 The above Pod will only run on the node `kube-01`.
 
+## nominatedNodeName
+
+{{< feature-state feature_gate_name="NominatedNodeNameForExpectation" >}}
+
+`nominatedNodeName` can be used for external components to nominate node for a pending pod.
+This nomination is best effort: it might be ignored if the scheduler determines the pod cannot go to a nominated node.
+
+Also, this field can be (over)written by the scheduler:
+- If the scheduler finds a node to nominate via the preemption.
+- If the scheduler decides where the pod is going, and move it to the binding cycle.
+  - Note that, in this case, `nominatedNodeName` is put only when the pod has to go through `WaitOnPermit` or `PreBind` extension points.
+
+Here is an example of a Pod status using the `nominatedNodeName` field:
+
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+  name: nginx
+...
+status:
+  nominatedNodeName: kube-01
+```
+
 ## Pod topology spread constraints
 
 You can use _topology spread constraints_ to control how {{< glossary_tooltip text="Pods" term_id="Pod" >}}