Move ephemeral storage contents out of container resource page

Author: tengqm

content/en/docs/concepts/configuration/manage-resources-containers.md

@@ -268,7 +268,8 @@ limits you defined.
   as restartable, Kubernetes restarts the container.
 - The memory limit for the Pod or container can also apply to pages in memory backed
   volumes, such as an `emptyDir`. The kubelet tracks `tmpfs` emptyDir volumes as container
-  memory use, rather than as local ephemeral storage.　When using memory backed `emptyDir`,
+  memory use, rather than as local [ephemeral storage](/docs/concepts/storage/ephemeral-storage/).
+  When using memory backed `emptyDir`,
   be sure to check the notes [below](#memory-backed-emptydir).
 
 If a container exceeds its memory request and the node that it runs on becomes short of
@@ -337,295 +338,10 @@ As an alternative, a cluster administrator can enforce size limits for
 
 ## Local ephemeral storage
 
-<!-- feature gate LocalStorageCapacityIsolation -->
-{{< feature-state for_k8s_version="v1.25" state="stable" >}}
-
-Nodes have local ephemeral storage, backed by
-locally-attached writeable devices or, sometimes, by RAM.
-"Ephemeral" means that there is no long-term guarantee about durability.
-
-Pods use ephemeral local storage for scratch space, caching, and for logs.
-The kubelet can provide scratch space to Pods using local ephemeral storage to
-mount [`emptyDir`](/docs/concepts/storage/volumes/#emptydir)
- {{< glossary_tooltip term_id="volume" text="volumes" >}} into containers.
-
-The kubelet also uses this kind of storage to hold
-[node-level container logs](/docs/concepts/cluster-administration/logging/#logging-at-the-node-level),
-container images, and the writable layers of running containers.
-
-{{< caution >}}
-If a node fails, the data in its ephemeral storage can be lost.
-Your applications cannot expect any performance SLAs (disk IOPS for example)
-from local ephemeral storage.
-{{< /caution >}}
-
-
-{{< note >}}
-To make the resource quota work on ephemeral-storage, two things need to be done:
-
-* An admin sets the resource quota for ephemeral-storage in a namespace.
-* A user needs to specify limits for the ephemeral-storage resource in the Pod spec.
-
-If the user doesn't specify the ephemeral-storage resource limit in the Pod spec,
-the resource quota is not enforced on ephemeral-storage.
-
-{{< /note >}}
-
-Kubernetes lets you track, reserve and limit the amount
-of ephemeral local storage a Pod can consume.
-
-### Configurations for local ephemeral storage
-
-Kubernetes supports two ways to configure local ephemeral storage on a node:
-{{< tabs name="local_storage_configurations" >}}
-{{% tab name="Single filesystem" %}}
-In this configuration, you place all different kinds of ephemeral local data
-(`emptyDir` volumes, writeable layers, container images, logs) into one filesystem.
-The most effective way to configure the kubelet means dedicating this filesystem
-to Kubernetes (kubelet) data.
-
-The kubelet also writes
-[node-level container logs](/docs/concepts/cluster-administration/logging/#logging-at-the-node-level)
-and treats these similarly to ephemeral local storage.
-
-The kubelet writes logs to files inside its configured log directory (`/var/log`
-by default); and has a base directory for other locally stored data
-(`/var/lib/kubelet` by default).
-
-Typically, both `/var/lib/kubelet` and `/var/log` are on the system root filesystem,
-and the kubelet is designed with that layout in mind.
-
-Your node can have as many other filesystems, not used for Kubernetes,
-as you like.
-{{% /tab %}}
-{{% tab name="Two filesystems" %}}
-You have a filesystem on the node that you're using for ephemeral data that
-comes from running Pods: logs, and `emptyDir` volumes. You can use this filesystem
-for other data (for example: system logs not related to Kubernetes); it can even
-be the root filesystem.
-
-The kubelet also writes
-[node-level container logs](/docs/concepts/cluster-administration/logging/#logging-at-the-node-level)
-into the first filesystem, and treats these similarly to ephemeral local storage.
-
-You also use a separate filesystem, backed by a different logical storage device.
-In this configuration, the directory where you tell the kubelet to place
-container image layers and writeable layers is on this second filesystem.
-
-The first filesystem does not hold any image layers or writeable layers.
-
-Your node can have as many other filesystems, not used for Kubernetes,
-as you like.
-{{% /tab %}}
-{{< /tabs >}}
-
-The kubelet can measure how much local storage it is using. It does this provided
-that you have set up the node using one of the supported configurations for local
-ephemeral storage.
-
-If you have a different configuration, then the kubelet does not apply resource
-limits for ephemeral local storage.
-
-{{< note >}}
-The kubelet tracks `tmpfs` emptyDir volumes as container memory use, rather
-than as local ephemeral storage.
-{{< /note >}}
-
-{{< note >}}
-The kubelet will only track the root filesystem for ephemeral storage. OS layouts that mount a separate disk to `/var/lib/kubelet` or `/var/lib/containers` will not report ephemeral storage correctly.
-{{< /note >}}
-
-### Setting requests and limits for local ephemeral storage
-
-You can specify `ephemeral-storage` for managing local ephemeral storage. Each
-container of a Pod can specify either or both of the following:
-
-* `spec.containers[].resources.limits.ephemeral-storage`
-* `spec.containers[].resources.requests.ephemeral-storage`
-
-Limits and requests for `ephemeral-storage` are measured in byte quantities.
-You can express storage as a plain integer or as a fixed-point number using one of these suffixes:
-E, P, T, G, M, k. You can also use the power-of-two equivalents: Ei, Pi, Ti, Gi,
-Mi, Ki. For example, the following quantities all represent roughly the same value:
-
-- `128974848`
-- `129e6`
-- `129M`
-- `123Mi`
-
-Pay attention to the case of the suffixes. If you request `400m` of ephemeral-storage, this is a request
-for 0.4 bytes. Someone who types that probably meant to ask for 400 mebibytes (`400Mi`)
-or 400 megabytes (`400M`).
-
-In the following example, the Pod has two containers. Each container has a request of
-2GiB of local ephemeral storage. Each container has a limit of 4GiB of local ephemeral
-storage. Therefore, the Pod has a request of 4GiB of local ephemeral storage, and
-a limit of 8GiB of local ephemeral storage. 500Mi of that limit could be
-consumed by the `emptyDir` volume.
-
-```yaml
-apiVersion: v1
-kind: Pod
-metadata:
-  name: frontend
-spec:
-  containers:
-  - name: app
-    image: images.my-company.example/app:v4
-    resources:
-      requests:
-        ephemeral-storage: "2Gi"
-      limits:
-        ephemeral-storage: "4Gi"
-    volumeMounts:
-    - name: ephemeral
-      mountPath: "/tmp"
-  - name: log-aggregator
-    image: images.my-company.example/log-aggregator:v6
-    resources:
-      requests:
-        ephemeral-storage: "2Gi"
-      limits:
-        ephemeral-storage: "4Gi"
-    volumeMounts:
-    - name: ephemeral
-      mountPath: "/tmp"
-  volumes:
-    - name: ephemeral
-      emptyDir:
-        sizeLimit: 500Mi
-```
-
-### How Pods with ephemeral-storage requests are scheduled
-
-When you create a Pod, the Kubernetes scheduler selects a node for the Pod to
-run on. Each node has a maximum amount of local ephemeral storage it can provide for Pods.
-For more information, see
-[Node Allocatable](/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable).
-
-The scheduler ensures that the sum of the resource requests of the scheduled containers is less than the capacity of the node.
-
-### Ephemeral storage consumption management {#resource-emphemeralstorage-consumption}
-
-If the kubelet is managing local ephemeral storage as a resource, then the
-kubelet measures storage use in:
-
-- `emptyDir` volumes, except _tmpfs_ `emptyDir` volumes
-- directories holding node-level logs
-- writeable container layers
-
-If a Pod is using more ephemeral storage than you allow it to, the kubelet
-sets an eviction signal that triggers Pod eviction.
-
-For container-level isolation, if a container's writable layer and log
-usage exceeds its storage limit, the kubelet marks the Pod for eviction.
-
-For pod-level isolation the kubelet works out an overall Pod storage limit by
-summing the limits for the containers in that Pod. In this case, if the sum of
-the local ephemeral storage usage from all containers and also the Pod's `emptyDir`
-volumes exceeds the overall Pod storage limit, then the kubelet also marks the Pod
-for eviction.
-
-{{< caution >}}
-If the kubelet is not measuring local ephemeral storage, then a Pod
-that exceeds its local storage limit will not be evicted for breaching
-local storage resource limits.
-
-However, if the filesystem space for writeable container layers, node-level logs,
-or `emptyDir` volumes falls low, the node
-{{< glossary_tooltip text="taints" term_id="taint" >}} itself as short on local storage
-and this taint triggers eviction for any Pods that don't specifically tolerate the taint.
-
-See the supported [configurations](#configurations-for-local-ephemeral-storage)
-for ephemeral local storage.
-{{< /caution >}}
-
-The kubelet supports different ways to measure Pod storage use:
-
-{{< tabs name="resource-emphemeralstorage-measurement" >}}
-{{% tab name="Periodic scanning" %}}
-The kubelet performs regular, scheduled checks that scan each
-`emptyDir` volume, container log directory, and writeable container layer.
-
-The scan measures how much space is used.
-
-{{< note >}}
-In this mode, the kubelet does not track open file descriptors
-for deleted files.
-
-If you (or a container) create a file inside an `emptyDir` volume,
-something then opens that file, and you delete the file while it is
-still open, then the inode for the deleted file stays until you close
-that file but the kubelet does not categorize the space as in use.
-{{< /note >}}
-{{% /tab %}}
-{{% tab name="Filesystem project quota" %}}
-
-{{< feature-state feature_gate_name="LocalStorageCapacityIsolationFSQuotaMonitoring" >}}
-
-Project quotas are an operating-system level feature for managing
-storage use on filesystems. With Kubernetes, you can enable project
-quotas for monitoring storage use. Make sure that the filesystem
-backing the `emptyDir` volumes, on the node, provides project quota support.
-For example, XFS and ext4fs offer project quotas.
-
-{{< note >}}
-Project quotas let you monitor storage use; they do not enforce limits.
-{{< /note >}}
-
-Kubernetes uses project IDs starting from `1048576`. The IDs in use are
-registered in `/etc/projects` and `/etc/projid`. If project IDs in
-this range are used for other purposes on the system, those project
-IDs must be registered in `/etc/projects` and `/etc/projid` so that
-Kubernetes does not use them.
-
-Quotas are faster and more accurate than directory scanning. When a
-directory is assigned to a project, all files created under a
-directory are created in that project, and the kernel merely has to
-keep track of how many blocks are in use by files in that project.
-If a file is created and deleted, but has an open file descriptor,
-it continues to consume space. Quota tracking records that space accurately
-whereas directory scans overlook the storage used by deleted files.
-
-To use quotas to track a pod's resource usage, the pod must be in 
-a user namespace. Within user namespaces, the kernel restricts changes 
-to projectIDs on the filesystem, ensuring the reliability of storage 
-metrics calculated by quotas.
-
-If you want to use project quotas, you should:
-
-* Enable the `LocalStorageCapacityIsolationFSQuotaMonitoring=true`
-  [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
-  using the `featureGates` field in the
-  [kubelet configuration](/docs/reference/config-api/kubelet-config.v1beta1/).
-
-* Ensure the `UserNamespacesSupport` 
-  [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
-  is enabled, and that the kernel, CRI implementation and OCI runtime support user namespaces.
-
-* Ensure that the root filesystem (or optional runtime filesystem)
-  has project quotas enabled. All XFS filesystems support project quotas.
-  For ext4 filesystems, you need to enable the project quota tracking feature
-  while the filesystem is not mounted.
-
-  ```bash
-  # For ext4, with /dev/block-device not mounted
-  sudo tune2fs -O project -Q prjquota /dev/block-device
-  ```
-
-* Ensure that the root filesystem (or optional runtime filesystem) is
-  mounted with project quotas enabled. For both XFS and ext4fs, the
-  mount option is named `prjquota`.
-
-
-If you don't want to use project quotas, you should:
-
-* Disable the `LocalStorageCapacityIsolationFSQuotaMonitoring`
-  [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
-  using the `featureGates` field in the
-  [kubelet configuration](/docs/reference/config-api/kubelet-config.v1beta1/).
-{{% /tab %}}
-{{< /tabs >}}
+For general concepts about local ephemeral storage and hints about
+configuring the requests and/or limits of ephemeral storage for a container,
+please check the [local ephemeral storage](/docs/concepts/storage/ephemeral-storage/)
+page.
 
 ## Extended resources
 
@@ -940,7 +656,7 @@ memory limit (and possibly request) for that container.
 * Get hands-on experience [assigning CPU resources to containers and Pods](/docs/tasks/configure-pod-container/assign-cpu-resource/).
 * Read how the API reference defines a [container](/docs/reference/kubernetes-api/workload-resources/pod-v1/#Container)
   and its [resource requirements](/docs/reference/kubernetes-api/workload-resources/pod-v1/#resources)
-* Read about [project quotas](https://www.linux.org/docs/man8/xfs_quota.html) in XFS
+* Read more about the [local ephemeral storage](/docs/concepts/storage/ephemeral-storage/)
 * Read more about the [kube-scheduler configuration reference (v1)](/docs/reference/config-api/kube-scheduler-config.v1/)
 * Read more about [Quality of Service classes for Pods](/docs/concepts/workloads/pods/pod-qos/)
 * Read more about [Extended Resource allocation by DRA](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#extended-resource)