Skip to content

Commit 9f83868

Browse files
authored
Update troubleshoot-container-storage.md
Add "Pod pending creation due to ephemeral volume size above available capacity"
1 parent 4a06bc8 commit 9f83868

File tree

1 file changed

+51
-0
lines changed

1 file changed

+51
-0
lines changed

articles/storage/container-storage/troubleshoot-container-storage.md

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,57 @@ Because ephemeral disks (local NVMe and Temp SSD) are ephemeral and not durable,
7171

7272
If you need a persistent volume, where the volume has a lifecycle independent of any individual pod that's using the volume, Azure Container Storage supports replication for NVMe. You can create a storage pool with replication and create persistent volumes from there. See [Create storage pool with volume replication](use-container-storage-with-local-nvme-replication.md) for guidance. Note that because ephemeral disk storage pools consume all the available NVMe disks, you must delete any existing ephemeral disk storage pools before creating a new storage pool with replication enabled. If you don't need persistence, you can create a generic ephemeral volume.
7373

74+
### Pod pending creation due to ephemeral volume size above available capacity
75+
76+
An ephemeral volume is allocated on single node. When you configure the size of emphemeral volumes for your pods, the size should be less than the available capacity of single node ephemeral disk. Otherwise, the pod creation will be in pending status.
77+
78+
1. Use the following command to check if your pod creation is pending status.
79+
80+
```output
81+
$ kubectl get pods
82+
NAME READY STATUS RESTARTS AGE
83+
fiopod 0/1 Pending 0 17s
84+
```
85+
86+
In this example, the pod `fiopod` is in `Pending` status.
87+
88+
1. Use the following command to check if the pod has the warning event for persistentvolumeclaim creation.
89+
90+
```output
91+
$ kubectl describe pod fiopod
92+
...
93+
Events:
94+
Type Reason Age From Message
95+
---- ------ ---- ---- -------
96+
Warning FailedScheduling 40s default-scheduler 0/3 nodes are available: waiting for ephemeral volume controller to create the persistentvolumeclaim "fiopod-ephemeralvolume". preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling..
97+
```
98+
99+
In this example, the pod shows the warning event on creating persistent volume claim `fiopod-ephemeralvolume`.
100+
101+
1. Use the following command to check if the persistent volume claim fails to provision due to insufficient capacity.
102+
103+
```output
104+
$ kubectl describe pvc fiopod-ephemeralvolume
105+
...
106+
Warning ProvisioningFailed 107s (x13 over 20m) containerstorage.csi.azure.com_aks-nodepool1-29463073-vmss000000_7f5bd88d-be76-40d2-a59e-e51ce000e35e failed to provision volume with StorageClass "acstor-ephemeraldisk-temp": rpc error: code = Internal desc = Operation failed: GenericOperation("error in response: status code '507 Insufficient Storage', content: 'RestJsonError { details: \"Operation failed due to insufficient resources: Not enough suitable pools available, 0/1\", message: \"SvcError :: NotEnoughResources\", kind: ResourceExhausted }'")
107+
```
108+
109+
In this example, `Insufiicient Storage` is shown as the reason of volume provisioning failure.
110+
111+
1. Run the following command to check the available capacity of single node ephemeral disk.
112+
113+
```output
114+
$ kubectl get diskpool -n acstor
115+
NAME CAPACITY AVAILABLE USED RESERVED READY AGE
116+
ephemeraldisk-temp-diskpool-jaxwb 75660001280 75031990272 628011008 560902144 True 21h
117+
ephemeraldisk-temp-diskpool-wzixx 75660001280 75031990272 628011008 560902144 True 21h
118+
ephemeraldisk-temp-diskpool-xbtlj 75660001280 75031990272 628011008 560902144 True 21h
119+
```
120+
121+
In this example, the available capacity of single node temp disk is `75031990272` bytes or 69 GiB.
122+
123+
1. Adjust the volume storage size below available capacity and re-deploy your pod. See [Deploy a pod with a generic ephemeral volume](use-container-storage-with-temp-ssd.md#3-deploy-a-pod-with-a-generic-ephemeral-volume).
124+
74125
## See also
75126

76127
- [Azure Container Storage FAQ](container-storage-faq.md)

0 commit comments

Comments
 (0)