Merge pull request #270410 from lorbichara/docs-editor/troubleshoot-container-storage-1711569955

Update troubleshoot-container-storage.md

Author: American-Dipper

articles/storage/container-storage/troubleshoot-container-storage.md

@@ -28,7 +28,7 @@ az aks update -n <cluster-name> -g <resource-group> --enable-azure-container-sto
 
 ### Can't set storage pool type to NVMe
 
-If you try to install Azure Container Storage with ephemeral disk, specifically with local NVMe on a cluster where the virtual machine (VM) SKU doesn't have NVMe drives, you get the following error message: *Cannot set --storage-pool-option as NVMe as none of the node pools can support ephemeral NVMe disk*.
+If you try to install Azure Container Storage with Ephemeral Disk, specifically with local NVMe on a cluster where the virtual machine (VM) SKU doesn't have NVMe drives, you get the following error message: *Cannot set --storage-pool-option as NVMe as none of the node pools can support ephemeral NVMe disk*.
 
 To remediate, create a node pool with a VM SKU that has NVMe drives and try again. See [storage optimized VMs](../../virtual-machines/sizes-storage.md).
 
@@ -42,7 +42,7 @@ If you're trying to create an Elastic SAN storage pool, you might see the messag
 
 ### No block devices found
 
-If you see this message, you're likely trying to create an ephemeral disk storage pool on a cluster where the VM SKU doesn't have NVMe drives.
+If you see this message, you're likely trying to create an Ephemeral Disk storage pool on a cluster where the VM SKU doesn't have NVMe drives.
 
 To remediate, create a node pool with a VM SKU that has NVMe drives and try again. See [storage optimized VMs](../../virtual-machines/sizes-storage.md).
 
@@ -64,6 +64,13 @@ If you created an Elastic SAN storage pool, you might not be able to delete the
 
 To resolve this, sign in to the [Azure portal](https://portal.azure.com?azure-portal=true) and select **Resource groups**. Locate the resource group that AKS created (the resource group name starts with **MC_**). Select the SAN resource object within that resource group. Manually remove all volumes and volume groups. Then retry deleting the resource group that includes your AKS cluster.
 
+## Troubleshoot persistent volume issues
+
+### Can't create persistent volumes from ephemeral disk storage pools
+Because ephemeral disks (local NVMe and Temp SSD) are ephemeral and not durable, we enforce the use of [Kubernetes Generic Ephemeral Volumes](https://kubernetes.io/docs/concepts/storage/ephemeral-volumes/#generic-ephemeral-volumes). If you try to create a persistent volume claim using an ephemeral disk pool, you'll see the following error: *Error from server (Forbidden): error when creating "eph-pvc.yaml": admission webhook "pvc.acstor.azure.com" denied the request: only generic ephemeral volumes are allowed in unreplicated ephemeralDisk storage pools*.
+
+If you need a persistent volume, where the volume has a lifecycle independent of any individual pod that's using the volume, Azure Container Storage supports replication for NVMe. You can create a storage pool with replication and create persistent volumes from there. See [Create storage pool with volume replication](use-container-storage-with-local-disk.md#optional-create-storage-pool-with-volume-replication-nvme-only) for guidance. Note that because ephemeral disk storage pools consume all the available NVMe disks, you must delete any existing ephemeral disk storage pools before creating a new storage pool with replication enabled. If you don't need persistence, you can create a generic ephemeral volume.
+
 ## See also
 
 - [Azure Container Storage FAQ](container-storage-faq.md)

articles/storage/container-storage/use-container-storage-with-local-disk.md

@@ -14,7 +14,7 @@ ms.custom: references_regions
 [Azure Container Storage](container-storage-introduction.md) is a cloud-based volume management, deployment, and orchestration service built natively for containers. This article shows you how to configure Azure Container Storage to use Ephemeral Disk as back-end storage for your Kubernetes workloads. At the end, you'll have a pod that's using either local NVMe or temp SSD as its storage.
 
 > [!IMPORTANT]
-> Local disks are ephemeral, meaning that they're created on the local virtual machine (VM) storage and not saved to an Azure storage service. Data will be lost on these disks if you stop/deallocate your VM.
+> Local disks are ephemeral, meaning that they're created on the local virtual machine (VM) storage and not saved to an Azure storage service. Data will be lost on these disks if you stop/deallocate your VM. You can only create [Kubernetes generic ephemeral volumes](https://kubernetes.io/docs/concepts/storage/ephemeral-volumes/#generic-ephemeral-volumes) from an Ephemeral Disk storage pool. If you want to create a persistent volume, you have to enable [replication for your storage pool](#optional-create-storage-pool-with-volume-replication-nvme-only).
 
 ## Prerequisites
 
@@ -129,56 +129,13 @@ Run `kubectl get sc` to display the available storage classes. You should see a
 > [!IMPORTANT]
 > Don't use the storage class that's marked **internal**. It's an internal storage class that's needed for Azure Container Storage to work.
 
-## Create a persistent volume claim
-
-A persistent volume claim (PVC) is used to automatically provision storage based on a storage class. Follow these steps to create a PVC using the new storage class.
+## Deploy a pod with a generic ephemeral volume
 
-1. Use your favorite text editor to create a YAML manifest file such as `code acstor-pvc.yaml`.
-
-1. Paste in the following code and save the file. The PVC `name` value can be whatever you want.
-
-   ```yml
-   apiVersion: v1
-   kind: PersistentVolumeClaim
-   metadata:
-     name: ephemeralpvc
-   spec:
-     accessModes:
-       - ReadWriteOnce
-     storageClassName: acstor-ephemeraldisk # replace with the name of your storage class if different
-     resources:
-       requests:
-         storage: 100Gi
-   ```
-
-1. Apply the YAML manifest file to create the PVC.
-   
-   ```azurecli-interactive
-   kubectl apply -f acstor-pvc.yaml
-   ```
-   
-   You should see output similar to:
-   
-   ```output
-   persistentvolumeclaim/ephemeralpvc created
-   ```
-   
-   You can verify the status of the PVC by running the following command:
-   
-   ```azurecli-interactive
-   kubectl describe pvc ephemeralpvc
-   ```
-
-Once the PVC is created, it's ready for use by a pod.
-
-## Deploy a pod and attach a persistent volume
-
-Create a pod using [Fio](https://github.com/axboe/fio) (Flexible I/O Tester) for benchmarking and workload simulation, and specify a mount path for the persistent volume. For **claimName**, use the **name** value that you used when creating the persistent volume claim.
+Create a pod using [Fio](https://github.com/axboe/fio) (Flexible I/O Tester) for benchmarking and workload simulation, that uses a generic ephemeral volume.
 
 1. Use your favorite text editor to create a YAML manifest file such as `code acstor-pod.yaml`.
 
 1. Paste in the following code and save the file.
-
    ```yml
    kind: Pod
    apiVersion: v1
@@ -187,10 +144,6 @@ Create a pod using [Fio](https://github.com/axboe/fio) (Flexible I/O Tester) for
    spec:
      nodeSelector:
        acstor.azure.com/io-engine: acstor
-     volumes:
-       - name: ephemeralpv
-         persistentVolumeClaim:
-           claimName: ephemeralpvc
      containers:
        - name: fio
          image: nixery.dev/shell/fio
@@ -199,7 +152,20 @@ Create a pod using [Fio](https://github.com/axboe/fio) (Flexible I/O Tester) for
            - "1000000"
          volumeMounts:
            - mountPath: "/volume"
-             name: ephemeralpv
+             name: ephemeralvolume
+     volumes:
+       - name: ephemeralvolume
+         ephemeral:
+           volumeClaimTemplate:
+             metadata:
+               labels:
+                 type: my-ephemeral-volume
+             spec:
+               accessModes: [ "ReadWriteOnce" ]
+               storageClassName: "acstor-ephemeraldisk-nvme" # replace with the name of your storage class if different
+               resources:
+                 requests:
+                   storage: 1Gi
    ```
 
 1. Apply the YAML manifest file to deploy the pod.
@@ -214,11 +180,11 @@ Create a pod using [Fio](https://github.com/axboe/fio) (Flexible I/O Tester) for
    pod/fiopod created
    ```
 
-1. Check that the pod is running and that the persistent volume claim has been bound successfully to the pod:
+1. Check that the pod is running and that the ephemeral volume claim has been bound successfully to the pod:
 
    ```azurecli-interactive
    kubectl describe pod fiopod
-   kubectl describe pvc ephemeralpvc
+   kubectl describe pvc fiopod-ephemeralvolume
    ```
 
 1. Check fio testing to see its current status:
@@ -229,18 +195,6 @@ Create a pod using [Fio](https://github.com/axboe/fio) (Flexible I/O Tester) for
 
 You've now deployed a pod that's using Ephemeral Disk as its storage, and you can use it for your Kubernetes workloads.
 
-## Detach and reattach a persistent volume
-
-To detach a persistent volume, delete the pod that the persistent volume is attached to. Replace `<pod-name>` with the name of the pod, for example **fiopod**.
-
-```azurecli-interactive
-kubectl delete pods <pod-name>
-```
-
-To reattach a persistent volume, simply reference the persistent volume claim name in the YAML manifest file as described in [Deploy a pod and attach a persistent volume](#deploy-a-pod-and-attach-a-persistent-volume).
-
-To check which persistent volume a persistent volume claim is bound to, run `kubectl get pvc <persistent-volume-claim-name>`.
-
 ## Expand a storage pool
 
 You can expand storage pools backed by local NVMe or temp SSD to scale up quickly and without downtime. Shrinking storage pools isn't currently supported.
@@ -267,14 +221,18 @@ kubectl delete sp -n acstor <storage-pool-name>
 
 ## Optional: Create storage pool with volume replication (NVMe only)
 
-Applications that use local NVMe can leverage storage replication for improved resiliency. Replication isn't currently supported for local SSD.
+Applications that use local NVMe can leverage storage replication for improved resiliency. Replication isn't currently supported for temp SSD.
 
 Azure Container Storage currently supports three-replica and five-replica configurations. If you specify three replicas, you must have at least three nodes in your AKS cluster. If you specify five replicas, you must have at least five nodes.
 
 Follow these steps to create a storage pool using local NVMe with replication.
 
+> [!NOTE]
+> Because Ephemeral Disk storage pools consume all the available NVMe disks, you must delete any existing Ephemeral Disk local NVMe storage pools before creating a new storage pool with replication.
+
 1. Use your favorite text editor to create a YAML manifest file such as `code acstor-storagepool.yaml`.
 
+
 1. Paste in the following code and save the file. The storage pool **name** value can be whatever you want. Set replicas to 3 or 5.
 
    ```yml
@@ -308,7 +266,119 @@ Follow these steps to create a storage pool using local NVMe with replication.
    kubectl describe sp <storage-pool-name> -n acstor
    ```
 
-When the storage pool is created, Azure Container Storage will create a storage class on your behalf, using the naming convention `acstor-<storage-pool-name>`. Now you can [display the available storage classes](#display-the-available-storage-classes) and [create a persistent volume claim](#create-a-persistent-volume-claim).
+When the storage pool is created, Azure Container Storage will create a storage class on your behalf, using the naming convention `acstor-<storage-pool-name>`. Now you can [display the available storage classes](#display-the-available-storage-classes) and create a persistent volume claim.
+
+## Create a persistent volume claim
+
+A persistent volume claim (PVC) is used to automatically provision storage based on a storage class. Follow these steps to create a PVC using the new storage class.
+
+1. Use your favorite text editor to create a YAML manifest file such as `code acstor-pvc.yaml`.
+
+1. Paste in the following code and save the file. The PVC `name` value can be whatever you want.
+
+   ```yml
+   apiVersion: v1
+   kind: PersistentVolumeClaim
+   metadata:
+     name: ephemeralpvc
+   spec:
+     accessModes:
+       - ReadWriteOnce
+     storageClassName: acstor-ephemeraldisk-nvme # replace with the name of your storage class if different
+     resources:
+       requests:
+         storage: 100Gi
+   ```
+
+1. Apply the YAML manifest file to create the PVC.
+   
+   ```azurecli-interactive
+   kubectl apply -f acstor-pvc.yaml
+   ```
+   
+   You should see output similar to:
+   
+   ```output
+   persistentvolumeclaim/ephemeralpvc created
+   ```
+   
+   You can verify the status of the PVC by running the following command:
+   
+   ```azurecli-interactive
+   kubectl describe pvc ephemeralpvc
+   ```
+
+Once the PVC is created, it's ready for use by a pod.
+
+## Deploy a pod and attach a persistent volume
+
+Create a pod using [Fio](https://github.com/axboe/fio) (Flexible I/O Tester) for benchmarking and workload simulation, and specify a mount path for the persistent volume. For **claimName**, use the **name** value that you used when creating the persistent volume claim.
+
+1. Use your favorite text editor to create a YAML manifest file such as `code acstor-pod.yaml`.
+
+1. Paste in the following code and save the file.
+
+   ```yml
+   kind: Pod
+   apiVersion: v1
+   metadata:
+     name: fiopod
+   spec:
+     nodeSelector:
+       acstor.azure.com/io-engine: acstor
+     volumes:
+       - name: ephemeralpv
+         persistentVolumeClaim:
+           claimName: ephemeralpvc
+     containers:
+       - name: fio
+         image: nixery.dev/shell/fio
+         args:
+           - sleep
+           - "1000000"
+         volumeMounts:
+           - mountPath: "/volume"
+             name: ephemeralpv
+   ```
+
+1. Apply the YAML manifest file to deploy the pod.
+   
+   ```azurecli-interactive
+   kubectl apply -f acstor-pod.yaml
+   ```
+   
+   You should see output similar to the following:
+   
+   ```output
+   pod/fiopod created
+   ```
+
+1. Check that the pod is running and that the persistent volume claim has been bound successfully to the pod:
+
+   ```azurecli-interactive
+   kubectl describe pod fiopod
+   kubectl describe pvc ephemeralpvc
+   ```
+
+1. Check fio testing to see its current status:
+
+   ```azurecli-interactive
+   kubectl exec -it fiopod -- fio --name=benchtest --size=800m --filename=/volume/test --direct=1 --rw=randrw --ioengine=libaio --bs=4k --iodepth=16 --numjobs=8 --time_based --runtime=60
+   ```
+
+You've now deployed a pod that's using Ephemeral Disk as its storage, and you can use it for your Kubernetes workloads.
+
+## Detach and reattach a persistent volume
+
+To detach a persistent volume, delete the pod that the persistent volume is attached to. Replace `<pod-name>` with the name of the pod, for example **fiopod**.
+
+```azurecli-interactive
+kubectl delete pods <pod-name>
+```
+
+To reattach a persistent volume, simply reference the persistent volume claim name in the YAML manifest file as described in [Deploy a pod and attach a persistent volume](#deploy-a-pod-and-attach-a-persistent-volume).
+
+To check which persistent volume a persistent volume claim is bound to, run `kubectl get pvc <persistent-volume-claim-name>`.
 
 ## See also