You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Longhorn V2 Data Engine harnesses the power of the Storage Performance Development Kit (SPDK) to significantly reduce I/O latency while boosting IOPS and throughput. The result is a high-performance storage solution that is capable of meeting diverse workload demands.
@@ -33,23 +33,20 @@ Every node with an active Longhorn V2 Data Engine requires the following dedicat
33
33
The Longhorn V2 Data Engine currently does not support the following operations:
34
34
35
35
- Backing image creation and usage
36
-
- Live migration
37
36
- Storage network
38
37
- Volume cloning
39
38
- Volume encryption
40
39
- Volume expansion
41
40
42
41
:::
43
42
44
-
- Within the Harvester context, volumes backed by the Longhorn V2 Data Engine must be added to virtual machines as extra disks. The boot disk of each virtual machine must still be added from an image that is backed by the Longhorn V1 Data Engine.
45
-
46
-
- Harvester is unable to live-migrate virtual machines with V2 volumes attached, which means those virtual machines will be shut down during Harvester cluster upgrades. Moreover, snapshots of V2 volumes cannot be created because snapshot and restoration functionality in Harvester relies on volume cloning.
43
+
- Snapshots of V2 volumes cannot be created because snapshot and restoration functionality in Harvester relies on volume cloning.
47
44
48
45
- SSDs and other non-NVMe disks are managed using the SPDK AIO bdev driver, which does not support the unmap operation. If you are using non-NVMe disks, avoid trimming the filesystem because this results in I/O errors and paused virtual machines. For example, when creating an ext4 filesystem on a Linux virtual machine, use `mkfs.ext4 -E nodiscard /dev/vdb` (assuming `/dev/vdb` is your device path). On Windows virtual machines, you can disable trimming for NTFS by running the command `fsutil behavior set disabledeletenotify NTFS 1`.
49
46
50
47
## Using the Longhorn V2 Data Engine
51
48
52
-
The Longhorn V2 Data Engine is only available for newly created volumes. Existing volumes, virtual machine images and virtual machine root volumes will continue to use the V1 Data Engine.
49
+
The Longhorn V2 Data Engine is only available for newly created volumes and images. Existing volumes, virtual machine images and virtual machine root volumes will continue to use the V1 Data Engine.
53
50
54
51
1. On the Harvester UI, go to **Advanced** > **Settings**.
55
52
@@ -86,6 +83,41 @@ The Longhorn V2 Data Engine is only available for newly created volumes. Existin
86
83
87
84
Set the `Provisioner` to `Longhorn V2 (CSI)`.
88
85
89
-
1. Use the new StorageClass when creating new volumes (either on the **Volumes** screen or during virtual machine creation).
86
+
1. Use the new StorageClass:
87
+
- When creating new volumes (either on the **Volumes** screen or during virtual machine creation)
88
+
- When creating images on the **Images** screen
89
+
90
+
Volumes and images created using the new StorageClass are backed by the Longhorn V2 Data Engine.
91
+
92
+
## Upgrading from Harvester v1.4.x
93
+
94
+
In Harvester v1.4 (which uses Longhorn v1.7), V2 volumes did not support live migration, nor could the V2 data engine be used for virtual machine images, which meant VM boot volumes could not use the V2 Data Engine.
95
+
96
+
Starting with Harvester v1.5.0 and Longhorn v1.8.1, these limitations are removed, but only for volumes and images that are created _after_ the system is upgraded. Any V2 StorageClass created with Harvester v1.4.0 will have the migratable option set to "false", and like other StorageClass properties, this cannot be changed once set. Similarly, any existing V2 volumes will remain non-migratable after the upgrade. If you have used the V2 data engine on Harvester v1.4, and later upgrade to Harvester v1.5, you will need to create a new V2 StorageClass, which will default to having migratable set to "true". Volumes and images created using _this_ Storage Class _will_ be live-migratable.
97
+
98
+
:::info important
99
+
100
+
- If you are using the SPDK AIO bdev driver (i.e. if your disks were added using `/dev/sd*` device paths), _V2 volumes created before the upgrade will be unusable after upgrading, and cannot be recovered_. For more details see https://github.com/longhorn/longhorn/issues/10461.
101
+
102
+
- If you are using the SPDK NVMe bdev driver (i.e. your disks were added using `/dev/nvme*` device paths), V2 volumes created before the upgrade will function after the upgrade, but will continue to use the Longhorn v1.7.x engine. As mentioned above, these volumes will remain non-migratable, but it is possible to export the data and create new migratable volumes (see below for details).
103
+
104
+
- All virtual machines with V2 volumes attached need to be stopped before upgrading to Harvester v1.5.0. If there are any V2 volumes active during the upgrade, the process will stall part way through "upgrading system services". The logs of the `apply-manifests` pod will show repeated messages similar to the following:
105
+
106
+
```
107
+
instance-manager (aio)(v2) (image=longhornio/longhorn-instance-manager:v1.8.1) state is not running on node harvester-node-0, will retry...
108
+
```
109
+
110
+
Stopping all Virtual Machines that are using V2 volumes will allow the upgrade to proceed.
111
+
112
+
113
+
:::
114
+
115
+
If you have existing virtual machines with V2 non-migratable volumes attached, and you are using the SPDK NVMe bdev driver (i.e. your disks were added using `/dev/nvme*` device paths), it's possible to transition to live-migratable volumes as follows:
90
116
91
-
Volumes created using the new StorageClass are backed by the Longhorn V2 Data Engine.
117
+
1. Stop the Virtual Machine
118
+
1. For each V2 volume attached to the Virtual Machine, use the Export Image option to export that volume to an image that uses your new V2 StorageClass (with migratable set to "true"). This may take a while, depending on how much data needs to be copied.
119
+
1. Once complete, edit the Virtual Machine and on the Volumes tab:
120
+
- Remove the existing V2 volume(s).
121
+
- Use the "Add VM Image" button to add the image(s) that were exported in the previous step.
122
+
1. Start the VM. Again, this may take a while depending on how much data needs to be copied.
123
+
1. Delete the original volume(s) and the exported image(s) as these should no longer be necessary to keep around.
0 commit comments