|
| 1 | +--- |
| 2 | +title: How to configure LVM and RAID on-crypt on a Linux VM |
| 3 | +description: This article provides instructions on configuring LVM and RAID on crypt on Linux VMs. |
| 4 | +author: jofrance |
| 5 | +ms.service: security |
| 6 | +ms.topic: article |
| 7 | +ms.author: jofrance |
| 8 | +ms.date: 03/17/2020 |
| 9 | + |
| 10 | +ms.custom: seodec18 |
| 11 | + |
| 12 | +--- |
| 13 | + |
| 14 | +# How to configure LVM and RAID on-crypt |
| 15 | + |
| 16 | +This document is a step-by-step process about how to perform LVM on crypt and Raid on crypt configurations. |
| 17 | + |
| 18 | +### Environment |
| 19 | + |
| 20 | +- Linux Distributions |
| 21 | +- ADE Single Pass |
| 22 | +- ADE Dual Pass |
| 23 | + |
| 24 | + |
| 25 | +## Scenarios |
| 26 | + |
| 27 | +**This scenario is applicable to ADE dual-pass and single-pass extensions.** |
| 28 | + |
| 29 | +- Configure LVM on top of encrypted devices (LVM-on-Crypt) |
| 30 | +- Configure RAID on top of encrypted devices (RAID-on-Crypt) |
| 31 | + |
| 32 | +Once the underlying device(s) are encrypted, then you can create the LVM or RAID structures on top of that encrypted layer. |
| 33 | +The Physical Volumes (PV) are created on top of the encrypted Layer. |
| 34 | +The Physical Volumes are used to create the volume group. |
| 35 | +You create the volumes and add the required entries on /etc/fstab. |
| 36 | + |
| 37 | + |
| 38 | + |
| 39 | +In a similar way, the RAID device is created on top of the encrypted layer on the disks. A filesystem is created on top of the RAID device and added to /etc/fstab as a regular device. |
| 40 | + |
| 41 | +### Considerations |
| 42 | + |
| 43 | +The recommended method to use is LVM-on-Crypt. |
| 44 | + |
| 45 | +RAID is considered when LVM can't be used because of specific application/environment limitations. |
| 46 | + |
| 47 | +You'll use the EncryptFormatAll option, information about this feature is located here: https://docs.microsoft.com/azure/virtual-machines/linux/disk-encryption-linux#use-encryptformatall-feature-for-data-disks-on-linux-vms. |
| 48 | + |
| 49 | +While this method can be done when also encrypting the OS, we're just encrypting Data drives. |
| 50 | + |
| 51 | +This procedure assumes you already reviewed the pre-requisites mentioned here: https://docs.microsoft.com/azure/virtual-machines/linux/disk-encryption-linux and here https://docs.microsoft.com/azure/virtual-machines/linux/disk-encryption-cli-quickstart. |
| 52 | + |
| 53 | +The ADE dual pass version is on deprecation path and should no longer be use on new ADE encryptions. |
| 54 | + |
| 55 | +### Procedure |
| 56 | + |
| 57 | +When using the "on crypt" configurations, you'll be following the process outlined below: |
| 58 | + |
| 59 | +>[!NOTE] |
| 60 | +>We're using variables throughout the document, replace the values accordingly. |
| 61 | +## General steps |
| 62 | +### Deploy a VM |
| 63 | +>[!NOTE] |
| 64 | +>While this is optional we recommend you to apply this on a newly deployed VM. |
| 65 | +
|
| 66 | +PowerShell |
| 67 | +```powershell |
| 68 | +New-AzVm -ResourceGroupName ${RGNAME} ` |
| 69 | +-Name ${VMNAME} ` |
| 70 | +-Location ${LOCATION} ` |
| 71 | +-Size ${VMSIZE} ` |
| 72 | +-Image ${OSIMAGE} ` |
| 73 | +-Credential ${creds} ` |
| 74 | +-Verbose |
| 75 | +``` |
| 76 | +CLI: |
| 77 | +```bash |
| 78 | +az vm create \ |
| 79 | +-n ${VMNAME} \ |
| 80 | +-g ${RGNAME} \ |
| 81 | +--image ${OSIMAGE} \ |
| 82 | +--admin-username ${username} \ |
| 83 | +--admin-password ${password} \ |
| 84 | +-l ${LOCATION} \ |
| 85 | +--size ${VMSIZE} \ |
| 86 | +-o table |
| 87 | +``` |
| 88 | +### Attach disks to the vm: |
| 89 | +Repeat for $N number of new disks you want to attach to the VM |
| 90 | +PowerShell |
| 91 | +```powershell |
| 92 | +$storageType = 'Standard_LRS' |
| 93 | +$dataDiskName = ${VMNAME} + '_datadisk0' |
| 94 | +$diskConfig = New-AzDiskConfig -SkuName $storageType -Location $LOCATION -CreateOption Empty -DiskSizeGB 5 |
| 95 | +$dataDisk1 = New-AzDisk -DiskName $dataDiskName -Disk $diskConfig -ResourceGroupName ${RGNAME} |
| 96 | +$vm = Get-AzVM -Name ${VMNAME} -ResourceGroupName ${RGNAME} |
| 97 | +$vm = Add-AzVMDataDisk -VM $vm -Name $dataDiskName -CreateOption Attach -ManagedDiskId $dataDisk1.Id -Lun 0 |
| 98 | +Update-AzVM -VM ${VM} -ResourceGroupName ${RGNAME} |
| 99 | +``` |
| 100 | +CLI: |
| 101 | +```bash |
| 102 | +az vm disk attach \ |
| 103 | +-g ${RGNAME} \ |
| 104 | +--vm-name ${VMNAME} \ |
| 105 | +--name ${VMNAME}datadisk1 \ |
| 106 | +--size-gb 5 \ |
| 107 | +--new \ |
| 108 | +-o table |
| 109 | +``` |
| 110 | +### Verify the disks are attached to the VM: |
| 111 | +PowerShell: |
| 112 | +```powershell |
| 113 | +$VM = Get-AzVM -ResourceGroupName ${RGNAME} -Name ${VMNAME} |
| 114 | +$VM.StorageProfile.DataDisks | Select-Object Lun,Name,DiskSizeGB |
| 115 | +``` |
| 116 | + |
| 117 | +CLI: |
| 118 | +```bash |
| 119 | +az vm show -g ${RGNAME} -n ${VMNAME} --query storageProfile.dataDisks -o table |
| 120 | +``` |
| 121 | + |
| 122 | +Portal: |
| 123 | + |
| 124 | +OS: |
| 125 | +```bash |
| 126 | +lsblk |
| 127 | +``` |
| 128 | + |
| 129 | +### Configure the disks to be encrypted |
| 130 | +This configuration is done that the operating system level, the corresponding disks are configured for a traditional ADE encryption: |
| 131 | + |
| 132 | +Filesystems are created on top of the disks. |
| 133 | + |
| 134 | +Temporary mount points are created to mount the filesystems. |
| 135 | + |
| 136 | +The Filesystems are configured on /etc/fstab to be mounted at boot time. |
| 137 | + |
| 138 | +Check the device letter assigned to the new disks, on this example we're using four data disks |
| 139 | + |
| 140 | +```bash |
| 141 | +lsblk |
| 142 | +``` |
| 143 | + |
| 144 | + |
| 145 | +### Create a filesystem on top of each disk. |
| 146 | +This command iterates an ext4 filesystem creation on each disk defined on the "in" part of the "for" cycle. |
| 147 | +```bash |
| 148 | +for disk in c d e f; do echo mkfs.ext4 -F /dev/sd${disk}; done |bash |
| 149 | +``` |
| 150 | + |
| 151 | +Find the UUID of the filesystems recently created, create a temporary folder to mount it, add the corresponding entries on /etc/fstab and mount all the filesystems. |
| 152 | + |
| 153 | +This command also iterates on each disk defined on the "in" part of the "for" cycle: |
| 154 | +```bash |
| 155 | +for disk in c d e f; do diskuuid="$(blkid -s UUID -o value /dev/sd${disk})"; \ |
| 156 | +mkdir /tempdata${disk}; \ |
| 157 | +echo "UUID=${diskuuid} /tempdata${disk} ext4 defaults,nofail 0 0" >> /etc/fstab; \ |
| 158 | +mount -a; \ |
| 159 | +done |
| 160 | +``` |
| 161 | +### Verify the disks are mounted properly: |
| 162 | +```bash |
| 163 | +lsblk |
| 164 | +``` |
| 165 | + |
| 166 | +And configured: |
| 167 | +```bash |
| 168 | +cat /etc/fstab |
| 169 | +``` |
| 170 | + |
| 171 | +### Encrypt the data disks: |
| 172 | +PowerShell using KEK: |
| 173 | +```powershell |
| 174 | +$sequenceVersion = [Guid]::NewGuid() |
| 175 | +Set-AzVMDiskEncryptionExtension -ResourceGroupName $RGNAME ` |
| 176 | +-VMName ${VMNAME} ` |
| 177 | +-DiskEncryptionKeyVaultUrl $diskEncryptionKeyVaultUrl ` |
| 178 | +-DiskEncryptionKeyVaultId $KeyVaultResourceId ` |
| 179 | +-KeyEncryptionKeyUrl $keyEncryptionKeyUrl ` |
| 180 | +-KeyEncryptionKeyVaultId $KeyVaultResourceId ` |
| 181 | +-VolumeType 'DATA' ` |
| 182 | +-EncryptFormatAll ` |
| 183 | +-SequenceVersion $sequenceVersion ` |
| 184 | +-skipVmBackup; |
| 185 | +``` |
| 186 | +CLI using KEK: |
| 187 | +```bash |
| 188 | +az vm encryption enable \ |
| 189 | +--resource-group ${RGNAME} \ |
| 190 | +--name ${VMNAME} \ |
| 191 | +--disk-encryption-keyvault ${KEYVAULTNAME} \ |
| 192 | +--key-encryption-key ${KEYNAME} \ |
| 193 | +--key-encryption-keyvault ${KEYVAULTNAME} \ |
| 194 | +--volume-type "DATA" \ |
| 195 | +--encrypt-format-all \ |
| 196 | +-o table |
| 197 | +``` |
| 198 | +### Verify the encryption status |
| 199 | + |
| 200 | +Continue to the next step only when all the disks are encrypted. |
| 201 | + |
| 202 | +PowerShell: |
| 203 | +```powershell |
| 204 | +Get-AzVmDiskEncryptionStatus -ResourceGroupName ${RGNAME} -VMName ${VMNAME} |
| 205 | +``` |
| 206 | + |
| 207 | +CLI: |
| 208 | +```bash |
| 209 | +az vm encryption show -n ${VMNAME} -g ${RGNAME} -o table |
| 210 | +``` |
| 211 | + |
| 212 | +Portal: |
| 213 | + |
| 214 | +OS Level: |
| 215 | +```bash |
| 216 | +lsblk |
| 217 | +``` |
| 218 | + |
| 219 | + |
| 220 | +The extension will add the filesystems to "/var/lib/azure_disk_encryption_config/azure_crypt_mount" (an old encryption) or added to "/etc/crypttab" (new encryptions). |
| 221 | + |
| 222 | +Do not modify any of these files. |
| 223 | + |
| 224 | +This file is going to be taking care of activating these disks during the boot process so they can be later used by LVM or RAID. |
| 225 | + |
| 226 | +Do not worry about the mount points on this file, as ADE will lose the ability to get the disks mounted as a normal file system after we do create a physical volume or a raid device on top of those encrypted devices(which will get rid of the file system format we used during the preparation process). |
| 227 | +### Remove the temp folders and temp fstab entries |
| 228 | +You unmount the filesystems on the disks that will be used as part of LVM |
| 229 | +```bash |
| 230 | +for disk in c d e f; do umount /tempdata${disk}; done |
| 231 | +``` |
| 232 | +And remove the /etc/fstab entries: |
| 233 | +```bash |
| 234 | +vi /etc/fstab |
| 235 | +``` |
| 236 | +### Verify that the disks are not mounted and that the entries on /etc/fstab were removed |
| 237 | +```bash |
| 238 | +lsblk |
| 239 | +``` |
| 240 | + |
| 241 | +And configured: |
| 242 | +```bash |
| 243 | +cat /etc/fstab |
| 244 | +``` |
| 245 | + |
| 246 | +## For LVM-on-crypt |
| 247 | +Now that the underlying disks are encrypted, you can proceed to create the LVM structures. |
| 248 | + |
| 249 | +Instead of using the device name, use the /dev/mapper paths for each of the disks to create a physical volume (on the crypt layer on top of the disk not on the disk itself). |
| 250 | +### Configure LVM on top of the encrypted layers |
| 251 | +#### Create the physical volumes |
| 252 | +You'll get a warning asking if it's OK to wipe out the filesystem signature. |
| 253 | + |
| 254 | +You may continue by entering 'y' or use the echo "y" as shown: |
| 255 | +```bash |
| 256 | +echo "y" | pvcreate /dev/mapper/c49ff535-1df9-45ad-9dad-f0846509f052 |
| 257 | +echo "y" | pvcreate /dev/mapper/6712ad6f-65ce-487b-aa52-462f381611a1 |
| 258 | +echo "y" | pvcreate /dev/mapper/ea607dfd-c396-48d6-bc54-603cf741bc2a |
| 259 | +echo "y" | pvcreate /dev/mapper/4159c60a-a546-455b-985f-92865d51158c |
| 260 | +``` |
| 261 | + |
| 262 | +>[!NOTE] |
| 263 | +>The /dev/mapper/device names here need to be replaced for your actual values based on the output of lsblk. |
| 264 | +#### Verify the physical volumes information |
| 265 | +```bash |
| 266 | +pvs |
| 267 | +``` |
| 268 | + |
| 269 | +#### Create the volume group |
| 270 | +Create the VG using the same devices already initialized |
| 271 | +```bash |
| 272 | +vgcreate vgdata /dev/mapper/ |
| 273 | +``` |
| 274 | +### Check the volume group information |
| 275 | +```bash |
| 276 | +vgdisplay -v vgdata |
| 277 | +``` |
| 278 | +```bash |
| 279 | +pvs |
| 280 | +``` |
| 281 | + |
| 282 | +#### Create logical volumes |
| 283 | +```bash |
| 284 | +lvcreate -L 10G -n lvdata1 vgdata |
| 285 | +lvcreate -L 7G -n lvdata2 vgdata |
| 286 | +``` |
| 287 | +#### Check the logical volumes created |
| 288 | +```bash |
| 289 | +lvdisplay |
| 290 | +lvdisplay vgdata/lvdata1 |
| 291 | +lvdisplay vgdata/lvdata2 |
| 292 | +``` |
| 293 | + |
| 294 | +#### Create filesystems on top of the logical volume(s) structure(s) |
| 295 | +```bash |
| 296 | +echo "yes" | mkfs.ext4 /dev/vgdata/lvdata1 |
| 297 | +echo "yes" | mkfs.ext4 /dev/vgdata/lvdata2 |
| 298 | +``` |
| 299 | +#### Create the mount points for the new filesystems |
| 300 | +```bash |
| 301 | +mkdir /data0 |
| 302 | +mkdir /data1 |
| 303 | +``` |
| 304 | +#### Add the new file systems to /etc/fstab and mount them |
| 305 | +```bash |
| 306 | +echo "/dev/mapper/vgdata-lvdata1 /data0 ext4 defaults,nofail 0 0" >>/etc/fstab |
| 307 | +echo "/dev/mapper/vgdata-lvdata2 /data1 ext4 defaults,nofail 0 0" >>/etc/fstab |
| 308 | +mount -a |
| 309 | +``` |
| 310 | +#### Verify that the new filesystems are mounted |
| 311 | +```bash |
| 312 | +lsblk -fs |
| 313 | +df -h |
| 314 | +``` |
| 315 | + |
| 316 | +On this variation of lsblk, we're listing the devices showing the dependencies on reverse order, this option helps to identify the devices grouped by the logical volume instead of the original /dev/sd[disk] device names. |
| 317 | + |
| 318 | +Important: Make sure the "nofail" option is added to the mount point options of the LVM volumes created on top of an ADE encrypted device. Is important to avoid the OS from getting stuck during the boot process (or in maintenance mode). |
| 319 | + |
| 320 | +The encrypted disk are unlock at the end of the boot process, the LVM volumes and file systems will be automatically mounted. |
| 321 | + |
| 322 | +If the nofail option isn't used, the OS will never get into the stage where ADE is started, and the data disk(s) are unlocked and mounted. |
| 323 | + |
| 324 | +You can test rebooting the VM and validating the file systems are also automatically getting mounted after boot time. |
| 325 | + |
| 326 | +Take under consideration that this process may take several minutes depending on the number of file systems and the sizes |
| 327 | +#### Reboot the VM and verify after reboot |
| 328 | +```bash |
| 329 | +shutdown -r now |
| 330 | +``` |
| 331 | +```bash |
| 332 | +lsblk |
| 333 | +df -h |
| 334 | +``` |
| 335 | +## For RAID-on-Crypt |
| 336 | +Now the underlying disks are encrypted you can continue to create the RAID structures, same as LVM, instead of using the device name, use the /dev/mapper paths for each of the disks. |
| 337 | + |
| 338 | +#### Configure RAID on top of the encrypted layer of the disks |
| 339 | +```bash |
| 340 | +mdadm --create /dev/md10 \ |
| 341 | +--level 0 \ |
| 342 | +--raid-devices=4 \ |
| 343 | +/dev/mapper/c49ff535-1df9-45ad-9dad-f0846509f052 \ |
| 344 | +/dev/mapper/6712ad6f-65ce-487b-aa52-462f381611a1 \ |
| 345 | +/dev/mapper/ea607dfd-c396-48d6-bc54-603cf741bc2a \ |
| 346 | +/dev/mapper/4159c60a-a546-455b-985f-92865d51158c |
| 347 | +``` |
| 348 | + |
| 349 | +>[!NOTE] |
| 350 | +>The /dev/mapper/device names here need to be replaced for your actual values based on the output of lsblk. |
| 351 | +#### Check/monitor the RAID creation: |
| 352 | +```bash |
| 353 | +watch -n1 cat /proc/mdstat |
| 354 | +mdadm --examine /dev/mapper/[] |
| 355 | +mdadm --detail /dev/md10 |
| 356 | +``` |
| 357 | + |
| 358 | +#### Create a filesystem on top of the new Raid device: |
| 359 | +```bash |
| 360 | +mkfs.ext4 /dev/md10 |
| 361 | +``` |
| 362 | +Create a new mountpoint for the filesystem, add the new file system to /etc/fstab, and mount it |
| 363 | +```bash |
| 364 | +for device in md10; do diskuuid="$(blkid -s UUID -o value /dev/${device})"; \ |
| 365 | +mkdir /raiddata; \ |
| 366 | +echo "UUID=${diskuuid} /raiddata ext4 defaults,nofail 0 0" >> /etc/fstab; \ |
| 367 | +mount -a; \ |
| 368 | +done |
| 369 | +``` |
| 370 | +Verify that the new filesystems are mounted |
| 371 | +```bash |
| 372 | +lsblk -fs |
| 373 | +df -h |
| 374 | +``` |
| 375 | + |
| 376 | + |
| 377 | +Important: Make sure the "nofail" option is added to the mount point options of the RAID volumes created on top of an ADE encrypted device. |
| 378 | + |
| 379 | +Is very important to avoid the OS from getting stuck during the boot process (or in maintenance mode). |
| 380 | + |
| 381 | +The encrypted disk will be unlock at the end of the boot process and the RAID volumes and file systems will be automatically mounted until they're unlocked by ADE, if the nofail option is not used. |
| 382 | + |
| 383 | +The OS will never get into the stage where ADE is started, and the data disks are unlocked and mounted. |
| 384 | + |
| 385 | +You can test rebooting the VM and validating the file systems are also automatically getting mounted after boot time. Take under consideration that this process may take several minutes depending on the number of file systems and the sizes |
| 386 | +```bash |
| 387 | +shutdown -r now |
| 388 | +``` |
| 389 | +And when you can log in: |
| 390 | +```bash |
| 391 | +lsblk |
| 392 | +df -h |
| 393 | +``` |
| 394 | +## Next steps |
| 395 | + |
| 396 | +- [Azure Disk Encryption troubleshooting](disk-encryption-troubleshooting.md) |
| 397 | + |
0 commit comments