Linstor: encryption support

[rp-]

Description

This PR main purpose is adding encryption support for Linstor, as Linstor handles the encryption layer (LUKS)
to allow DRBD running on top of it, we needed a new mode for encryption in CloudStack.
So that CloudStack knows the volume is encrypted, but the encryption/deencryption is handled by the primary storage.
So qemu only gets the final block device path and opening/closing is handled by Linstor.

Additionally there are a few commits for cleanups and a few unittests for Linstor code.
For this to work at least Linstor 1.30.x is needed, as the new cloning code is needed.

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)
• build/CI
• test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Screenshots (if appropriate):

How Has This Been Tested?

Linstor cluster with encrypted disk offering.

How did you try to break this feature and the system with this change?

[codecov]

Codecov Report

Attention: Patch coverage is 50.00000% with 40 lines in your changes missing coverage. Please review.

Project coverage is 15.13%. Comparing base (0a77eb7) to head (ce3c93c).
Report is 26 commits behind head on 4.19.

Files with missing lines	Patch %	Lines
...tore/driver/LinstorPrimaryDataStoreDriverImpl.java	38.88%	21 Missing and 1 partial ⚠️
...e/wrapper/LinstorBackupSnapshotCommandWrapper.java	0.00%	10 Missing ⚠️
api/src/main/java/com/cloud/storage/Storage.java	89.65%	3 Missing ⚠️
...ervisor/kvm/resource/LibvirtComputingResource.java	0.00%	0 Missing and 2 partials ⚠️
...ud/hypervisor/kvm/storage/KVMStorageProcessor.java	0.00%	2 Missing ⚠️
.../hypervisor/kvm/storage/LinstorStorageAdaptor.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               4.19   #10126      +/-   ##
============================================
- Coverage     15.14%   15.13%   -0.01%     
- Complexity    11283    11290       +7     
============================================
  Files          5408     5408              
  Lines        473823   474321     +498     
  Branches      57824    57848      +24     
============================================
+ Hits          71764    71804      +40     
- Misses       394037   394486     +449     
- Partials       8022     8031       +9     

Flag	Coverage Δ
uitests	4.29% <ø> (ø)
unittests	15.85% <50.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[rp-]

@blueorangutan package

[blueorangutan]

@rp- a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11857

[blueorangutan]

[SF] Trillian test result (tid-11960)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 45339 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10126-t11960-kvm-ol8.zip
Smoke tests completed. 132 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_secure_vm_migration	Error	266.89	test_vm_life_cycle.py

[github-actions]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[DaanHoogland]

@slavkap, can you please have a quick look at this? It does not seem too exiting from a generic storage point of view, but I'd like to be sure.

[slavkap]

code LGTM, but I cannot test it in the coming days

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12239

[DaanHoogland]

@blueorangutan test

[blueorangutan]

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-12214)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 46249 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10126-t12214-kvm-ol8.zip
Smoke tests completed. 133 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

[DaanHoogland]

I am not sure how we can test this further . @rg9975 @slavkap , can you spend a regression on your storage solutions?
@borisstoyanov any other ideas?

[slavkap]

@DaanHoogland, if it's not late I'll try to test it on Monday

[DaanHoogland]

@DaanHoogland, if it's not late I'll try to test it on Monday

@slavkap it is never too late of course, but for 19.2 we would need another RC. We can of course decide to have that if you storage people think your fixes are needed. cc @rp- @rg9975

[DaanHoogland]

@slavkap , you have tested?

[slavkap]

sorry, @DaanHoogland, still not. I'm experiencing some problems with the setup and trying to find out what's happening

[slavkap]

@DaanHoogland, all looks good with the main StorPool functionality

[DaanHoogland]

@rg9975 you want to test as well?
Smoke tests (i.e. NFS) looks good.

[rg9975]

Approved