Build: drop EL7 support, support JRE17 for packages and sonar check

[weizhouapache]

Description

This PR fixes the issue with sonar check

Error:  Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.9.1.2184:sonar (default-cli) on project cloudstack:
Error:
Error:  The version of Java (11.0.22) used to run this analysis is deprecated, and SonarCloud no longer supports it. Please upgrade to Java 17 or later.
Error:  You can find more information here: https://docs.sonarsource.com/sonarcloud/appendices/scanner-environment/

main changes

• Support build/packaging using JDK17
• Still supports JDK11 for building
• Support JRE17 for use in production installation
• Drop EL7 support

The community packages will be still packaged using JDK11.
If uses want, they can build by JDK17 as well.

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)
• build/CI

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 28.40%. Comparing base (d5d8b6d) to head (b16d00a).
Report is 106 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #8609      +/-   ##
============================================
- Coverage     30.82%   28.40%   -2.42%     
+ Complexity    34015    30942    -3073     
============================================
  Files          5344     5344              
  Lines        375411   375411              
  Branches      54572    54572              
============================================
- Hits         115710   106646    -9064     
- Misses       244434   254531   +10097     
+ Partials      15267    14234    -1033     

Flag	Coverage Δ
simulator-marvin-tests	21.29% <ø> (-3.33%)	⬇️
uitests	4.38% <ø> (ø)
unit-tests	16.56% <ø> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[weizhouapache]

Just discussed with @vishesh92

it requires lots of changes to support packaging cloudstack with jre17
cc @rohityadavcloud @DaanHoogland

[github-actions]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 15.54%. Comparing base (7214c13) to head (97be6a6).
Report is 49 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #8609      +/-   ##
============================================
- Coverage     15.54%   15.54%   -0.01%     
- Complexity    11976    11977       +1     
============================================
  Files          5495     5495              
  Lines        481076   481076              
  Branches      60094    58769    -1325     
============================================
- Hits          74780    74777       -3     
- Misses       398033   398037       +4     
+ Partials       8263     8262       -1     

Flag	Coverage Δ
uitests	4.21% <ø> (ø)
unittests	16.30% <ø> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[weizhouapache]

@blueorangutan package

[blueorangutan]

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9525

[weizhouapache]

@blueorangutan package

[blueorangutan]

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9528

[weizhouapache]

@blueorangutan test rocky8 kvm-rocky8

[rohityadavcloud]

@blueorangutan package

[blueorangutan]

@rohityadavcloud a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10506

[github-actions]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[weizhouapache]

@blueorangutan package

[blueorangutan]

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10512

[rohityadavcloud]

@blueorangutan test matrix

[blueorangutan]

@rohityadavcloud a [SL] Trillian-Jenkins matrix job (EL8 mgmt + EL8 KVM, Ubuntu22 mgmt + Ubuntu22 KVM, EL8 mgmt + VMware 7.0u3, EL9 mgmt + XCP-ng 8.2 ) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-11016)
Environment: xcpng82 (x2), Advanced Networking with Mgmt server a9
Total time taken: 70098 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8609-t11016-xcpng82.zip
Smoke tests completed. 135 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_condensed_drs_algorithm	Failure	187.04	test_cluster_drs.py
test_02_balanced_drs_algorithm	Failure	166.70	test_cluster_drs.py
test_01_non_strict_host_anti_affinity	Error	232.32	test_nonstrict_affinity_group.py
test_02_non_strict_host_affinity	Error	116.83	test_nonstrict_affinity_group.py

[blueorangutan]

[SF] Trillian test result (tid-11023)
Environment: vmware-70u3 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 54529 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8609-t11023-vmware-70u3.zip
Smoke tests completed. 136 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_02_restore_vm_with_disk_offering	Error	58.21	test_restore_vm.py
test_03_restore_vm_with_disk_offering_custom_size	Error	55.11	test_restore_vm.py

[blueorangutan]

[SF] Trillian test result (tid-11022)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 54937 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8609-t11022-kvm-ol8.zip
Smoke tests completed. 137 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

[blueorangutan]

[SF] Trillian test result (tid-11024)
Environment: kvm-ubuntu22 (x2), Advanced Networking with Mgmt server u22
Total time taken: 56082 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8609-t11024-kvm-ubuntu22.zip
Smoke tests completed. 137 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

[rohityadavcloud]

@blueorangutan package

[blueorangutan]

@rohityadavcloud a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10563

[weizhouapache]

[SF] Trillian test result (tid-11016) Environment: xcpng82 (x2), Advanced Networking with Mgmt server a9 Total time taken: 70098 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8609-t11016-xcpng82.zip Smoke tests completed. 135 look OK, 2 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_01_condensed_drs_algorithm Failure 187.04 test_cluster_drs.py
test_02_balanced_drs_algorithm Failure 166.70 test_cluster_drs.py
test_01_non_strict_host_anti_affinity Error 232.32 test_nonstrict_affinity_group.py
test_02_non_strict_host_affinity Error 116.83 test_nonstrict_affinity_group.py

these are known issues with tests on xcpng82
see #8586 (due to migration issue)

[weizhouapache]

test_02_restore_vm_with_disk_offering

error while deployed vm

======================================================================
ERROR: Test restore virtual machine
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/marvin/tests/smoke/test_restore_vm.py", line 111, in test_02_restore_vm_with_disk_offering
    virtual_machine.restore(self.apiclient, self.template_t2.id, self.disk_offering.id, expunge=True)
  File "/usr/local/lib/python3.6/site-packages/marvin/lib/base.py", line 799, in restore
    return apiclient.restoreVirtualMachine(cmd)
  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", line 951, in restoreVirtualMachine
    response = self.connection.marvinRequest(command, response_type=response, method=method)
  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 381, in marvinRequest
    raise e
  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 376, in marvinRequest
    raise self.__lastError
  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 105, in __poll
    % async_response)
Exception: Job failed: {accountid : '96392970-5173-11ef-b1de-1e00d300016a', account : 'admin', domainid : '5dda90d4-5173-11ef-b1de-1e00d300016a', domainpath : 'ROOT', userid : '9639e21e-5173-11ef-b1de-1e00d300016a', cmd : 'org.apache.cloudstack.api.command.admin.vm.RestoreVMCmdByAdmin', jobstatus : 2, jobprocstatus : 0, jobresultcode : 530, jobresulttype : 'object', jobresult : {errorcode : 530, errortext : 'Unable to start VM with specified idUnable to start a VM [15b87163-fd78-41d9-89d0-3432b2db71da] due to [Unable to create a deployment for VM instance {"id":275,"instanceName":"i-2-275-VM","type":"User","uuid":"15b87163-fd78-41d9-89d0-3432b2db71da"}].'}, jobinstancetype : 'VirtualMachine', jobinstanceid : '15b87163-fd78-41d9-89d0-3432b2db71da', created : '2024-08-03T16:13:38+0000', completed : '2024-08-03T16:13:56+0000', jobid : 'fbd3f1e4-a137-4817-be01-ac35768d59cd'}

logs

2024-08-03T16:13:51,281 INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-132:[ctx-7b634276, 10.0.33.4, job-2476/job-2477, cmd: StartCommand]) (logid:fbd3f1e4) Found existing disk info from volume path: i-2-275-VM
2024-08-03T16:13:51,281 WARN  [c.c.h.v.r.VmwareResource] (DirectAgent-132:[ctx-7b634276, 10.0.33.4, job-2476/job-2477, cmd: StartCommand]) (logid:fbd3f1e4) Disk chain length for the VM is greater than one, this is not supported
2024-08-03T16:13:51,281 INFO  [c.c.h.v.u.VmwareHelper] (DirectAgent-132:[ctx-7b634276, 10.0.33.4, job-2476/job-2477, cmd: StartCommand]) (logid:fbd3f1e4) [ignored]failed to get message for exception: Unsupported VM disk chain length: 2
2024-08-03T16:13:51,281 ERROR [c.c.h.v.r.VmwareResource] (DirectAgent-132:[ctx-7b634276, 10.0.33.4, job-2476/job-2477, cmd: StartCommand]) (logid:fbd3f1e4) StartCommand failed due to [Exception: com.cloud.utils.exception.CloudRuntimeException
Message: Unsupported VM disk chain length: 2
]. com.cloud.utils.exception.CloudRuntimeException: Unsupported VM disk chain length: 2
    at com.cloud.hypervisor.vmware.resource.VmwareResource.resizeRootDiskOnVMStart(VmwareResource.java:3101)
    at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:2356)
    at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:564)
    at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:299)
    at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
    at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:840)

2024-08-03T16:13:51,282 DEBUG [c.c.a.m.D.Task] (DirectAgent-132:[ctx-7b634276]) (logid:fbd3f1e4) Seq 2-5621899709841932466: Response Received:
2024-08-03T16:13:51,283 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (DirectAgent-132:[ctx-7b634276]) (logid:fbd3f1e4) Seq 2-5621899709841932466: MgmtId 32986959446759 : Resp: Routing to peer

this happens in the two tests on vmware (67u3, 70u3)
need check

update: this is consistent with health check PR, which seems to happen since beginning #9008

[vladimirpetrov]

LGTM based on manual testing, tried several upgrades from 4.17, 4.18 and 4.19, JRE11 to 17, KVM and VMWare and the most common operations. No issues found.