Skip to content

VM cross cluster migration not working #10078

New issue
New issue
Closed
#10266
Closed
VM cross cluster migration not working#10078
#10266
Assignees
weizhouapache
Labels
component:migrationtype:bug
Milestone
4.19.3
@ahmedali6

Description

@ahmedali6
ahmedali6
opened on Dec 10, 2024
ISSUE TYPE
  • Bug Report
COMPONENT NAME
API
CLOUDSTACK VERSION
4.19.1.2
CONFIGURATION

Advanced Networking

OS / ENVIRONMENT

OS: Red Hat 9
Fresh installation of CLoudStack

SUMMARY

When trying to migrate a vm between clusters the migration fails

STEPS TO REPRODUCE
Create a Zone
Create a Cluster A
Create a Cluster B
Create a cluster-wide primary storage for cluster A
Create a cluster-wide primary storage for cluster B
Create a VM in cluster A
Migrate the VM with storage to cluster B
EXPECTED RESULTS
The instance to be migrated to cluster B
ACTUAL RESULTS
The migration fails and a volume create and stuck in `Migrating` state

image

image

Management server logs

2024-12-10 09:27:04,052 DEBUG [c.c.a.ApiServlet] (qtp1390913202-391:[ctx-434f8d2a, ctx-fb760dc7]) (logid:1efe446c) ===END===  10.0.0.43 -- GET  command=readyForShutdown&response=json&sessionkey=oxYEmliCuKzvEtObrqHQyNyP678
2024-12-10 09:27:04,201 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-4:[]) (logid:) SeqA 2-2287: Seq 2-2287:  { Cmd , MgmtId: -1, via: 2, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":"2","_loadInfo":"{
  "connections": [],
  "removedSessions": []
}","wait":"0","bypassHostMaintenance":"false"}}] }
2024-12-10 09:27:04,212 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-4:[]) (logid:) SeqA 2-: Sending 2287
2024-12-10 09:27:04,234 DEBUG [c.c.a.t.Request] (AgentManager-Handler-6:[]) (logid:) Seq 4-7171982406587515702: Processing:  { Ans: , MgmtId: 52231313017, via: 4, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":"false","details":"com.cloud.utils.exception.CloudRuntimeException: Can't find volume:88ea6edb-5cf2-413b-abc3-9be3b6177813
	at com.cloud.hypervisor.kvm.storage.LibvirtStoragePool.getPhysicalDisk(LibvirtStoragePool.java:159)
	at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getPhysicalDisk(KVMStoragePoolManager.java:369)
	at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyTargetsCommandWrapper.execute(LibvirtModifyTargetsCommandWrapper.java:57)
	at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyTargetsCommandWrapper.execute(LibvirtModifyTargetsCommandWrapper.java:38)
	at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
	at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1945)
	at com.cloud.agent.Agent.processRequest(Agent.java:686)
	at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1109)
	at com.cloud.utils.nio.Task.call(Task.java:83)
	at com.cloud.utils.nio.Task.call(Task.java:29)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)
","wait":"0","bypassHostMaintenance":"false"}}] }
2024-12-10 09:27:04,235 DEBUG [c.c.a.t.Request] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) Seq 4-7171982406587515702: Received:  { Ans: , MgmtId: 52231313017, via: 4(localhost.localdomain), Ver: v1, Flags: 10, { Answer } }
2024-12-10 09:27:04,235 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) Details from executing class com.cloud.agent.api.ModifyTargetsCommand: com.cloud.utils.exception.CloudRuntimeException: Can't find volume:88ea6edb-5cf2-413b-abc3-9be3b6177813
	at com.cloud.hypervisor.kvm.storage.LibvirtStoragePool.getPhysicalDisk(LibvirtStoragePool.java:159)
	at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getPhysicalDisk(KVMStoragePoolManager.java:369)
	at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyTargetsCommandWrapper.execute(LibvirtModifyTargetsCommandWrapper.java:57)
	at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyTargetsCommandWrapper.execute(LibvirtModifyTargetsCommandWrapper.java:38)
	at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
	at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1945)
	at com.cloud.agent.Agent.processRequest(Agent.java:686)
	at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1109)
	at com.cloud.utils.nio.Task.call(Task.java:83)
	at com.cloud.utils.nio.Task.call(Task.java:29)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

2024-12-10 09:27:04,267 DEBUG [o.a.c.s.v.VolumeServiceImpl] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) Failed to copy volume java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be cast to class com.cloud.agent.api.ModifyTargetsAnswer (com.cloud.agent.api.Answer and com.cloud.agent.api.ModifyTargetsAnswer are in unnamed module of loader 'app')
	at org.apache.cloudstack.storage.motion.StorageSystemDataMotionStrategy.sendModifyTargetsCommand(StorageSystemDataMotionStrategy.java:2401)
	at org.apache.cloudstack.storage.motion.StorageSystemDataMotionStrategy.connectHostToVolume(StorageSystemDataMotionStrategy.java:2366)
	at org.apache.cloudstack.storage.motion.StorageSystemDataMotionStrategy.copyAsync(StorageSystemDataMotionStrategy.java:2075)
	at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:136)
	at org.apache.cloudstack.storage.volume.VolumeServiceImpl.migrateVolumes(VolumeServiceImpl.java:2334)
	at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.migrateVolumes(VolumeOrchestrator.java:1438)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
	at jdk.proxy3/jdk.proxy3.$Proxy266.migrateVolumes(Unknown Source)
	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:3295)
	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:5515)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:106)
	at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5591)
	at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:99)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:652)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:600)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

2024-12-10 09:27:04,269 ERROR [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) Failed to migrate VM [VM instance {"id":3,"instanceName":"i-2-3-VM","type":"User","uuid":"57df225e-a02f-4249-b731-fd48767a442f"}] along with its volumes due to [java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be cast to class com.cloud.agent.api.ModifyTargetsAnswer (com.cloud.agent.api.Answer and com.cloud.agent.api.ModifyTargetsAnswer are in unnamed module of loader 'app')].
2024-12-10 09:27:04,270 INFO  [c.c.v.ClusteredVirtualMachineManagerImpl] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) Migration was unsuccessful.  Cleaning up: VM instance {"id":3,"instanceName":"i-2-3-VM","type":"User","uuid":"57df225e-a02f-4249-b731-fd48767a442f"}
2024-12-10 09:27:04,305 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) Releasing [1] volumes for VM [VM instance {"id":3,"instanceName":"i-2-3-VM","type":"User","uuid":"57df225e-a02f-4249-b731-fd48767a442f"}] from host [Host {"id":4,"name":"localhost.localdomain","type":"Routing","uuid":"f20e35b7-a4b1-445a-bb3f-fd267760eb57"}].
2024-12-10 09:27:04,323 WARN  [c.c.a.AlertManagerImpl] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) alertType=[17] dataCenterId=[1] podId=[1] clusterId=[null] message=[Unable to migrate vm i-2-3-VM from host localhost.localdomain in zone zone-1 and pod zone-1].
2024-12-10 09:27:04,327 DEBUG [c.c.a.AlertManagerImpl] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) Have already sent: 1 emails for alert type '17' -- skipping send email
2024-12-10 09:27:04,331 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) VM instance {"id":3,"instanceName":"i-2-3-VM","type":"User","uuid":"57df225e-a02f-4249-b731-fd48767a442f"} is a [User], returning null for control Nic IP.
2024-12-10 09:27:04,351 DEBUG [c.c.a.m.ClusteredAgentAttache] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) Seq 4-7171982406587515719: Routed from 52231313017
2024-12-10 09:27:04,352 DEBUG [c.c.a.t.Request] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) Seq 4-7171982406587515719: Sending  { Cmd , MgmtId: 52231313017, via: 4(localhost.localdomain), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":"false","forceStop":"false","vlanToPersistenceMap":{"650":"true"},"volumesToDisconnect":[],"vmName":"i-2-3-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}] }
2024-12-10 09:27:04,416 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) VM instance {"id":3,"instanceName":"i-2-3-VM","type":"User","uuid":"57df225e-a02f-4249-b731-fd48767a442f"} state transited from [Migrating] to [Running] with event [OperationFailed]. VM's original host: Host {"id":1,"name":"localhost.localdomain","type":"Routing","uuid":"b1df0bbe-ab48-4d5e-8a49-af9adc95ab3a"}, new host: Host {"id":1,"name":"localhost.localdomain","type":"Routing","uuid":"b1df0bbe-ab48-4d5e-8a49-af9adc95ab3a"}, host before state transition: Host {"id":4,"name":"localhost.localdomain","type":"Routing","uuid":"f20e35b7-a4b1-445a-bb3f-fd267760eb57"}
2024-12-10 09:27:04,436 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) Hosts's actual total CPU: 9200 and CPU after applying overprovisioning: 9200
2024-12-10 09:27:04,437 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) Hosts's actual total RAM: (6.98 GB) 7492751360 and RAM after applying overprovisioning: (6.98 GB) 7492751360
2024-12-10 09:27:04,445 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) release cpu from host: 4, old used: 1000,reserved: 0, actual total: 9200, total with overprovisioning: 9200; new used: 500,reserved:0; movedfromreserved: false,moveToReserveredfalse
2024-12-10 09:27:04,447 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) release mem from host: 4, old used: (1.00 GB) 1073741824,reserved: (0 bytes) 0, total: (6.98 GB) 7492751360; new used: (512.00 MB) 536870912,reserved:(0 bytes) 0; movedfromreserved: false,moveToReserveredfalse
2024-12-10 09:27:04,476 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) Invocation exception, caused by: com.cloud.utils.exception.CloudRuntimeException: Failed to migrate VM [VM instance {"id":3,"instanceName":"i-2-3-VM","type":"User","uuid":"57df225e-a02f-4249-b731-fd48767a442f"}] along with its volumes due to [java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be cast to class com.cloud.agent.api.ModifyTargetsAnswer (com.cloud.agent.api.Answer and com.cloud.agent.api.ModifyTargetsAnswer are in unnamed module of loader 'app')].
2024-12-10 09:27:04,476 INFO  [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23, ctx-107ec426]) (logid:6c4e0747) Rethrow exception com.cloud.utils.exception.CloudRuntimeException: Failed to migrate VM [VM instance {"id":3,"instanceName":"i-2-3-VM","type":"User","uuid":"57df225e-a02f-4249-b731-fd48767a442f"}] along with its volumes due to [java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be cast to class com.cloud.agent.api.ModifyTargetsAnswer (com.cloud.agent.api.Answer and com.cloud.agent.api.ModifyTargetsAnswer are in unnamed module of loader 'app')].
2024-12-10 09:27:04,477 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23]) (logid:6c4e0747) Done with run of VM work job: com.cloud.vm.VmWorkMigrateWithStorage for VM 3, job origin: 22
2024-12-10 09:27:04,478 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23]) (logid:6c4e0747) Unable to complete AsyncJobVO: {id:23, userId: 2, accountId: 2, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkMigrateWithStorage, cmdInfo: rO0ABXNyACVjb20uY2xvdWQudm0uVm1Xb3JrTWlncmF0ZVdpdGhTdG9yYWdlsew9z6UxtXMCAANKAApkZXN0SG9zdElkSgAJc3JjSG9zdElkTAAMdm9sdW1lVG9Qb29sdAAPTGphdmEvdXRpbC9NYXA7eHIAE2NvbS5jbG91ZC52bS5WbVdvcmufmbZW8CVnawIABEoACWFjY291bnRJZEoABnVzZXJJZEoABHZtSWRMAAtoYW5kbGVyTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO3hwAAAAAAAAAAIAAAAAAAAAAgAAAAAAAAADdAAZVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbAAAAAAAAAAEAAAAAAAAAAFzcgARamF2YS51dGlsLkhhc2hNYXAFB9rBwxZg0QMAAkYACmxvYWRGYWN0b3JJAAl0aHJlc2hvbGR4cD9AAAAAAAAMdwgAAAAQAAAAAXNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cgAQamF2YS5sYW5nLk51bWJlcoaslR0LlOCLAgAAeHAAAAAAAAAAA3NxAH4ACAAAAAAAAAACeA, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 52231313017, completeMsid: null, lastUpdated: null, lastPolled: null, created: Tue Dec 10 09:21:59 EST 2024, removed: null}, job origin: 22 com.cloud.utils.exception.CloudRuntimeException: Failed to migrate VM [VM instance {"id":3,"instanceName":"i-2-3-VM","type":"User","uuid":"57df225e-a02f-4249-b731-fd48767a442f"}] along with its volumes due to [java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be cast to class com.cloud.agent.api.ModifyTargetsAnswer (com.cloud.agent.api.Answer and com.cloud.agent.api.ModifyTargetsAnswer are in unnamed module of loader 'app')].
	at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.migrateVolumes(VolumeOrchestrator.java:1444)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
	at jdk.proxy3/jdk.proxy3.$Proxy266.migrateVolumes(Unknown Source)
	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:3295)
	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:5515)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:106)
	at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5591)
	at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:99)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:652)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:600)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

2024-12-10 09:27:04,512 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23]) (logid:6c4e0747) Complete async job-23, jobStatus: FAILED, resultCode: 0, result: rO0ABXNyAC9jb20uY2xvdWQudXRpbHMuZXhjZXB0aW9uLkNsb3VkUnVudGltZUV4Y2VwdGlvblZNT3AAAAACAwABSQALY3NFcnJvckNvZGV4cgAaamF2YS5sYW5nLlJ1bnRpbWVFeGNlcHRpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0aW9u0P0fPho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0ABVMamF2YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJpbmc7WwAKc3RhY2tUcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHByZXNzZWRFeGNlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0AZBGYWlsZWQgdG8gbWlncmF0ZSBWTSBbVk0gaW5zdGFuY2UgeyJpZCI6MywiaW5zdGFuY2VOYW1lIjoiaS0yLTMtVk0iLCJ0eXBlIjoiVXNlciIsInV1aWQiOiI1N2RmMjI1ZS1hMDJmLTQyNDktYjczMS1mZDQ4NzY3YTQ0MmYifV0gYWxvbmcgd2l0aCBpdHMgdm9sdW1lcyBkdWUgdG8gW2phdmEubGFuZy5DbGFzc0Nhc3RFeGNlcHRpb246IGNsYXNzIGNvbS5jbG91ZC5hZ2VudC5hcGkuQW5zd2VyIGNhbm5vdCBiZSBjYXN0IHRvIGNsYXNzIGNvbS5jbG91ZC5hZ2VudC5hcGkuTW9kaWZ5VGFyZ2V0c0Fuc3dlciAoY29tLmNsb3VkLmFnZW50LmFwaS5BbnN3ZXIgYW5kIGNvbS5jbG91ZC5hZ2VudC5hcGkuTW9kaWZ5VGFyZ2V0c0Fuc3dlciBhcmUgaW4gdW5uYW1lZCBtb2R1bGUgb2YgbG9hZGVyICdhcHAnKV0udXIAHltMamF2YS5sYW5nLlN0YWNrVHJhY2VFbGVtZW50OwJGKjw8_SI5AgAAeHAAAAAhc3IAG2phdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudGEJxZomNt2FAgAIQgAGZm9ybWF0SQAKbGluZU51bWJlckwAD2NsYXNzTG9hZGVyTmFtZXEAfgAFTAAOZGVjbGFyaW5nQ2xhc3NxAH4ABUwACGZpbGVOYW1lcQB-AAVMAAptZXRob2ROYW1lcQB-AAVMAAptb2R1bGVOYW1lcQB-AAVMAA1tb2R1bGVWZXJzaW9ucQB-AAV4cAEAAAWkdAADYXBwdAA9b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmVuZ2luZS5vcmNoZXN0cmF0aW9uLlZvbHVtZU9yY2hlc3RyYXRvcnQAF1ZvbHVtZU9yY2hlc3RyYXRvci5qYXZhdAAObWlncmF0ZVZvbHVtZXNwcHNxAH4ADAL____-cHQALWpkay5pbnRlcm5hbC5yZWZsZWN0Lk5hdGl2ZU1ldGhvZEFjY2Vzc29ySW1wbHQAHU5hdGl2ZU1ldGhvZEFjY2Vzc29ySW1wbC5qYXZhdAAHaW52b2tlMHQACWphdmEuYmFzZXQABzE3LjAuMTNzcQB-AAwCAAAATXBxAH4AE3EAfgAUdAAGaW52b2tlcQB-ABZxAH4AF3NxAH4ADAIAAAArcHQAMWpkay5pbnRlcm5hbC5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXEAfgAWcQB-ABdzcQB-AAwCAAACOXB0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZcQB-ABZxAH4AF3NxAH4ADAEAAAFYcQB-AA50AChvcmcuc3ByaW5nZnJhbWV3b3JrLmFvcC5zdXBwb3J0LkFvcFV0aWxzdAANQW9wVXRpbHMuamF2YXQAHmludm9rZUpvaW5wb2ludFVzaW5nUmVmbGVjdGlvbnBwc3EAfgAMAQAAAMZxAH4ADnQAPG9yZy5zcHJpbmdmcmFtZXdvcmsuYW9wLmZyYW1ld29yay5SZWZsZWN0aXZlTWV0aG9kSW52b2NhdGlvbnQAH1JlZmxlY3RpdmVNZXRob2RJbnZvY2F0aW9uLmphdmF0AA9pbnZva2VKb2lucG9pbnRwcHNxAH4ADAEAAACjcQB-AA5xAH4AJXEAfgAmdAAHcHJvY2VlZHBwc3EAfgAMAQAAAGFxAH4ADnQAP29yZy5zcHJpbmdmcmFtZXdvcmsuYW9wLmludGVyY2VwdG9yLkV4cG9zZUludm9jYXRpb25JbnRlcmNlcHRvcnQAIEV4cG9zZUludm9jYXRpb25JbnRlcmNlcHRvci5qYXZhcQB-ABlwcHNxAH4ADAEAAAC6cQB-AA5xAH4AJXEAfgAmcQB-AClwcHNxAH4ADAEAAADXcQB-AA50ADRvcmcuc3ByaW5nZnJhbWV3b3JrLmFvcC5mcmFtZXdvcmsuSmRrRHluYW1pY0FvcFByb3h5dAAXSmRrRHluYW1pY0FvcFByb3h5LmphdmFxAH4AGXBwc3EAfgAMAP____9wdAAUamRrLnByb3h5My4kUHJveHkyNjZwcQB-ABF0AApqZGsucHJveHkzcHNxAH4ADAEAAAzfcQB-AA50ACZjb20uY2xvdWQudm0uVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbHQAHlZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwuamF2YXQAHW9yY2hlc3RyYXRlTWlncmF0ZVdpdGhTdG9yYWdlcHBzcQB-AAwBAAAVi3EAfgAOcQB-ADVxAH4ANnEAfgA3cHBzcQB-AAwC_____nBxAH4AE3EAfgAUcQB-ABVxAH4AFnEAfgAXc3EAfgAMAgAAAE1wcQB-ABNxAH4AFHEAfgAZcQB-ABZxAH4AF3NxAH4ADAIAAAArcHEAfgAbcQB-ABxxAH4AGXEAfgAWcQB-ABdzcQB-AAwCAAACOXBxAH4AHnEAfgAfcQB-ABlxAH4AFnEAfgAXc3EAfgAMAQAAAGpxAH4ADnQAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnBwc3EAfgAMAQAAFddxAH4ADnEAfgA1cQB-ADZxAH4AQHBwc3EAfgAMAQAAAGNxAH4ADnQAIGNvbS5jbG91ZC52bS5WbVdvcmtKb2JEaXNwYXRjaGVydAAYVm1Xb3JrSm9iRGlzcGF0Y2hlci5qYXZhdAAGcnVuSm9icHBzcQB-AAwBAAACjHEAfgAOdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0cHBzcQB-AAwBAAAAMXEAfgAOdAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1bm5hYmxlLmphdmF0AANydW5wcHNxAH4ADAEAAAA4cQB-AA50AEJvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0JDF0ABpEZWZhdWx0TWFuYWdlZENvbnRleHQuamF2YXQABGNhbGxwcHNxAH4ADAEAAABncQB-AA50AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-AFB0AA9jYWxsV2l0aENvbnRleHRwcHNxAH4ADAEAAAA1cQB-AA5xAH4AU3EAfgBQdAAOcnVuV2l0aENvbnRleHRwcHNxAH4ADAEAAAAucQB-AA50ADxvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGVxAH4ATHEAfgBNcHBzcQB-AAwBAAACWHEAfgAOcQB-AEdxAH4ASHEAfgBNcHBzcQB-AAwCAAACG3B0AC5qYXZhLnV0aWwuY29uY3VycmVudC5FeGVjdXRvcnMkUnVubmFibGVBZGFwdGVydAAORXhlY3V0b3JzLmphdmFxAH4AUXEAfgAWcQB-ABdzcQB-AAwCAAABCHB0AB9qYXZhLnV0aWwuY29uY3VycmVudC5GdXR1cmVUYXNrdAAPRnV0dXJlVGFzay5qYXZhcQB-AE1xAH4AFnEAfgAXc3EAfgAMAgAABHBwdAAnamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5Xb3JrZXJxAH4AFnEAfgAXc3EAfgAMAgAAAntwdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZWN1dG9yJFdvcmtlcnEAfgBicQB-AE1xAH4AFnEAfgAXc3EAfgAMAgAAA0hwdAAQamF2YS5sYW5nLlRocmVhZHQAC1RocmVhZC5qYXZhcQB-AE1xAH4AFnEAfgAXc3IAH2phdmEudXRpbC5Db2xsZWN0aW9ucyRFbXB0eUxpc3R6uBe0PKee3gIAAHhweAAAEJp3CAAAAAAAAAAAeA
2024-12-10 09:27:04,514 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23]) (logid:6c4e0747) Publish async job-23 complete on message bus
2024-12-10 09:27:04,514 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23]) (logid:6c4e0747) Wake up jobs related to job-23
2024-12-10 09:27:04,514 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23]) (logid:6c4e0747) Update db status for job-23
2024-12-10 09:27:04,535 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23]) (logid:6c4e0747) Wake up jobs joined with job-23 and disjoin all subjobs created from job- 23
2024-12-10 09:27:04,630 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23]) (logid:6c4e0747) Done executing com.cloud.vm.VmWorkMigrateWithStorage for job-23
2024-12-10 09:27:04,644 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-5:[ctx-d8c8e5df, job-22/job-23]) (logid:6c4e0747) Remove job-23 from job monitoring
2024-12-10 09:27:04,676 WARN  [o.a.c.m.w.WebhookServiceImpl] (API-Job-Executor-15:[ctx-f549996f, job-22, ctx-7c67f785]) (logid:6c4e0747) Skipping delivering event [ID: null, description: {"event":"VM.MIGRATE","status":"Completed"}] to any webhook as account ID is missing
2024-12-10 09:27:04,676 WARN  [o.a.c.f.e.EventDistributorImpl] (API-Job-Executor-15:[ctx-f549996f, job-22, ctx-7c67f785]) (logid:6c4e0747) Failed to publish event [category: ActionEvent, type: VM.MIGRATE] on bus webhookEventBus
2024-12-10 09:27:04,713 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-15:[ctx-f549996f, job-22]) (logid:6c4e0747) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd com.cloud.utils.exception.CloudRuntimeException: Failed to migrate VM [VM instance {"id":3,"instanceName":"i-2-3-VM","type":"User","uuid":"57df225e-a02f-4249-b731-fd48767a442f"}] along with its volumes due to [java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be cast to class com.cloud.agent.api.ModifyTargetsAnswer (com.cloud.agent.api.Answer and com.cloud.agent.api.ModifyTargetsAnswer are in unnamed module of loader 'app')].
	at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.migrateVolumes(VolumeOrchestrator.java:1444)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
	at jdk.proxy3/jdk.proxy3.$Proxy266.migrateVolumes(Unknown Source)
	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:3295)
	at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:5515)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:106)
	at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5591)
	at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:99)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:652)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:600)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

2024-12-10 09:27:04,725 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-15:[ctx-f549996f, job-22]) (logid:6c4e0747) Complete async job-22, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Failed to migrate VM [VM instance {"id":3,"instanceName":"i-2-3-VM","type":"User","uuid":"57df225e-a02f-4249-b731-fd48767a442f"}] along with its volumes due to [java.lang.ClassCastException: class com.cloud.agent.api.Answer cannot be cast to class com.cloud.agent.api.ModifyTargetsAnswer (com.cloud.agent.api.Answer and com.cloud.agent.api.ModifyTargetsAnswer are in unnamed module of loader 'app')]."}
2024-12-10 09:27:04,728 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-15:[ctx-f549996f, job-22]) (logid:6c4e0747) Publish async job-22 complete on message bus
2024-12-10 09:27:04,728 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-15:[ctx-f549996f, job-22]) (logid:6c4e0747) Wake up jobs related to job-22
2024-12-10 09:27:04,728 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-15:[ctx-f549996f, job-22]) (logid:6c4e0747) Update db status for job-22
2024-12-10 09:27:04,731 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-15:[ctx-f549996f, job-22]) (logid:6c4e0747) Wake up jobs joined with job-22 and disjoin all subjobs created from job- 22
2024-12-10 09:27:04,789 DEBUG [c.c.a.ApiServer] (API-Job-Executor-15:[ctx-f549996f, job-22]) (logid:6c4e0747) Retrieved cmdEventType from job info: VM.MIGRATE
2024-12-10 09:27:04,850 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5] (API-Job-Executor-15:[ctx-f549996f, job-22]) (logid:6c4e0747) Done executing org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd for job-22
2024-12-10 09:27:04,850 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-15:[ctx-f549996f, job-22]) (logid:6c4e0747) Remove job-22 from job monitoring
2024-12-10 09:27:05,248 DEBUG [c.c.a.t.Request] (AgentManager-Handler-5:[]) (logid:) Seq 4-7171982406587515719: Processing:  { Ans: , MgmtId: 52231313017, via: 4, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
2024-12-10 09:27:05,249 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-5:[]) (logid:) Seq 4-7171982406587515719: Unable to find listener.

Agent logs

2024-12-10 05:50:28,601 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:[]) (logid:fed9001a) Trying to fetch storage pool ef8eef61-8e8c-3b13-9f04-2d854a6e0fc6 from libvirt
2024-12-10 05:50:28,612 WARN  [cloud.agent.Agent] (agentRequest-Handler-1:[]) (logid:fed9001a) Caught: com.cloud.utils.exception.CloudRuntimeException: Can't find volume:efedc0b5-8674-4bd8-a723-7c971c5bce40
	at com.cloud.hypervisor.kvm.storage.LibvirtStoragePool.getPhysicalDisk(LibvirtStoragePool.java:159)
	at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getPhysicalDisk(KVMStoragePoolManager.java:369)
	at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyTargetsCommandWrapper.execute(LibvirtModifyTargetsCommandWrapper.java:57)
	at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyTargetsCommandWrapper.execute(LibvirtModifyTargetsCommandWrapper.java:38)
	at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
	at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1945)
	at com.cloud.agent.Agent.processRequest(Agent.java:686)
	at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1109)
	at com.cloud.utils.nio.Task.call(Task.java:83)
	at com.cloud.utils.nio.Task.call(Task.java:29)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

2024-12-10 05:50:28,758 INFO  [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:[]) (logid:fed9001a) No existing libvirtd connection found. Opening a new one
2024-12-10 05:50:28,764 WARN  [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:[]) (logid:fed9001a) Can not find a connection for Instance i-2-3-VM. Assuming the default connection.
2024-12-10 05:50:29,473 WARN  [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-2:[]) (logid:fed9001a) Groovy script '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not available. Transformations will not be applied.
2024-12-10 05:50:29,473 WARN  [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-2:[]) (logid:fed9001a) Groovy scripting engine is not initialized. Data transformation skipped.
2024-12-10 05:50:55,629 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-4:[]) (logid:a8dc1538) Trying to fetch storage pool ef8eef61-8e8c-3b13-9f04-2d854a6e0fc6 from libvirt
2024-12-10 05:50:55,633 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-4:[]) (logid:a8dc1538) Asking libvirt to refresh storage pool ef8eef61-8e8c-3b13-9f04-2d854a6e0fc6

Metadata

Metadata

Assignees

Labels

component:migrationtype:bug

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions