VMware to KVM migration failed with error: Locator does not refer to an object

[gioporta]

problem

I am attempting to migrate VMs from an external VMware cluster to CloudStack using the Import-Export tool.

The VMware VMs are not located within the root folder of the datacenter, but rather, they are organized into VM folders.

When the CloudStack agent attempts to run ovftool to export the VM from VMware, it is not passing the full path of the VM to ovftool, instead it assumes that the VM is in the root folder.

The following error is displayed in GUI:
The convert process failed for instance vm1 from VMware to KVM on host host1: Export OVA for the VM vm1 failed

Logs from CloudStack Agent on the KVM host (sensitive information changed):

2025-02-14 15:25:57,592 INFO  [resource.wrapper.LibvirtConvertInstanceCommandWrapper] (agentRequest-Handler-1:[]) (logid:ffec67e0) Attempting to convert the instance vm1 from VMware to KVM
2025-02-14 15:25:58,028 INFO  [resource.wrapper.LibvirtConvertInstanceCommandWrapper] (Script-1:[]) (logid:) (export ovf) Opening VI source: vi://svc-cloudstack@vcenter:443/DC1/vm/vm1
2025-02-14 15:25:58,065 INFO  [resource.wrapper.LibvirtConvertInstanceCommandWrapper] (Script-1:[]) (logid:) (export ovf) Error: Locator does not refer to an object: vi://svc-cloudstack@vcenter:443/DC1/vm/vm1
2025-02-14 15:25:58,066 INFO  [resource.wrapper.LibvirtConvertInstanceCommandWrapper] (Script-1:[]) (logid:) (export ovf) Completed with errors
2025-02-14 15:25:58,069 WARN  [resource.wrapper.LibvirtConvertInstanceCommandWrapper] (agentRequest-Handler-1:[]) (logid:ffec67e0) Execution of process [206434] for command [ovftool --noSSLVerify vi://svc-cloudstack:xxxxx@vcenter/DC1/vm/vm1 /mnt/47fc42f3-fc97-3df6-8ab7-bffdc04a9a3e/6a0ec3e8-0aca-4e64-a38a-d45e7bf8adc6/ ] failed.
2025-02-14 15:25:58,069 WARN  [resource.wrapper.LibvirtConvertInstanceCommandWrapper] (agentRequest-Handler-1:[]) (logid:ffec67e0) Exception [Stream closed] occurred when attempting to run command [ovftool --noSSLVerify vi://svc-cloudstack:xxxxx@vcenter/DC1/vm/vm1 /mnt/47fc42f3-fc97-3df6-8ab7-bffdc04a9a3e/6a0ec3e8-0aca-4e64-a38a-d45e7bf8adc6/ ]. java.io.IOException: Stream closed
        at java.base/java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:168)
        at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:334)
        at java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:287)
        at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:330)
        at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:190)
        at java.base/java.io.InputStreamReader.read(InputStreamReader.java:177)
        at java.base/java.io.BufferedReader.fill(BufferedReader.java:162)
        at java.base/java.io.BufferedReader.readLine(BufferedReader.java:329)
        at java.base/java.io.BufferedReader.readLine(BufferedReader.java:396)
        at com.cloud.utils.script.OutputInterpreter.processError(OutputInterpreter.java:41)
        at com.cloud.utils.script.Script.execute(Script.java:314)
        at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtConvertInstanceCommandWrapper.exportOVAFromVMOnVcenter(LibvirtConvertInstanceCommandWrapper.java:391)
        at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtConvertInstanceCommandWrapper.execute(LibvirtConvertInstanceCommandWrapper.java:117)
        at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtConvertInstanceCommandWrapper.execute(LibvirtConvertInstanceCommandWrapper.java:61)
        at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1945)
        at com.cloud.agent.Agent.processRequest(Agent.java:686)
        at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1109)
        at com.cloud.utils.nio.Task.call(Task.java:83)
        at com.cloud.utils.nio.Task.call(Task.java:29)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)

2025-02-14 15:25:58,069 ERROR [resource.wrapper.LibvirtConvertInstanceCommandWrapper] (agentRequest-Handler-1:[]) (logid:ffec67e0) Export OVA for the VM vm1 failed

When I ran the ovftool command directly, I saw the issue. It is throwing the error Error: Locator does not refer to an object because the VM at that path does not exist.

versions

CloudStack Management Server 4.20.0.0-1 on AlmaLinux 9.5
CloudStack Agent 4.20.0.0-1 on AlmaLinux 9.5
VMware vCenter 8.0.1 build 22088981
VMware ovftool 4.6.3 (build-24031167)

The steps to reproduce the bug

1. Create a VM on vCenter that is located inside a VM folder.
2. Attempt to migrate the VM to CloudStack via the external vCenter server.
3. Observe that the migration fails with error in GUI
4. Check agent logs and confirm ovftool results in error Error: Locator does not refer to an object
5. Move VM to root folder of datacenter in vCenter, re-run migration, confirming it now works OK

What to do about it?

When CloudStack runs ovftool, it should pass the full path of the VM including any folders or subfolders that it is in. Passing only the VM name is not sufficient.

For example, if the VM vm1 is located in the folder Test-Folder within datacenter DC1, the full path for ovftool would be similiar to this:
vi://username:password@vcenter/DC1/vm/Test-Folder/vm1

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[gioporta]

The source of the problem appears to be the getExportOVAUrlFromRemoteInstance method in LibvirtConvertInstanceCommandWrapper.java, which simply adds the instance name to the export URL without regard to the folder it is located in.

cloudstack/plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtConvertInstanceCommandWrapper.java

Lines 190 to 201 in 864751d

	private String getExportOVAUrlFromRemoteInstance(RemoteInstanceTO vmwareInstance) {
	String vcenter = vmwareInstance.getVcenterHost();
	String username = vmwareInstance.getVcenterUsername();
	String password = vmwareInstance.getVcenterPassword();
	String datacenter = vmwareInstance.getDatacenterName();
	String vm = vmwareInstance.getInstanceName();
	String encodedUsername = encodeUsername(username);
	String encodedPassword = encodeUsername(password);
	return String.format("vi://%s:%s@%s/%s/vm/%s",
	encodedUsername, encodedPassword, vcenter, datacenter, vm);
	}

[kiranchavala]

@gioporta

It looks like a valid issue

I have tried in an environment where there is no ovftool installed and the import was successful with virt-v2v

I will check after installing ovftool

[sureshanaparti]

thanks for raising the issue @gioporta, able to reproduce this.

[DaanHoogland]

fixed by #10411