qemu error: cannot support legacy mode due to existing devices at address 1f.0

[K1ngjulien]

Hi! Awesome work you've done here. I just tried running this on my Thinkpad E590 running Fedora 34. The steps setup check configure and iso all worked fine but when I ran ./mbpt start I got the following error :

qemu-system-x86_64: -device vfio-pci,bus=pcie.0,addr=02.0,sysfsdev=/sys/bus/pci/devices/0000:00:02.0/1205058e-1fd4-11ec-840b-e86a64ff767e,x-igd-opregion=on,rombar=0,display=on: IGD device 1205058e-1fd4-11ec-840b-e86a64ff767e cannot support legacy mode due to existing devices at address 1f.0
qemu-system-x86_64: vfio_err_notifier_handler(0000:03:00.0) Unrecoverable error detected. Please collect any data possible and then kill the guest

Running spicy -h localhost -p 5900 opened up two black windows with no content.

During setup I used all the default values except for the amdgpu driver for the dGPU and setting the right PCIe addresses from lspci (00:02.0 for the Intel iGPU and 03:00.0 for the AMD dGPU)

Have you run into this issue before? Sorry if this is something simple I missed.

Some more system info:

[~/opt/MobilePassThrough (master)]: uname -r
5.13.19-200.fc34.x86_64

[~/opt/MobilePassThrough (master)]: ./mbpt.sh check
lspci: -s: Invalid slot number
[OK] VT-X / AMD-V virtualization is enabled in the UEFI.
[OK] VT-D / IOMMU is enabled in the UEFI.
[OK] The IOMMU kernel parameters are set.
[Success] GPU with ID '00:02.0' could be passed through to a virtual machine!
[Success] GPU with ID '03:00.0' could be passed through to a virtual machine!
[Success] There are 2 GPU(s) in this system that could be passed through to a VM!

Is Compatible?  Name                                                    IOMMU_GROUP  PCI Address
Yes             WhiskeyLake-U GT2 [UHD Graphics 620]                    19           pci@0000:00:02.0
Yes             Lexa PRO [Radeon 540/540X/550/550X / RX 540X/550/550X]  15           pci@0000:03:00.0

[OK] You have GPUs that are not in the same IOMMU group. At least one of these could be passed through to a VM and at least one of the remaining ones could be used for the host system.
[Info] Device name: 20NB0029GE
[Info] BIOS version: R0YET42W (1.25 )
[Info] This system is probably MUX-less. (The connection between the GPU(s) and the [internal display]/[display outputs] is not multiplexed.)
[OK] Logs have been created in /home/julian/opt/MobilePassThrough/logs/20NB0029GE/R0YET42W (1.25 )
If you found a notebook that appears to be GPU passthrough compatible, please open an issue on Github and let me know.
You may now proceed and run './mbpt.sh configure' as mentioned in the README.

the lspci -s: invalid slot number in the first line here seems suspicious to me.

I'd be happy to provide more details and can send you the full check logs if you want.

[T-vK]

I can't say I recognize the error, but the failing lspci could be a problem indeed.
I really need to merge the unattended-win-install branch into the master branch. It contains a lot of fixes. The only reason I haven't merged it yet is because it doesn't work on Ubuntu yet.
You should definitely give it a try.
https://github.com/T-vK/MobilePassThrough/tree/unattended-win-install
You need to remove your old config though.

[K1ngjulien]

Ok I'll give it a try! Do I need to delete anything other than user.conf?

[K1ngjulien]

I was about to report that this worked perfectly but after the installation finished and the VM shut down it got stuck when trying to unbind the dGPU from vfio to the amdgpu driver, froze my entire system and the fans kicked in at full speed 😅

After a reboot I tried running ./mbpt.sh auto again but got the following output with a black spice window:

> Unbinding device '03:00.0 Display controller [0380]: Advanced Micro Devices, Inc. [AMD/ATI] Lexa PRO [Radeon 540/540X/550/550X / RX 540X/550/550X] [1002:699f] (rev c0)' from its driver, then bind it to the vfio-pci driver...
> [Background task] Starting the spice client at localhost:5904...
> [Background task] Starting RDP autoconnect...
> Starting the Virtual Machine using qemu...
QEMU 5.2.0 monitor - type 'help' for more information
(qemu) qemu-system-x86_64: vfio_err_notifier_handler(0000:03:00.0) Unrecoverable error detected. Please collect any data possible and then kill the guest
(qemu)
(qemu)
(qemu) qemu-system-x86_64: terminating on signal 2
Cleaning up...
> Unbinding device '03:00.0 Display controller [0380]: Advanced Micro Devices, Inc. [AMD/ATI] Lexa PRO [Radeon 540/540X/550/550X / RX 540X/550/550X] [1002:699f] (rev c0) from the vfio-pci driver, then bind it back to its original driver...

I'll try this again tomorrow. Maybe rebooting a few times will fix it 😄. The check output had a few red error lines in it before I started the vm but those now seem to be gone.

Sidenote: I don't think its necessary to start remina/rdp when running the first setup as you can watch the progress in the spice window anyway :)

[T-vK]

Removing user.conf should be enough.

I haven't gotten that unrecoverable state error before, but your GPU might be having the reset bug. You could try this fix:
https://github.com/gnif/vendor-reset

I will say that I sometimes have binding issues with the dGPU as well, but for me it just gets stuck without errors and I have to reboot to recover from that situation.

Regarding your sidenote, the default display mode is 5 if I recall correctly, you can create a new one that keeps USE_RDP false, this way it won't start Remmina. I just decided to make use RDP and dmabuf at the same time by default so that you can see the boot screen which wouldn't be possible with RDP.