Raspberry Pi support (arm64)?

[geerlingguy]

I've been trying to get an RTX 3080 Ti working on my Raspberry Pi 5 — see issue here: geerlingguy/raspberry-pi-pcie-devices#549

I've been able to get the amdgpu kernel driver to work with full desktop environment and hardware acceleration support, though we still run into occasional bugs that we're squashing.

The Nvidia drivers seem to run better on supported arm64 platforms, like Ampere Altra and AmpereOne, and many have reported success on other arm64 platforms (like Honeycomb or NXP dev boards).

Judging by my latest attempt with the newest open kernel driver (installed with sudo ./NVIDIA-Linux-aarch64-550.127.05.run -m=kernel-open), there may be a few memory mapping-related errors, in my testing on a Pi 5 with 8 GB of RAM:

[    6.496707] NVRM: clCheckUpstreamLtrSupport_IMPL: PCIE config space is inaccessible!
[    6.595440] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: no initialized WPR2 found
[    6.595447] NVRM: kgspExecuteFwsec_TU102: (note: VBIOS version 94.02.71.80.74)
[    6.595451] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:445
[    6.595463] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[    6.596427] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0xffff:1792)
[    6.596676] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0

It would be great if I could get any level of working support for the card (or newer generation cards, 40 series or workstation cards) on the Pi 5.

Here is a picture showing an overview of my hardware setup for testing (in this case, with an A400 plugged in):

[geerlingguy]

Same error happens with an A400, A4000, and 4090 on the Pi, also tested with the latest driver (both proprietary and open), version 565.77. See: geerlingguy/raspberry-pi-pcie-devices#677

[aritger]

Hi @geerlingguy . I'm sorry I missed your earlier posts here. I've filed NVIDIA bug 5053788 internally for this.
Can you please retest with the latest release (565.77, as of this writing), and then generate and attach an nvidia-bug-report.log.gz? Thanks.

[geerlingguy]

Will do! For my own reference, looks like I should su to root and then run nvidia-bug-report.sh.

[geerlingguy]

I have run the following to install the 565.77 driver, on a brand new / clean install of Pi OS 12/Bookworm:

sudo apt update
sudo apt install -y raspberrypi-kernel-headers
sudo apt dist-upgrade -y
sudo reboot

...

cd Downloads
wget https://us.download.nvidia.com/XFree86/aarch64/565.77/NVIDIA-Linux-aarch64-565.77.run
chmod +x NVIDIA-Linux-aarch64-565.77.run
sudo ./NVIDIA-Linux-aarch64-565.77.run

I installed the MIT/GPL version, and rebooted...

After the reboot, I got the same errors in dmesg:

[   16.186734] NVRM: _issueRpcAndWait: rpcRecvPoll timedout for fn 47!
[   16.186736] NVRM: kgspInitRm_IMPL: SET_GUEST_SYSTEM_INFO failed: 0x65
[   16.186828] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   16.188354] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x65:1863)
[   16.188740] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0

I ran nvidia-bug-report.sh, and the vulkaninfo process seemed to use 100% CPU for a while, while many messages went by in dmesg like the following:

[  120.167666] NVRM: gpuHandleSanityCheckRegReadError_GM107: Possible bad register read: addr: 0x110100,  regvalue: 0xbadf5720,  error code: Unknown SYS_PRI_ERROR_CODE

The full nvidia-bug-report.log.gz is attached below:

nvidia-bug-report.log.gz

[zackify]

Dang jeff, i thought this already worked so I bought the parts and I am receiving them tomorrow lol! My fault for reading your amd article and assuming it would.

I am installing the latest version now, 575.64, and let you all know if it happens to work yet, using a 2070 super

[aritger]

I could also use a description of your Raspberry Pi hardware configuration: what is the combination of hardware modules that you're using to connect a GPU to the Raspberry Pi?

[geerlingguy]

For most of my testing, this is the hardware setup (pictured above):

• Raspberry Pi 5
• Pineboards HatDrive! Bottom
• JMT Oculink to PCIe Dock
• chenyang Oculink cable
• Lian-Li 750W SFX Power supply, with 24p header connected to JMT dock

For quick testing with smaller cards, I use a smaller HAT that puts a PCIe slot directly on top of the Pi:

• Raspberry Pi 5 model B (8 GB, and also testing 16 GB now that it is out)
• Pineboards HAT uPCIty Lite
• Lian-Li 750W SFX Power supply, connected to the card and the uPCIty Lite board via 4-pin CPU header (for slot power)

[mickkael]

@aritger could you share if there is any update from the internal ticket ?

[aritger]

I'm sorry, I'm not aware of any progress so far.

[jstockdale]

Any luck with progress here @aritger? I also would be excited to run NVIDIA hardware on the rapsberry pi 5 / cm5 platform. Thanks!

[aritger]

I'm sorry, there hasn't been any interesting progress to report, yet.

[geerlingguy]

Just wanted to post an update from over in the Pi PCIe repo, @RSC-Games is exploring IOMMU support on the Pi, possibly helping both Intel and Nvidia drivers work better on the Pi 5's PCIe bus, but it's very experimental currently, definitely with no success guaranteed: geerlingguy/raspberry-pi-pcie-devices#754

[RSC-Games]

I have yet to do testing yet on amdgpu with forced cache flushes between the test ring read and write (with TTM set to cached) but if that works, and we can somehow get the IOMMU to map all memory as uncached, regardless of what any driver requests, then we're pretty much solid on almost all PCIe device drivers in the kernel, barring x64-specific code.

As Jeff said above, check the issue for more updates and a live view of what's going on with the testing and development of these patches. I'll update this comment every now and then (when/if we make any breakthroughs) so stay posted!