How to improve latency between Sunshine and Moonlight

[pogojotz]

When comparing my Sunshine-Moonlight setup with the in-home streaming with two Steam instances or Steam-Steamlink, I see a much higher delay. This is totally fine for watching movies (what I mostly do) since the audio seems to be in sync, but would be unusable for gaming.

Test setup

• 1 monitor connected to Sunshine device
• 1 laptop/tablet running Moonlight right next to the first monitor

Observation

Stuff happening on the first monitor happens earlier than the same stuff on Moonlight device. My guess the latency is around 100-200ms - not measured!

I have tried a bunch of Sunshine settings but nothing yielded significant improvements. I also tried different machines, running Moonlight, both wired and wireless LAN. I don't even know if the delay is caused by Sunshine at all or if Moonlight is the culprit or both.

Do you have any tips for better settings. Or how to debug this situation the best. Or is it simply not feasible (yet) to achieve same delays as e.g. Steam?

[KuleRucket]

The first thing I would do is identify where the lag is, encode/network/decode. ctrl-alt-shift-s should help with that. You don't say what OS or hardware you are using. If nvidia I would suggest things like ensuring NvFBC works properly (as steam can use).

For me steam is absolutely terrible compared to sunshine/moonlight. I.e. 40ms lag with 1080p@60fps using steam compared to 1-2ms with1440p@120fps using moonlight/sunshine.

[pogojotz]

Thank you @KuleRucket. This is certainly a starting point.
I use Arch Linux with an RTX 2070 (driver nvidia-dkms 510.54) so I thought the hardware should not be the issue here.

After you mentioning NvFBC I realized that I didn't even have installed CUDA and thus compiled without NvFBC. So I installed it. I had to run cmake like this CUDACXX=/opt/cuda/bin/nvcc cmake .., otherwise my cmake would not find CUDA (/opt/cuda is the default file installation folder of the cuda package on Arch Linux).

But now with CUDA support compiled in, the very first log message I get from Sunshine is Error: Failed to create session: This hardware does not support NvFBC. Is that possible or a false flag? I had assumed that the RTX 2070 would support NvFBC, but I cannot find any (publicly available) data on which Nvidia cards do support it.

[KuleRucket]

I also use Arch with an RTX 3070 so can provide some relevant info. My client is an Intel NUC 11 with Intel Iris Xe integrated GPU.

1. You can add 'cuda' 'libdrm' 'libcap' to depends=() in the PKGBUILD of the sunshine AUR package to include cuda in the build. You can't use a helper like yay though and must makepkg manually when there is an update.
2. You need a patched nvidia driver to enable nvfbc. Nvidia only allows it on commercial Quadro cards and not on consumer Geforce cards even though the card can do it. An easy way to do this on Arch is to install nvidia-utils-nvlax from AUR (replacing nvidia-utils)
3. Due to issue Error: Failed to sync desktop to thread [0x000000AA] on Windows 10 #100 you currently need to downgrade vulkan libraries and block their update as described in the ticket comments.

If it's all fine you should see these info messages within the log output:
Info: Screencasting with NvFBC
Info: Found encoder nvenc: [h264_nvenc, hevc_nvenc]

For reference, this is the sort of performance I get on a wired LAN.
https://ibb.co/3mT0Zg7

[pogojotz]

Thank you again. I patched the driver with nvidia-patch and saw that it is now using NvFBC. Sadly my network gave up at that time and I had to go to bed anyway. I will try the vulkan downgrade later today and test again.

Your screenshot shows the delays after the frame left the Sunshine machine (network, decoding, rendering) but what about the delay on the server (capturing, encoding) or total delay? Is there a way to get the timings, without special equipment like a high speed camera or photo diodes?

[KuleRucket]

It's not possible to objectively, but CPU load on sunshine is very low and the GPU video encoder is only working at 50% according to nvtop so it's not working hard at all.

[lastmetroid]

Is there maybe a up to date guide what settings you should use or how to improve performance and latency if you use AMD or Nvidia on a Sunshine host? Is it still possible to use NVFBC somehow on Windows, I read there was this hack but I dont know how that works or if it still works with latest Nvidia drivers.

https://github.com/illnyang/nvlax/

I get around 20-30 less FPS if I stream with Sunshine with my RTX4080 compared to native play, which is a bit huge of a performance loss honestly.

[lastmetroid]

No I have turned it off because the bug which causes the stream to freeze with sound still playing is still happening for me with HAGS on, no idea why (LizardByte/Sunshine#1281 (comment)).

Can you just install a nightly build over a current stable build? IF I try to install the nightly it asks me if I want to uninstall the previous version should I say yes or no?

[cgutman]

The HAGS issue is so strange. I've streamed for hours on my RTX 4080 with HAGS enabled without encountering it, yet some people hit it in minutes.

Yes, you can install a nightly build over the stable build. Click "yes" to uninstalling the previous version.

There is a change in the current nightlies that seems to significantly reduce the likelihood of the HAGS hang issue, so you may consider trying it again if overhead is still too high.

[lastmetroid]

I was not able to make it freeze while playing a game so far with HAGS on, just inside Windows 11 desktop environment, for example opening GFE and clicking around. Also happened in Minecraft launcher clicking on a button. But I got this pc just since a few days and havent played for so long so far. Will test the nightly the next days with HAGS on again.

[lastmetroid]

Whats the best way with latest Sunshine to get a stream smooth while keeping latency low? I just tested a little bit, and if I dont turn on VSync on the host game, micro stuttering will happen in Moonlight client (which also needs to use VSync on or tearing will happen), double VSync is not the best scenario I guess, even with frame limiter on 60. But turning on VSync on the host adds a lot of latency. I also noticed using borderless windowed (0.4ms) mode in Moonlight dramatically lowers render time on the client, compared to fullscreen (12ms).

[lastmetroid]

Ive tested 2-3 hours again yesterday with HAGS on and the nightly and had no freeze so far anymore, maybe it was fixed in nightly. Will test more.