NVENC with NVFbc higher latency than software encoding?

[entropicdrifter]

I'm running a GTX1080 on Linux Mint 20.3 with the Ubuntu 20.04 version of the 0.13 deb package installed. I also have an Intel CPU with an iGPU capable of h264 encoding but haven't gotten VA-API to work despite the fact that I can get it to work with other programs.

So far, comparing the Nvidia card with NvFbc enabled (patched the driver and logs show it as working), I find there's a noticable amount of extra latency with the hardware encoding vs the software encoder. Ideally I'd like to cut down on the hardware encoder's latency rather than give up CPU cores to the software encoder.

As far as moonlight's performance stats show, I'm getting 3-4ms network latency and ~1ms decoding time , for a total of ~5ms average rendering time. This is what it shows for both encoder settings, so I know it must be the hardware encoder adding the extra latency.

Any ideas of what I should be doing? Best settings, etc? Right now I've got it on the llhp preset, the cbr_ld_hq rate control and cavlc as the coder.

Would it work better if I built the master branch from scratch using the docker script instead of the pre-compiled .deb?

[ReenigneArcher]

Would it work better if I built the master branch from scratch using the docker script instead of the pre-compiled .deb?

No. That likely would make zero difference.

FYI 5ms is 0.005 seconds.

[entropicdrifter]

Right, I know. My point is there's noticeable latency despite the 5 ms total network + decoding latency, which led me to the conclusion that it must be the encoder.

[entropicdrifter]

What I'm saying is:

1. The UI doesn't show the encoders' latency
2. The latency it does show is ludicrously tiny, as you've pointed out
3. Therefore, the latency I can clearly see, but don't have a way to measure must be from the encoder, since switching it to the software encoder largely eliminates it.

What I'm looking for are the best settings for the Nvidia encoder in order to reduce the extra latency I'm seeing with it compared to software encoding on the superfast preset.

[ReenigneArcher]

It might be good to post your config, and then me (or someone else) can review to see if anything is way out of line.

[entropicdrifter]

gamepad = x360
port = 47989
vt_coder = auto
vt_realtime = enabled
key_rightalt_to_key_win = disabled
encoder = nvenc
amd_rc = auto
hevc_mode = 0
nv_rc = cbr
channels = 1
origin_pin_allowed = pc
resolutions = [
    352x240,
    480x360,
    858x480,
    1280x720,
    1920x1080,
    
]
fec_percentage = 30
origin_web_ui_allowed = lan
amd_quality = default
vt_software = auto
sw_preset = superfast
min_log_level = 2
output_name = 0
adapter_name = /dev/dri/renderD128
nv_preset = llhp
upnp = disabled
file_apps = apps_linux.json
nv_coder = auto
sw_tune = zerolatency
fps = [10,30,60,90,120]

[entropicdrifter]

Alright, so to get the lowest latency you need to use the cavlc coder, the llhp preset, and cbr as the rate control setting.

cbr_lowdelay_hq still uses hq settings that add delay, and my config that I posted in my last comment before this one has all the settings right except the coder is set to auto, which when used with ll, llhp, or llhq will use cabac instead of cavlc, which seems to add some noticeable latency.

The nvenc fast preset works pretty well too except it occasionally has lag spikes because it expects there to be a buffer on the decoder end, that's why the llhp preset is the way to go.

[KuleRucket]

For the latest nvidia SDK (used e.g. by Arch) it complains that the ll/llhp/llhq etc presents are being deprecated. I use p3 and don't have any latency problems. There is more information about this change here:
https://developer.nvidia.com/blog/introducing-video-codec-sdk-10-presets/
and a handy mapping between the old and new style presents here:
https://docs.nvidia.com/video-technologies/video-codec-sdk/nvenc-preset-migration-guide/

[ReenigneArcher]

Can you open an issue about this change in presets?