VAAPI fails: vaGetDriverNames() failed with operation failed #316

xTeixeira · 2023-09-30T21:25:57Z

xTeixeira
Sep 30, 2023

Is there an existing issue for this?

I have searched the existing issues

Is your issue described in the documentation?

I have read the documentation

Is your issue present in the nightly release?

This issue is present in the nightly release

Describe the Bug

Recently I haven't been able to get vaapi working on Arch Linux with amdgpu.
It fails with the following log messages:

[2023:09:30:18:19:59]: Info: vaGetDriverNames() failed with operation failed
[2023:09:30:18:19:59]: Error: Couldn't initialize va display: operation failed

Sunshine then uses software encoding as fallback.

Expected Behavior

Sunshine launches and vaapi encoding works.

Additional Context

Hardware decoding with vaapi seems to be working fine in mpv.
vainfo output:

Trying display: wayland
Trying display: x11
vainfo: VA-API version: 1.20 (libva 2.20.0)
vainfo: Driver version: Mesa Gallium driver 23.1.8-arch1.1 for AMD Radeon RX 7900 XTX (gfx1100, LLVM 16.0.6, DRM 3.54, 6.5.5-zen1-1-zen)
vainfo: Supported profile and entrypoints
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileHEVCMain10             :	VAEntrypointEncSlice
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileVP9Profile0            :	VAEntrypointVLD
      VAProfileVP9Profile2            :	VAEntrypointVLD
      VAProfileAV1Profile0            :	VAEntrypointVLD
      VAProfileNone                   :	VAEntrypointVideoProc

Host Operating System

Linux

Operating System Version

Arch Linux

Architecture

64 bit

Sunshine commit or version

0.20.0

Package

Linux - PKGBUILD

GPU Type

AMD

GPU Model

RX 7900 XTX

GPU Driver/Mesa Version

Mesa 23.1.8, Kernel 6.5.5

Capture Method (Linux Only)

KMS

Config

output_name = 0
capture = kms
log_path = /home/teixeira/.sunshine.log
resolutions = [
    352x240,
    480x360,
    858x480,
    1280x720,
    1920x1080,
    2560x1080,
    3440x1440,
    1920x1200,
    3860x2160,
    3840x1600,
    3840x2160
]
min_log_level = 2
origin_web_ui_allowed = wan
hevc_mode = 0
encoder = vaapi

Apps

No response

Relevant log output

[encoder] -- [vaapi]
[hevc_mode] -- [0]
[origin_web_ui_allowed] -- [wan]
[min_log_level] -- [2]
[resolutions] -- [[
    352x240,
    480x360,
    858x480,
    1280x720,
    1920x1080,
    2560x1080,
    3440x1440,
    1920x1200,
    3860x2160,
    3840x1600,
    3840x2160
]]
[log_path] -- [/home/teixeira/.sunshine.log]
[capture] -- [kms]
[output_name] -- [0]
[2023:09:30:18:19:59]: Info: Sunshine version: 0.20.0.f76879eb0dc287f09ae3810bd61b191f21344f78
[2023:09:30:18:19:59]: Error: Environment variable WAYLAND_DISPLAY has not been defined
[2023:09:30:18:19:59]: Info: Trying encoder [vaapi]
[2023:09:30:18:19:59]: Info: Screencasting with KMS
[2023:09:30:18:19:59]: Info: Found monitor for DRM screencasting
[2023:09:30:18:19:59]: Info: Screencasting with KMS
[2023:09:30:18:19:59]: Info: Found monitor for DRM screencasting
[2023:09:30:18:19:59]: Info: System tray created
[2023:09:30:18:19:59]: Info: SDR color coding [Rec. 601]
[2023:09:30:18:19:59]: Info: Color depth: 8-bit
[2023:09:30:18:19:59]: Info: Color range: [JPEG]
[2023:09:30:18:19:59]: Info: vaGetDriverNames() failed with operation failed
[2023:09:30:18:19:59]: Error: Couldn't initialize va display: operation failed
[2023:09:30:18:19:59]: Info: Screencasting with KMS
[2023:09:30:18:19:59]: Info: Found monitor for DRM screencasting
[2023:09:30:18:19:59]: Info: SDR color coding [Rec. 601]
[2023:09:30:18:19:59]: Info: Color depth: 8-bit
[2023:09:30:18:19:59]: Info: Color range: [JPEG]
[2023:09:30:18:19:59]: Info: vaGetDriverNames() failed with operation failed
[2023:09:30:18:19:59]: Error: Couldn't initialize va display: operation failed
[2023:09:30:18:19:59]: Info: Screencasting with KMS
[2023:09:30:18:19:59]: Info: Found monitor for DRM screencasting
[2023:09:30:18:19:59]: Info: SDR color coding [Rec. 601]
[2023:09:30:18:19:59]: Info: Color depth: 8-bit
[2023:09:30:18:19:59]: Info: Color range: [JPEG]
[2023:09:30:18:19:59]: Info: vaGetDriverNames() failed with operation failed
[2023:09:30:18:19:59]: Error: Couldn't initialize va display: operation failed
[2023:09:30:18:19:59]: Info: Screencasting with KMS
[2023:09:30:18:19:59]: Info: Found monitor for DRM screencasting
[2023:09:30:18:20:00]: Info: SDR color coding [Rec. 601]
[2023:09:30:18:20:00]: Info: Color depth: 8-bit
[2023:09:30:18:20:00]: Info: Color range: [JPEG]
[2023:09:30:18:20:00]: Info: vaGetDriverNames() failed with operation failed
[2023:09:30:18:20:00]: Error: Couldn't initialize va display: operation failed
[2023:09:30:18:20:00]: Info: Encoder [vaapi] failed
[2023:09:30:18:20:00]: Error: Couldn't find any working encoder matching [vaapi]
[2023:09:30:18:20:00]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. //
[2023:09:30:18:20:00]: Info: Trying encoder [nvenc]
[2023:09:30:18:20:00]: Info: Screencasting with KMS
[2023:09:30:18:20:00]: Info: Found monitor for DRM screencasting
[2023:09:30:18:20:00]: Info: Screencasting with KMS
[2023:09:30:18:20:00]: Info: Found monitor for DRM screencasting
[2023:09:30:18:20:00]: Info: SDR color coding [Rec. 601]
[2023:09:30:18:20:00]: Info: Color depth: 8-bit
[2023:09:30:18:20:00]: Info: Color range: [JPEG]
[2023:09:30:18:20:00]: Error: [AVHWDeviceContext @ 0x561402baf280] Cannot load libcuda.so.1
[2023:09:30:18:20:00]: Error: [AVHWDeviceContext @ 0x561402baf280] Could not dynamically load CUDA
[2023:09:30:18:20:00]: Error: Failed to create a CUDA device: Operation not permitted
[2023:09:30:18:20:00]: Info: Screencasting with KMS
[2023:09:30:18:20:00]: Info: Found monitor for DRM screencasting
[2023:09:30:18:20:00]: Info: SDR color coding [Rec. 601]
[2023:09:30:18:20:00]: Info: Color depth: 8-bit
[2023:09:30:18:20:00]: Info: Color range: [JPEG]
[2023:09:30:18:20:00]: Error: [AVHWDeviceContext @ 0x561402987ac0] Cannot load libcuda.so.1
[2023:09:30:18:20:00]: Error: [AVHWDeviceContext @ 0x561402987ac0] Could not dynamically load CUDA
[2023:09:30:18:20:00]: Error: Failed to create a CUDA device: Operation not permitted
[2023:09:30:18:20:00]: Info: Screencasting with KMS
[2023:09:30:18:20:00]: Info: Found monitor for DRM screencasting
[2023:09:30:18:20:00]: Info: SDR color coding [Rec. 601]
[2023:09:30:18:20:00]: Info: Color depth: 8-bit
[2023:09:30:18:20:00]: Info: Color range: [JPEG]
[2023:09:30:18:20:00]: Error: [AVHWDeviceContext @ 0x561402b43f80] Cannot load libcuda.so.1
[2023:09:30:18:20:00]: Error: [AVHWDeviceContext @ 0x561402b43f80] Could not dynamically load CUDA
[2023:09:30:18:20:00]: Error: Failed to create a CUDA device: Operation not permitted
[2023:09:30:18:20:00]: Info: Screencasting with KMS
[2023:09:30:18:20:00]: Info: Found monitor for DRM screencasting
[2023:09:30:18:20:00]: Info: SDR color coding [Rec. 601]
[2023:09:30:18:20:00]: Info: Color depth: 8-bit
[2023:09:30:18:20:00]: Info: Color range: [JPEG]
[2023:09:30:18:20:00]: Error: [AVHWDeviceContext @ 0x561402b30340] Cannot load libcuda.so.1
[2023:09:30:18:20:00]: Error: [AVHWDeviceContext @ 0x561402b30340] Could not dynamically load CUDA
[2023:09:30:18:20:00]: Error: Failed to create a CUDA device: Operation not permitted
[2023:09:30:18:20:00]: Info: Encoder [nvenc] failed
[2023:09:30:18:20:00]: Info: Trying encoder [software]
[2023:09:30:18:20:00]: Info: Screencasting with KMS
[2023:09:30:18:20:00]: Info: Found monitor for DRM screencasting
[2023:09:30:18:20:00]: Info: Screencasting with KMS
[2023:09:30:18:20:00]: Info: Found monitor for DRM screencasting
[2023:09:30:18:20:00]: Info: SDR color coding [Rec. 601]
[2023:09:30:18:20:00]: Info: Color depth: 8-bit
[2023:09:30:18:20:00]: Info: Color range: [JPEG]
[2023:09:30:18:20:00]: Info: [libx264 @ 0x561402ac0400] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[2023:09:30:18:20:00]: Info: [libx264 @ 0x561402ac0400] profile High, level 4.2, 4:2:0, 8-bit
[2023:09:30:18:20:00]: Info: [libx264 @ 0x561402ac0400] frame I:1     Avg QP: 6.21  size:  1169
[2023:09:30:18:20:00]: Info: [libx264 @ 0x561402ac0400] mb I  I16..4: 100.0%  0.0%  0.0%
[2023:09:30:18:20:00]: Info: [libx264 @ 0x561402ac0400] 8x8 transform intra:0.0%
[2023:09:30:18:20:00]: Info: [libx264 @ 0x561402ac0400] coded y,uvDC,uvAC intra: 0.0% 0.0% 0.0%
[2023:09:30:18:20:00]: Info: [libx264 @ 0x561402ac0400] i16 v,h,dc,p: 99%  0%  1%  0%
[2023:09:30:18:20:00]: Info: [libx264 @ 0x561402ac0400] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu:  0%  0% 100%  0%  0%  0%  0%  0%  0%
[2023:09:30:18:20:00]: Info: [libx264 @ 0x561402ac0400] i8c dc,h,v,p: 100%  0%  0%  0%
[2023:09:30:18:20:00]: Info: [libx264 @ 0x561402ac0400] kb/s:561.12
[2023:09:30:18:20:00]: Info: 
[2023:09:30:18:20:00]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2023:09:30:18:20:00]: Info: 
[2023:09:30:18:20:00]: Info: Found H.264 encoder: libx264 [software]
[2023:09:30:18:20:00]: Info: Configuration UI available at [https://localhost:47990]
[2023:09:30:18:20:00]: Info: Adding avahi service Sunshine
[2023:09:30:18:20:00]: Info: Avahi service Sunshine successfully established.

Answered by cgutman

Oct 2, 2023

I have a fix for this bug in #1694. Please test that and check that it fixes the warnings.

Test packages are available here: https://github.com/LizardByte/Sunshine/actions/runs/6376308574?pr=1694

View full answer

willthong · 2023-10-01T08:19:57Z

willthong
Oct 1, 2023

Same issue, same OS, with a 7900 XT. I've raised it on Discord: https://discord.com/channels/804382334370578482/1157705348073934899

0 replies

xTeixeira · 2023-10-01T10:43:05Z

xTeixeira
Oct 1, 2023
Author

Same issue, same OS, with a 7900 XT. I've raised it on Discord: https://discord.com/channels/804382334370578482/1157705348073934899

FYI, if you really need it to work right now you can try downgrading libva to 2.19 along with the mesa and libva-mesa-driver packages. Not sure which version of mesa and libva-mesa-driver you would need, but this worked for me for a while until I eventually had to upgrade my system.

0 replies

xTeixeira · 2023-10-01T10:43:21Z

xTeixeira
Oct 1, 2023
Author

Closed accidentally, sorry.

0 replies

ReenigneArcher · 2023-10-01T13:55:11Z

ReenigneArcher
Oct 1, 2023
Maintainer

If you are using PKGBUILD... whenever you update a dependency of Sunshine you need to recompile Sunshine... so re-install it!

1 reply

xTeixeira Oct 1, 2023
Author

I'm aware. I have recompiled and reinstalled it. I've also confirmed that downgrading libva to 2.19 (building from source) and building mesa against libva 2.19 fixes the issue. However doing the same with libva 2.20 results in the error.

KuleRucket · 2023-10-01T18:27:25Z

KuleRucket
Oct 1, 2023

I get this message at the moment during initialisation but it doesn't seem to prevent vaapi hardware encoding from working afterwards.

0 replies

kodemeister · 2023-10-02T01:56:56Z

kodemeister
Oct 2, 2023

Try to update your system to the latest Mesa 23.2.1 and libva 2.20.0. This solved the issue for me on RX 6700 XT.

Sunshine still keeps spamming the log with error messages but vaapi seems to work well now.

[2023:10:02:07:42:31]: Info: Screencasting with KMS
[2023:10:02:07:42:31]: Info: Found monitor for DRM screencasting
Xlib:  extension "DRI2" missing on display ":1".
libva error: vaGetDriverNames() failed with operation failed
[2023:10:02:07:42:31]: Error: Couldn't initialize va display: operation failed
[2023:10:02:07:42:31]: Warning: Monitor 0 doesn't support hardware encoding. Reverting back to GPU -> RAM -> GPU

1 reply

xTeixeira Oct 2, 2023
Author

Tried it. No change. I get the same warnings as in the original post and libx264 software encoding is used as fallback.

cgutman · 2023-10-02T05:45:08Z

cgutman
Oct 2, 2023

I have a fix for this bug in #1694. Please test that and check that it fixes the warnings.

Test packages are available here: https://github.com/LizardByte/Sunshine/actions/runs/6376308574?pr=1694

3 replies

xTeixeira Oct 2, 2023
Author

With this PR the warnings are gone and hevc hardware encoding via va-api works correctly. Thank you!

willthong Oct 2, 2023

I have found the same. Thanks so much @cgutman!

cgutman Oct 3, 2023

Thanks for verifying. The fix is now merged to nightly.

LizardByte-bot · 2023-10-03T00:03:21Z

LizardByte-bot
Oct 3, 2023
Maintainer

This issue has been fixed and will be available in the next release.

0 replies

VAAPI fails: vaGetDriverNames() failed with operation failed #316

Uh oh!

Uh oh!

Is there an existing issue for this?

Is your issue described in the documentation?

Is your issue present in the nightly release?

Describe the Bug

Expected Behavior

Additional Context

Host Operating System

Operating System Version

Architecture

Sunshine commit or version

Package

GPU Type

GPU Model

GPU Driver/Mesa Version

Capture Method (Linux Only)

Config

Apps

Relevant log output

Replies: 0 comments · 13 replies

Uh oh!

Uh oh!

xTeixeira Oct 1, 2023 Author

Uh oh!

xTeixeira Oct 1, 2023 Author

Uh oh!

ReenigneArcher Oct 1, 2023 Maintainer

Uh oh!

xTeixeira Oct 1, 2023 Author

Uh oh!

Uh oh!

Uh oh!

xTeixeira Oct 2, 2023 Author

Uh oh!

Uh oh!

xTeixeira Oct 2, 2023 Author

Uh oh!

Uh oh!

Uh oh!

LizardByte-bot Oct 3, 2023 Maintainer

Replies: 0 comments 13 replies

xTeixeira
Oct 1, 2023
Author

xTeixeira
Oct 1, 2023
Author

ReenigneArcher
Oct 1, 2023
Maintainer

xTeixeira Oct 1, 2023
Author

xTeixeira Oct 2, 2023
Author

xTeixeira Oct 2, 2023
Author

LizardByte-bot
Oct 3, 2023
Maintainer