Quest 2 Crashes on Selecting NDI Source - Unity 2022 Build

[eeease]

Upon selecting an NDI source on Quest 2, the system crashes instead of rendering the source.

This was tested with many different variables, the constant one being Unity version:

• The sample Klak NDI scene that comes with the package (with a dropdown, receiver, sender, and canvas to render the video)
• Unity 2022.3.4f1
• Android API 30, 31, 32, 33
• URP
• No URP
• ARM64
• NDI Launcher Tools Test Pattern for Windows, same Wi-fi network
• NDI Launcher Tools VLC video that is confirmed to work in Editor and Standalone builds
• MSAA on
• MSAA off
• Color Space: Gamma
• Color Space: Linear
• Graphics API: OpenGLES3 (which works in our 2020 version of the project)

Notes:

1. NDI rendering works fine in Editor (2022) and in desktop standalone builds (Windows, Mac).
2. Our project includes the changes from PR191, which were needed in order to detect sources in API 31+.
3. Also includes changes from PR186.
4. The changes from PR191 and PR186 work when we make builds from Unity 2020, API 30, 31, 32.

Logcat logs from when I select an NDI source:

Any suggestions or help would be greatly appreciated. We want to update our project to Unity 2022, but we need NDI streaming to work on Quest 2 builds.

Thanks in advance!

[keijiro]

First of all, I don't have a Quest 2 device, so unfortunately, I can't investigate the issue on my side.

I haven't had a firm idea, but I guess the issue is Unity version-specific because it runs on Unity 2020.

How about updating the project to the latest version of Unity 2022.3? Unity 2022.3.4 is a bit too outdated.

[eeease]

Thanks for the reply! At your suggestion, I tried making a build with Unity 2022.3.20f1, which seems to be the latest LTS version of 2022.
The same crash occurs when I select an NDI source from the dropdown on Quest.

We know that Vulkan works but we currently have a dependency on OpenGLES3 and we're unsure why OpenGLES3 works in 2020 but not 2022.

[keijiro]

How about changing the resolution of the video stream?

I still don't have any idea about the issue, but I guess it's related to the memory alignment of something. I wonder if it works with "compute friendly" dimensions like 256x256 or 1024x1024.

[eeease]

Good suggestion!
I tried a 1024x1024 video (modified via Handbrake) and the issue still occurs.
It's worth noting that I also tried making a build from Unity 2021.3.35f1, same issue.

For NDI source, I've also been testing using the Test Patterns from the official NDI Tools Launcher via Windows. Everything - test patterns, videos from VLC, etc - works fine from Unity 2020.

[keijiro]

How about disabling the compute shaders? You can disable them by commenting out the kernel functions in Decoder.compute.

https://github.com/keijiro/KlakNDI/blob/main/jp.keijiro.klak.ndi/Runtime/Resource/Decoder.compute

You should comment out only the body of the kernel functions -- You shouldn't comment out the function declarations.

[eeease]

I commented out the body of the following functions:
DecodeUYVY_sRGB
DecodeUYVY_Linear
DecodeUYVA_sRGB
DecodeUYVA_Linear

I made an Android build and attempted to receive 1) NDI Test Pattern and 2) VLC video.

The performance did not get majorly impacted when I selected a source.
However, the visuals didn't appear on the Canvas.

Did I comment out too much? (I'm not too knowledgeable of shaders.)
Hopefully that helps point in the right direction.

[keijiro]

Thanks for testing it.

However, the visuals didn't appear on the Canvas.

That's the intended behavior. I just wanted to check if the compute shader is relevant.

You turned it off, and the crash disappeared. It indicates that the crash happens in the compute shader or how GLES3 handles it.

[keijiro]

How about adding alpha channels to the sending frames? It's relevant because there are two versions of compute shaders (UYVY - without alpha, UYVA - with alpha)

You should revert commenting-out before testing it.

[eeease]

Hey @keijiro , I'm happy to try that out but not 100% sure what you mean.
Do you want me to change something in the code? I'm not a shader programmer so I'm not sure exactly how to add alpha channels to the sending frames.

Or should I try streaming something that has alpha?

[keijiro]

Just turning on/off the "Keep Alpha" switch on the sender component is enough for testing it.

[eeease]

Got it. When I select a source with the Keep Alpha bool set to true, the same issue occurs. I will try to capture the issue without revealing any of the protected images in our app.

[eeease]

NDICanvasQuest2.mp4

Here's a video of what happens when I select a source. I tried to capture the screen-tearing that I see in the headset, but it seems every way I try to capture video, the Quest just stops processing when the NDI source is selected.

[keijiro]

By the way, I wonder if GLES3 on Quest supports unordered access views (UAV), which is vital for the compute shaders. Could you check if SystemInfo.SupportsRandomWriteOnRenderTextureFormat returns true on the device?