Provide APIs for XR(AR/VR) compositor/WM use cases (unconstrained from 2D)

[technobaboo]

Hi friends, am trying to take this project seriously because I know many others won't...

so, i'm working on an XR (AR/VR) wayland compositor with its own display protocol (for the 3D elements) but I want to maintain Xorg support too ideally and I was wondering... since there's an X server that is in active development what could be done to adopt the things Wayland makes trivially easy for XR?

Specifically, single screen spanning multiple displays with every window having absolute coordinates just does NOT translate at ALL to 3D! there's no good way to translate it over as everything in 3D should have a 6DoF pose (3 position values and ideally a quaternion for rotation) and trying to map that in a way that makes sense with X's window parenting and overlap rules is highly confusing and it's no wonder only 1 person ever tried to make an X compositor for XR...

I don't know the deep details of X's architecture but I am curious what could be done to fix this issue... is there a way to make all windows exempt from the layering constraints of X? can windows placed relative to their parents be turned into surface local coordinates? is there a way to take windows at the edge of a display and let me interpret that in a different way or am I stuck using heuristics and guessing?

Wayland's lack of absolute coordinates makes this incredibly easy for me, as I can store my own for each toplevel surface and not worry about screens at all (as i don't support layer-shell)

tl;dr issue is xorg's global screen 2D absolute coordinate system not mapping well to infinite canvas 3D

[dec05eba]

There was a patch long ago made to support use-cases like this (at that time for 3d compositors) by I believe applying a matrix to either global input coordinate or window planes for input. Maybe somebody could find the patch and apply it.

[technobaboo]

hmm how would that help? it's a different cordinate system so it'd be better to fully decouple it (as honestly the X server does not understand 3D space well enough to work in XR) and have custom compositing/rendering entirely.... what I really need is the way to decouple windows from the 2D coordinate system (and yeah i know it''s pretty fundamental but idk a better way to do it)

[dec05eba]

it would turn the coordinate system of the window 3d and allow you to map it to your 3d world objects

[technobaboo]

afaik it'd allow for relative transformation of the window but

a) there's no such thing as a screen in XR, so relative to a screen isn't helpful
b) there's many different reference spaces that aren't just the screen
c) afaik the WM would need to do this but not have enough information to decide what parent/child windows are to map it properly?

[dec05eba]

IIRC the patch does address all of those issues. I will have to find it to verify. I have made my own VR window manager for X11, so I understand the issues you are talking about.

[technobaboo]

wait you're the 1 person who made a VR WM for X? wow how'd you pull that off?

[dec05eba]

I used randr hacks and faked mouse input by controlling it with software and controlling which window were visible on the screen to map that to input, etc, but yes it was jank

[callmetango]

Moving this to X11Libre Ideas · Discussions · GitHub to refine this further.

[callmetango]

Thank you for your contribution! We currently restructured the "Ideas" discussions and accordingly this discussion will be moved to the Good Ideas For Later category.