Confined cursors "stick" to region borders

[emekoi]

Copied from niri-wm/niri#3128

In niri, when a client attempts to confine the cursor to a given region, and the mouse approaches the borders of the region, it never actually reaches the borders of the region and instead gets stuck right before it. This happens in floating and tiled windows, but not in fullscreen windows as long as there is not a another monitor sharing region edge. This issue occurs with Wine, SDL3, and libwayland applications (I think Wine and SDL3 both use libwayland). The program in the video is in this gist. I don't have a good way of recording two monitors at once, so that isn't included in the video but the behavior on that shared edge is the same as in the floating case shown.

This behavior is reproducible in Anvil as well.

out.mp4

[YaLTeR]

It's not really a Smithay issue. It's just the simplest way to deal with the fact that the confine region might be an arbitrary region, not just a rectangle. Though probably in most cases it is just one rectangle, and could be special cased

[emekoi]

From what I can tell, the region must be "rectangle-like" since you can only add and subtract rectangles from wl_regions and the confinement region must be a wl_region. If I remove some rectangles from the top left and bottom right corners, the cursor still sticks to those edges in Niri and Anvil but not in other compositors.

[YaLTeR]

How do they clip the cursor to an arbitrary region (even one composed of rectangles)?

[emekoi]

From more testing, it seems only wlroots compositors do the non-rectangular region case correctly. Hyprland, starts warping the cursor all over the window. All the wlroots compositors seem to just use this code in wlroots:

• sway
• river
• dwl

[PolyMeilex]

@emekoi Hi, could you try Anvil from the improve-pointer-constrains branch? I'm playing around with different approaches, seems to run better so far.

[emekoi]

It is better but it only constrains the cursor on the topmost and leftmost edges. I haven't been able to test it with compund regions since Anvil doesn't seem to like my test program.

[emekoi]

I currently have a working POC, but I'm not sure how to handle the case when no region is set and the pointer should be confined to the input region. I tried using with_state and retrieving the current SurfaceAttributes, but the input region stored in it is alwaysNone.

[PolyMeilex]

Input region is optional, by default the input region covers the whole surface.
As far as I remember, you can get the surface size like so (minus the unwraps 😄 ):

use smithay::backend::renderer::utils::RendererSurfaceStateUserData;
use smithay::wayland::compositor::with_states;

let size = with_states(surface, |states| {
    let data = states.data_map.get::<RendererSurfaceStateUserData>().unwrap();
    let data = data.lock().unwrap();
    data.surface_size().unwrap()
});

dbg!(size);