GetVisualTreeElements uses "platform units" not pixels. VisualElement.Bounds uses pixels. How to convert? What are "platform units"?

[jonmdev]

I am trying to develop my own Gesture/Click methods using GetVisualTreeElements(point) here: https://learn.microsoft.com/en-us/dotnet/api/microsoft.maui.ivisualtreeelement?view=net-maui-7.0

However, results of my preliminary testing show this doesn't match up to VisualElement.Bounds. Ie. Using GetVisualTreeElements(point) with a point that should be within an element's bounds does not return that object.

It says on that page re: using a rectangle as an input, the input is "defined by its coordinates which are specified in platform units, not pixels." I am guessing this applies for using a Point as well.

I cannot find any information on what a "platform unit" is or how to convert this to pixels. Any guidance or resource? Thanks.

[jonmdev]

Figured it out.

https://github.com/dotnet/maui/tree/main/src/Core/src/WindowOverlay

Has all the methods for accessing click and touch data in a way that corresponds to GetVisualTreeElements(point).

Thank god (or whoever made it) for that class as I would have gone crazy trying to figure it out all on my own.

[Knexxxx]

I have the same problem. I want to define another movable pointer (that is different from the OS-pointer). For this new pointer, I need to know what elements (like buttons) lie beneath it, in order to be able to trigger events and apply actions.
would you have some kind of example how you use the WindowOverlay class please? Presently I do not see how it could be useful.

[jonmdev]

The WindowOverlay will not be usable directly for this. You will need to create your own class similar to the WindowOverlay with your own code in it. You can copy and paste the WindowOverlay class as a starting point though. It won't be fun, speaking from experience. Perhaps someone has another easier way. I can't share my code as it's proprietary, sorry.

[jonmdev]

I have the same problem. I want to define another movable pointer (that is different from the OS-pointer). For this new pointer, I need to know what elements (like buttons) lie beneath it, in order to be able to trigger events and apply actions. would you have some kind of example how you use the WindowOverlay class please? Presently I do not see how it could be useful.

If you are working within the general conventional Maui system and can use an object as a "click detector" with a tap gesture recognizer when needed, probably the best solution for you would be to use something simple like this:

TapGestureRecognizer tap = new();
tap.Tapped += delegate (object o, TappedEventArgs e) {
    Point pos = e.GetPosition(objectToGetPositionRelativeTo) ?? new(0,0); //can perhaps use ContentPage or Page as objectRelativeTo if you want absolute screen position here, may need to adjust further for safe space of top of mobile screen
    VisualTreeElementExtensions.GetVisualTreeElements(veToGetFrom, pos);
};

This uses:
https://learn.microsoft.com/en-us/dotnet/api/microsoft.maui.visualtreeelementextensions.getvisualtreeelements?view=net-maui-8.0

Which will return an array of objects at that point.

Probably some variation of this code will help solve your issue in a simple manner, if it is a simple case. Otherwise you will have to do something more complex.

[Knexxxx]

Thanks jonmdev. Using "VisualTreeElementExtensions.GetVisualTreeElements(veToGetFrom, pos)" I am able to iterate through the found elements and find the one underlying button. It works very well when the pangesture (pointer moving) is completed. With GestureStatus.Running, the movement of the cursor gets laggy and is not smooth anymore.

Now the next step will be to fire button events like PointerEntered (in order to give that button some visible focus), PointerExited and Clicked. Probably a custom behavior would be suitable for that, or a trigger?

[jonmdev]

To be honest I'm not sure about anything more than that as I don't use GestureRecognizers usually. It was just to give you an idea. But certainly that function VisualTreeElementExtensions.GetVisualTreeElements is very expensive depending on how deep the hierarchy you evaluate is. So if you are running it while dragging on ever drag motion on every position update it will slow you down.

You would want to run such a thing as least often as possible. And if you run it on the deepest relevant branch of your hierarchy it will be more efficient as well (less children to run through).

[Knexxxx]

thanks for you advice! the pointer control works really well this way. To also use the position with GestureStatus.Running (meaning: while doing the pangesture) I just had the idea to "plot a button map" using GetVisualTreeElements not continously, but only once (e.g. every time the page layout changes). This way, only a static ready-to-use-button map needs to be checked against the cursor position, which would probably use way less resources.
I will give it a try!