Pen Input Support on Views

[illeatmyhat]

Summary

Previous APIs support advanced graphics tablet features, such as querying pressure sensitivity and pen tilt from pen input. A new, cross-platform API should mirror or take the best parts of some of those previous APIs.

API Changes

I don't see any reason why it should be any different from UWP implementation, lifted straight from the docs:

String queryPointer(PointerPoint ptrPt)
{
    String details = "";

    switch (ptrPt.PointerDevice.PointerDeviceType)
    {
        case PointerDeviceType.Mouse:
            details += "\nPointer type: mouse";
            break;
        case PointerDeviceType.Pen:
            details += "\nPointer type: pen";
            if (ptrPt.IsInContact)
            {
                details += "\nPressure: " + ptrPt.Properties.Pressure;
                details += "\nrotation: " + ptrPt.Properties.Orientation;
                details += "\nTilt X: " + ptrPt.Properties.XTilt;
                details += "\nTilt Y: " + ptrPt.Properties.YTilt;
                details += "\nBarrel button pressed: " + ptrPt.Properties.IsBarrelButtonPressed;
            }
            break;
        case PointerDeviceType.Touch:
            details += "\nPointer type: touch";
            details += "\nrotation: " + ptrPt.Properties.Orientation;
            details += "\nTilt X: " + ptrPt.Properties.XTilt;
            details += "\nTilt Y: " + ptrPt.Properties.YTilt;
            break;
        default:
            details += "\nPointer type: n/a";
            break;
    }

    GeneralTransform gt = Target.TransformToVisual(page);
    Point screenPoint;

    screenPoint = gt.TransformPoint(new Point(ptrPt.Position.X, ptrPt.Position.Y));
    details += "\nPointer Id: " + ptrPt.PointerId.ToString() +
        "\nPointer location (parent): " + ptrPt.Position.X + ", " + ptrPt.Position.Y +
        "\nPointer location (screen): " + screenPoint.X + ", " + screenPoint.Y;
    return details;
}

Intended Use Case

This API can be used optimally for implementing drawing applications.
Drawing applications are more than just e.g. Krita or Clip Studio Paint.
Researchers and hobbyists often create extremely primitive drawing applications as a part of a showcase of what they've made (esp. in machine learning)
Primitive drawing applications are also common in video games (e.g. Pictionary, Jackbox)

WPF has StylusPointProperties, and UWP has PointerPointProperties. WinUI3 also has PointerPointProperties
Likewise, GTK provides GdkDevice, Qt provides QTabletEvent.
Apple provides the much larger NSEvent, along with PencilKit
There's also the PointerEvent Web API

The number of graphics tablet manufacturers is exploding beyond Wacom, Huion, XP-Pen, et al, so it's inadvisable for developers to implement each proprietary API. Ideally we should be able to use a single cross-platform API for handling pen input.

[ghost]

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

[williambohrmann3]

I think these APIs very much make sense for .NET MAUI, considering the multi-platform, multi-formfactor nature of the framework. For example, this would be powerful in a PointerGestureRecognzier_PointerEntered event handler - I might want to have different logic for a mouse than a pen.