MAUI Integration with Game Engines with .NET scripting (Unity, Stride, Urho, etc).

[najak3d]

Our goal is to bring MAUI GUI capability to the Gaming World. Most games force you to use their Game Engine GUI, which are usually lacking.

Already, with Avalonia, we've overcome this via a custom Window renderer which exports each Avalonia Window to a Bitmap, with input injected via an adapter interface. Therefore, with Avalonia, you can render it ANYWHERE you can render a bitmap, and it behaves EXACTLY the same. Therefore, for Games (right now doing this with Urho.NET), you can create your Avalonia GUI independent of Urho3D Game engine, and yet, it ports directly overtop the Urho3D OpenGL render window! (Via an OpenGL Texture that covers the screen.)

MAUI, we believe is superior to Avalonia, but we're not sure yet if this same feat can be accomplished with MAUI. We need for MAUI to render each Layout to a Bitmap, and allow and adapter-layer to inject user inputs for processing.

So this Adapter Layer would:

1. Output each MAUI Layout to it's own Bitmap (with transparency). (one bitmap per layout)
2. Permit an adapter layer to inject User Inputs (touch, drag, mouse, keyboard, other) for MAUI to process normally.

===
With this capability, we could simply adapt MAUI to run on ANY Game Engine that supports .NET scripting. The MAUI GUI would be exported to a bitmap, then Blitted to the Game Engine Texture on the GPU. (this type of blitting takes well under< 1 msec)

Is it feasible to do this now? If so, what is the best general approach?

[najak3d]

Answering my own question here now. The answer for MAUI is Maui.Graphics.Controls (i.e. DrawnControls) - which is the Maui Pixel-Perfect solution that output to a Bitmap, which then can be rendered inside a .NET-integrated Game Engine.

Maui.Graphics.Controls is the best thing Maui has going for it now, but it's currently only considered an "experimental add-on".

If you want Pixel-Perfect MAUI to become mainstream, come to this forum, and express your need for this. In short, Pixel-Perfect means that it Looks/Behaves the EXACT SAME on all platforms (Windows, Mac, Android, iOS, and even Web browser!).

Here are the forums for it:
https://github.com/dotnet/Microsoft.Maui.Graphics.Controls/discussions

[najak3d]

New Question though:

In order to make this work, MAUI needs to have a way to integrate a plug-in Renderer, as Avalonia allows. This then allows us to route user Inputs (mouse/keyboard) from the Game Window UI back to Avalonia, so that Urho.NET is the provider of Input, and also routes the Avalonia output to an Urho.NET Texture.

Here's what the plugin initialization code looks like:

Does MAUI have this sort of capability, to allow us to insert our own rendering platform similar to this?

[ParadiseFallen]

maybe create an issue?

[Xyncgas]

I wouldn't say it's the best solution, imo drawing abstract unified UI there should be a component that can be applied attributes to for styling, like

on the web, I was never a fan of all the component libraries people provided in Xamarin because you just have to apply your own style and build your own components and once you tackled it down it becomes easier to do in future projects, since they are not putting their efforts on giving me the ability to draw custom UI they are not building something I want, so I am just gonna using my internal solution where I can do things like this : https://user-images.githubusercontent.com/35312807/149654567-86c5c7d3-c958-4e16-a43b-01f6878e0add.mp4

[anose001]

Has there been any updates on this discussion/suggestion. Have been looking at different solutions for this but it doesn't look the best