Native Camera and IO access

[genifycom]

I love the idea of the Blazor Hybrid and am actively trying it out.

The current preview is in two separate projects. A .NET Standard Library project that does all of the display (Xamarin and Blazor) and a Xamarin Forms project that initiates the .NET Standard Library display project.

Currently the "native" piece is in the .NET Standard Library project and this is how the Counter is shared in this one project between both a Forms and a Blazor window.

I have two requirements that I would love to see. These both require native Android elements.

1. The ability to take a picture. Of course I can do this in the Android Forms project, but I do not yet see how to trigger this from the .NET Standard Library Application. Then I want to open the resulting picture in the Blazor window.
2. The ability to read from System.IO (e.g. barcode scanner). Again this can be done in the Forms project, but how do I get this information from one project to the other? I need to trigger the read and return the result back to the Razor component.

Am I missing a communication piece between the two projects? I do not see a linkage that we can event on or call. e.g. BlazorHybridAndroid.Init(); returns a void.

Thanks for any help.

[BenMakesGames]

I was able to view my laptop's camera, using OpenCVsharp (https://github.com/shimat/opencvsharp) to grab the camera, and setting an <img> tag's src to the base64-encoded frames. code-behind:

        public void DoStartVideo() // button handler
        {
            CameraState = CameraStateEnum.TurningOn; // used to control appearance of buttons in .razor
            Camera = new Thread(CaptureCamera);
            Camera.Start();
        }

        public void DoStopVideo() // button handler
        {
            IsCapturingVideo = false;
        }

        private void CaptureCamera()
        {
            // OpenCV time!
            var frame = new Mat();
            
            Video = new VideoCapture(0);
            Video.Open(0);

            if(Video.IsOpened())
            {
                CameraState = CameraStateEnum.On;
                IsCapturingVideo = true;

                while (IsCapturingVideo)
                {
                    Video.Read(frame);

                    if (!frame.Empty())
                    {
                        byte[] imageBytes = frame.ImEncode(); // encoded as .png by default

                        // CameraImg is used in .razor file: <img src="@CameraImg" width="640" height="480">
                        var base64 = Convert.ToBase64String(image);
                        CameraImg = $"data:image/png;base64,{base64}";

                        InvokeAsync(StateHasChanged); // tell Blazor that it needs to re-render
                    }

                    Thread.Yield();
                }

                Video.Release();
            }

            CameraState = CameraStateEnum.Off;
        }

this feels very inefficient to me. the framerate was fine in my simple application, but I think better performance could be achieved using a WebGL canvas. I made a couple attempts at rendering to a WebGL canvas following these tutorials, but couldn't get it working. my own failing, I think; not that of the tutorials. maybe you'll have better luck:

• https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/Tutorial/Animating_textures_in_WebGL
• https://webglfundamentals.org/webgl/lessons/webgl-2d-drawimage.html
• https://github.com/BlazorExtensions/Canvas (NuGet package for talking to a Canvas)

an important warning about BlazorExtensions.Canvas: it has a known bug in the TexImage2DAsync method: they're missing a "border" parameter, and some deserialization logic needed for TexImage2DAsync's pixels parameter. these cause errors during runtime. the authors of the package have promised they'll fix it, but activity has been low. in the meanwhile, I resorted to cloning the project locally and fixing it myself...

• Fixed missing "border" parameter in TexImage2D & TexImage2DAsync (WebGLContext) BlazorExtensions/Canvas#60
• Parameter missing BlazorExtensions/Canvas#96