Texture with a hole in it

[JeffM2501]

Here is a simple example of how to build a render texture with a transparent hole in it.
It involves using RLGL and OpenGL blend modes (Note the #defines at the top of the code).

/*******************************************************************************************
*
*   raylib [solutions] example - Texture With a Hole in it
*
*   Enjoy using raylib. :)
*
*   This example has been created using raylib 4.0 (www.raylib.com)
*   raylib is licensed under an unmodified zlib/libpng license (View raylib.h for details)
*
*   Copyright (c) 2021 Jeffery Myers
*
********************************************************************************************/

#include "raylib.h"
#include "rlgl.h"
#include "raymath.h"

// constants from OpenGL
#define GL_SRC_ALPHA 0x0302
#define GL_MIN 0x8007

int main(void)
{
    // Initialization
    //--------------------------------------------------------------------------------------
    const int screenWidth = 800;
    const int screenHeight = 450;

    InitWindow(screenWidth, screenHeight, "raylib [core] example - basic window");

    // make a basic checkerboard texture to use as a background
    Image checked = GenImageChecked(screenWidth, screenHeight, 20, 20, GREEN, DARKGREEN);
    Texture solidTexture = LoadTextureFromImage(checked);
    UnloadImage(checked);

    // make a render texture that we can put a hole into at the mouse pos
    RenderTexture holeTexture = LoadRenderTexture(screenWidth, screenHeight);

    SetTargetFPS(60);               // Set our game to run at 60 frames-per-second
    //--------------------------------------------------------------------------------------

    // Main game loop
    while (!WindowShouldClose())    // Detect window close button or ESC key
    {
        // update the hole to be where the mouse is
        BeginTextureMode(holeTexture);
        ClearBackground(BLACK);

        // force the blend mode to only set the alpha of the destination
        rlSetBlendFactors(GL_SRC_ALPHA, GL_SRC_ALPHA, GL_MIN);
        rlSetBlendMode(BLEND_CUSTOM);

        // draw a blank 'hole' in our texture
        DrawCircle(GetMousePosition().x, GetMousePosition().y, 50, BLANK);

        // go back to normal
        rlSetBlendMode(BLEND_ALPHA);

        EndTextureMode();

        // Draw
        //----------------------------------------------------------------------------------
        BeginDrawing();
            ClearBackground(RAYWHITE);

            DrawTexture(solidTexture, 0, 0, WHITE);

            // render textures are upside down from the screen, so we need to flip it to have it display correctly
            DrawTexturePro(holeTexture.texture, (Rectangle) { 0, 0, screenWidth, -screenHeight }, (Rectangle) { 0, 0, screenWidth, screenHeight }, Vector2Zero(), 0, WHITE);

        EndDrawing();
        //----------------------------------------------------------------------------------
    }

    // cleanup
    UnloadRenderTexture(holeTexture);
    UnloadTexture(solidTexture);
    return 0;
}

[JeffM2501]

The reason you have to change the blend mode is that by default the alpha channel of a destination buffer is unmodified. To get this effect we need our drawing to affect the alpha channel, and not just blend with the color channels. So we have to change the blend mode.
rlSetBlendFactors(GL_SRC_ALPHA, GL_SRC_ALPHA, GL_MIN);
This mode says "take the smallest value from the source or destination and make that be in the destination alpha value"
So when we draw the circle with an alpha value of 0, it changes the alpha channel in the destination texture.

The pixels in the circle have an alpha of 0, the pixels in the texture have an alpha of 1. So the mode compares the two, 0 is the smallest value of 0 and 1, so it writes 0 to the alpha channel for those pixels.
If you were to then draw another shape with 50% opacity over the same pixels, it would do the same comparison, 0.5 for the new pixel, 0 for the existing pixel, 0 is smaller, so 0 gets written.

This mode ensures that the most transparent pixels are kept regardless of what is drawn.

[raysan5]

Hey! That's a nice example!