FrameBuffer::GFX RGB888 XY framebuffer compatible with FastLED 2D/Adafruit::GFX/LEDMAatrix

[marcmerlin]

Howdy, I'm the author of https://github.com/marcmerlin/Framebuffer_GFX
More generally, it's the base lib for around 10 output libs so that code written against it can be output on lots of displays, including RGB Panels.
Details here: http://marc.merlins.org/perso/arduino/post_2020-03-16_Framebuffer_GFX_-Choosing-between-its-3-2D-APIs_-FastLED-XY_-NeoMatrix_-and-LEDMatrix_-and-detail-of-its-many-supported-hardware-backends.html

Basically the exact same code runs like this

(don't miss the small TFT at the bottom).

I already wrote a glue driver for the older SmartMatrix lib which does work on ESP32, but isn't as efficient and why @mrcodetastic wrote this lib a few years back.
I contributed a few early examples to this lib, but eventually switched to the rpi-rgb-panel lib since it has 3 outputs and 3 times the refresh speed or max output resolution.
That said, I've been waiting to write a glue driver for this lib too.

My base framebuffer is 24bits, but looking at this library, while you can change the bpp to 24 if need be, it does not seem to have an easy X x Y matrix in 24bpp like you would with FastLED, SmartMatrix or my lib.
For TFTs that are 16bpp and do not have a framebuffer you can read (only write), I had to write this driver https://github.com/marcmerlin/FastLED_SPITFT_GFX which effectively takes my framebuffer and iterates pushing it dot by dot after converting down from 24bpp to 16bpp like done here:
https://github.com/marcmerlin/FastLED_SPITFT_GFX/blob/7d92dbdb79117d5e4e6cb369cfc1fb61cba6141e/FastLED_SPITFT_GFX.cpp#L37

it's sadly a pixel by pixel loop although it pushes one line at the time for SPI speed reasons.

Question for @mrcodetastic and others who know the lib internals:

1. am I correct that the lib cannot be configured to work from a fastled like 24bpp array of the full framebuffer?
2. if not, it looks like I'll need a similar loop pushing line by line for every frame for each frame refresh on the drawing side ( matrix.show() ) . If so, what are the best lib methods I can use to push 24bpp line by line if that's the best way?
3. more generally is there a code example or sample that could be close to my "push a whole frame of bits"? I found MatrixPanel_I2S_DMA::updateMatrixDMABuffer in the code which does a bunch of Cache_WriteBack_Addr pixel by pixel?

I'll admit I don't fully understand what's the best way to push at least one line at a time to the internal framebuffer.
Suggestions very welcome :)

[mrcodetastic]

Hi Marc. Great to see you're still active and around. I did see years back that you took RGB panels to various EDM gigs which I thought was cool. Merry Christmas.

The short answer is: No, there isn't a way to do this, and there can't be.

Long answer:

You’ve hit on the fundamental architectural difference between a standard pixel-mapped framebuffer and a DMA-driven HUB75 driver.

To answer your question directly: No, there is no way to perform a simple memcpy of an RGB buffer into this library. This is because the library doesn't actually store "RGB data" in the way we usually think of it.

The "Apples and Oranges" Problem

Your Framebuffer_GFX and libs like FastLED treat memory as a Color Buffer (an array of pixels). However, ESP32-HUB75-MatrixPanel-DMA treats memory as a Hardware Bitstream.

When you call drawPixel(), the library doesn't just store a value in an array; it "shreds" that color into individual bits and distributes them across multiple DMA Bitplanes.

Why memcpy isn't possible:

1. Binary Code Modulation (BCM): To achieve color depth on HUB75 panels, the library uses BCM. If you are running 24-bit color, a single pixel's data is scattered across 8 or more different memory locations (one for each bitplane) that the DMA engine cycles through at specific timing intervals.
2. HUB75 Multiplexing: Because these panels are multiplexed (e.g., 1/32 scan), the data for Row 0 and Row 16 is actually bundled together in the same memory byte to be clocked out simultaneously to the shift registers.
3. Hardware Mapping: The internal DMA buffer is structured specifically to match the ESP32's I2S/LCD peripheral pin mapping. It is essentially a "pre-baked" bit-bang sequence.

Suggestions for your Glue Driver

Since there is no "raw RGB" destination to copy to, you will have to iterate through your source framebuffer. To make it as "least-slow" as possible:

• Avoid the GFX Layer: Don't use the standard drawPixel() if you can help it, as it adds overhead for coordinate rotation and bounds checking on every single call.
• The "Line-by-Line" approach: As you saw in updateMatrixDMABuffer, the library has to perform the bit-shredding somewhere. Your best bet is to loop through your 24bpp array and feed the data into the library's internal pixel-mapping functions.
• Parallelism: If you're on an ESP32, you might consider running your framebuffer-to-DMA-buffer conversion on the second core (Core 0) so it doesn't compete with your animation logic on Core 1.

In short, it's the difference between a Pixel Map to a Bitstream. The translation layer (the loop) is the "cost of entry" for using the ESP32's DMA hardware to achieve those high refresh rates and resolutions.

[marcmerlin]

@mrcodetastic thanks for the detailed answer . I should have mentioned that I understand that for RGBPanels the framebuffer gets broken down into bitplanes and I know about BCM, even the very first adafruit driver supported it, as well as my 8x8 direct wire scan display driver I wrote way back when.
If you ever have someone interested at "how does it all work", this is actually the simplest driver and use case: https://marc.merlins.org/perso/arduino/post_2015-01-06_Driver-for-direct-driving-single-to-3-color-LED-Matrices-with-software-PWM.html

But back to the question, other drivers like SmartMatrix or RPI-RGB-Panel, do keep a full resolution full color framebuffer that users write into, and then at render time, uses it to fill the color bitplanes used for BCM.
Questions:

1. Do I understand correctly that basically your driver skips that framebuffer entirely (maybe to save RAM), and has all writes go directly into the bitplanes?
2. If so, does it also mean that your driver does not allow reading back pixels that were written some time ago and are being displayed, because they've been split into color subcomponents and further split into bitplanes (a bit plus of my lib and the demo code I have is that it can read back pixel values from a frame to make the next frame)

But as I write this, since I worked on the Aurora code you are also using as one of your demos (I think I tried to give it to you as a list of examples a long time back), it does read back from the framebuffer, for instance:

    void DimAll(byte value)  {¬
        for (int i = 0; i < num_leds; i++)¬
          leds[i].nscale8(value);¬
    }

I also see

    // we do dynamic allocation for leds buffer, otherwise esp32 toolchain can't link static arrays of such a big size for 256+ matrices
    leds = (CRGB *)malloc((width * height + 1) * sizeof(CRGB));

and
virtualDisp = new VirtualMatrixPanel((*matrix), NUM_ROWS, NUM_COLS, PANEL_RES_X, PANEL_RES_Y, PANEL_CHAIN_TYPE);

and 
    for (int y=0; y<height; ++y){
          for (int x=0; x<width; ++x){
          uint16_t _pixel = XY16(x,y);
          virtualDisp->drawPixelRGB888( x, y, leds[_pixel].r, leds[_pixel].g, leds[_pixel].b);
        } // end loop to copy fast led to the dma matrix

So is this basically what I need/want to do?

• I already have my CRGB array in my lib
• then I need to create a virtualDisp object
• and iterate for every pixel and every frame.show() all the RGB888 pixels to your lib.

Does it sound like the best way to do things?

[mrcodetastic]

Yep. That's all you need to do. You are right in that this library does not have a rgb value framebuffer as it's scope has only been the hardware dma bcm bitstream part because of memory constraints more than anything.

I did also build a separate library of code to provide this functionality on top. It does exactly what you point to however.

https://github.com/mrcodetastic/ESP32-HUB75-MatrixPanel-DMA/blob/master/examples/ScrollingTextLayer/ScrollingTextLayer.ino

https://github.com/mrcodetastic/GFX_Lite

[marcmerlin]

My overlay framebuffer library is now done and published:
https://github.com/marcmerlin/FastLED_ESP32-HUB75-MatrixPanel_FrameBuffer_GFX/tree/main

I will give more examples and why it matters, but the short story is it allows running a huge amount of demos, some had to be ported manually and painstakingly in your lib, whereas with my glue framebuffer lib, I literally only have to change a single ifdef to switch from your lib to rpi-rgb-panel, or a FastLED Matrix, or a TFT, or displaying directly on my linux laptop

[marcmerlin]

Success! 24bpp fastled framebuffer demos now work with my glue library

https://github.com/marcmerlin/FastLED_ESP32-HUB75-MatrixPanel_FrameBuffer_GFX

[mrcodetastic]

This so so cool.

[marcmerlin]

and 2 videos:
https://www.youtube.com/watch?v=1d6U-6twpCk

and this is why coding against my multi API multi backend GFX/FastLED/LEDMatrix triple API makes sense, when you can run your exact same code on bigger arrays with different hardware:
https://youtu.be/MRRJw-Khkh0

[marcmerlin]

Ok, everything is integrated in https://github.com/marcmerlin/FastLED_ESP32-HUB75-MatrixPanel_FrameBuffer_GFX/
along with demo code and screenshots.
https://github.com/marcmerlin/FastLED_ESP32-HUB75-MatrixPanel_FrameBuffer_GFX/
@mrcodetastic I'd love your review and ability to add this in your docs somewhere for those who can benefit from this layer and the multi API + multi hardware backend compatibility it givres

It just needs MATRIX_WIDTH/MATRIX_HEIGHT removed or renamed in your core library since they conflict with so many demos, breaking their compile, and maybe a better name is PANEL_RES_X/Y which won't conflict with end demo/end code.
#891