ESP32-CAM framebuffers, grab_mode explanation?

[namgang]

Running an AI-Thinker ESP32-CAM/OV2640 module using the micropython-camera-API by cnadler86. JPEG image capturing at full resolution works fine, so far so good.

As a newcomer I don't understand how to use multiple framebuffers and how they interact with the grab mode. "GrabMode controls when a new frame is grabbed". Grabbed by whom? To where? The doco seems to mention a queue of framebuffers but I don't see any functions explicitly related to a queue. Does the capture function return a framebuffer? Where is the JPG conversion done? I'd like to have a basic understanding of the internal machinery in order to write efficient code and I don't see it in the API docs.

MicroPython v1.25.0 on 2025-04-16; Generic ESP32 module with SPIRAM with ESP32

[cnadler86]

I would like to answer this myself, but the AI did a great job on that.
Here the link on the explanation. Note that the information from chatgpt might be wrong in some details, but the explanation on those parameters is very accurate.

There is also an async version of the capture method in the acamera module, which should work in a non blocking way with a fb_count=2.

Some of the sensors (e.g. ov2640) encode the image itself, so this happens at the sensor and not in your board. If you would like to decode, use mp_jpeg for that.

I will upload a stub file for typing in the next days.

[namgang]

Great. After reading through the ChatGPT ramblings once or twice I think I'm getting it. So, camera.init() is when the camera starts recording, pushing frames to the queue of buffers, subject to buffers being available. camera.capture() pulls a frame out of the queue of buffers (which might have length 1). One holds on to the buffer, possibly stopping new frames, until camera.free_buffer().

The concluding ChatGPT remark seems like a useful hint: Switching to fb_count=2 and grab_mode=GrabMode.LATEST is usually the sweet spot for fast, low-latency JPEG capture in applications or web streaming.

Would you like to confirm (or correct)? I hesitate to mark this as an answer until my assumptions have been double-checked.

[cnadler86]

Yes, that's right. fb_count=2 and Latest gives the best performance, but of course more stress in the chip. So it depends on your application. Never the less, it is on moast of the cases better to capture in jpeg and if needed the convert the image into another format with mp_jpeg.

I recommend using the async version of the camera, because it doesn't block until a frame buffer is available, but this means you have to run your code on an async loop.