v1.1.0

Summary

• The following issues have been resolved:
  • #1
  • #2
  • #5
  • #7
• Added support for the SparkFun Red Vision Kit for RedBoard.
• Added a DVI/HDMI driver for RP2350 devices using the HSTX peripheral.
• The OV5640 can now be used with the RP2350 at full speed (320x240 at ~30FPS) after improvements to the DVP_RP2_PIO driver now better handling image buffers in PSRAM.
• The camera and display driver structures have been overhauled, with separate display/camera "device" drivers that each encapsulate a respective "interface" driver.
• Added new VideoCapture and VideoDisplay classes to more closely mirror standard OpenCV, which encapsulate a respective device driver.

New and Improved Drivers

• RP2350 devices can now interface with DVI and HDMI displays using the new HSTX driver.
• The DVP_RP2_PIO driver is also improved to better handle image buffers stored in PSRAM, enabling the OV5640 to run at full speed (~30 FPS at 320x240).

During development of the DVI driver, we learned a lot about how the RP2350 handles transfers to/from PSRAM, and especially the limitations of DMA tranfers when PSRAM is involved. Every time a transfer happens with data in PSRAM, if a memory mapped read/write is performed, a long bus delay can occur while the QSPI Memory Interface (QMI) is performing the transfer. This can cause long stalls for all DMA channels, as pointed out in section 4.4.3 of the RP2350 datasheet in the second paragraph:

...QMI serial transfers force lengthy bus stalls on the DMA... stalling the DMA prevents any other active DMA channels from making progress during this time, slowing overall DMA throughput.

Additionally, bulk PSRAM tranfers are typically limited to 37.5MBps due to the QSPI bus typically running at 75MHz, which imposes further limitations. To work around these things, the DVI driver uses multiple DMA channels with a complex sequence of control blocks to handle the entire transfers, utilizing the XIP streaming interface to avoid stalling the DMA controller. One DMA channel transfers each row of pixels from PSRAM to SRAM, while another DMA channel transfers the row buffer from SRAM to the HSTX FIFO. This ensures the HSTX FIFO always has data, but other limitations can occur. See the DVI_RP2_HSTX driver for more information.

After the DVI driver, we implemented similar principles to the DVP_RP2_PIO camera driver, which now enables the OV5640 to operate at full speed. Previously, the DMA was transferring directly from the PIO FIFO to PSRAM, resulting in bus stalls that caused pixels to be lost as the PIO FIFO was overrun. Now one DMA channel tranfers each row of pixels to a buffer in SRAM, then a second DMA channel transfers each row to PSRAM.

New Driver Structure

Previous camera and display drivers used double inheritance for every permutation of device (eg. HM01B0 or OV5640) and interface (eg. SPI or PIO). Now the device drivers encapsulate the interface drivers, separating low-level hardware details from higher level device logic, and reducing the number of classes required for every permutation of devices and interfaces, improving long-term maintainability. Specifically:

• New VideoCapture class to more closely match standard OpenCV's VideoCapture class. The constructor takes an object that implements the VideoCaptureDriver abstract base class (which itself, inherits the new VideoDriver abstract base class).
• New VideoDisplay class to mirror the VideoCapture class. This is not a feature of standard OpenCV (cv.imshow() takes a string for the name of a window to display in, instead of an object that represents the window), but this makes the display driver structure closely mirror the camera driver structure, and is backwards compatible with the existing API difference in MicroPython-OpenCV. The constructor takes an object that implements the VideoDisplayDriver abstract base class (which itself, inherits the new VideoDriver abstract base class).
• New utils subpackage with modules:
  • colors - Provides standard color mode constants intended for use across all Red Vision drivers, and color-related utility functions.
  • memory - Provides utility functions related to device memory, such as checking whether an object/address is located in internal or external memory (the slower speed of external memory has proven to require different implementations for best performance).
  • pins - Provides utility functions that drivers can leverage to check and modify machine.Pin behavior on the fly.
• Example rv_init package now encourages importing the entire Red Vision package (eg. import red_vision as rv) rather than wildcard imports from submodules (eg. from red_vision.cameras import *) to improve readability and parity with OpenCV (eg. import cv2 as cv).

Supported Products

• SparkFun Red Vision Kit for RedBoard.
• SparkFun Red Vision Kit for XRP

Supported Platforms

• Raspberry Pi RP2 series MCUs (RP2040 and RP2350)

Dependencies

This release uses the following:

• MicroPython-OpenCV v1.0.1 (updated from v1.0.0)

Firmware

Pre-compiled firmware for Red Vision compatible MicroPython devices is available here (RED_VISION_MICROPYTHON_xyz).

v1.0.0

This is the first release of SparkFun's Red Vision, providing drivers and examples for Red Vision devices while enabling image processing directly in MicroPython with OpenCV!

Currently, only the SparkFun Red Vision Kit for XRP is supported. Drivers for additional products may be added in future releases.

This release uses the following:

• MicroPython-OpenCV v1.0.0

Pre-compiled firmware for Red Vision compatible MicroPython devices is available here (RED_VISION_MICROPYTHON_xyz).