drivers: video: video_stm32_dcmi: Detect and recover from: HAL_DCMI_ErrorCallback

[KurtE]

Summary

With several of the different sketches that do things like display a camera image to a display of some form. Like Arduino Gigia Display shield, or some TFT display like an ST7796 or ILI9341, As I mentioned in the issue: #92352 I will receive several messages like:

[00:00:12.307,000] <dbg> video_stm32_dcmi: HAL_DCMI_FrameEventCallback: Failed to get buffer from fifo

Which in that issue I would hope to reuse the oldest frame...

But then I often times then get the message:
[00:00:14.851,000] <wrn> video_stm32_dcmi: HAL_DCMI_ErrorCallback
And the camera completely stops at that point

Describe the solution you'd like

I wish that the callback could correct the issue and recover.

Alternatives

Or wish there was a way for the client code to detect this and call something to recover.

Detecting this might simply be, that I would expect an image to be ready by my timeout... If not then call the recovery...

Additional Context

Most of my testing is done using the ArduinoCore-zephyr setup. Currently running the latest github code.
The west update from their points us to a zephyr snaphshot from about a month ago.

[josuah]

[00:00:12.307,000] <dbg> video_stm32_dcmi: HAL_DCMI_FrameEventCallback: Failed to get buffer from fifo

This seems like something that could be avoided by continuously enqueueing frames to make sure the driver never starves.

[00:00:14.851,000] <wrn> video_stm32_dcmi: HAL_DCMI_ErrorCallback

This looks like a bug on the DCMI HAL or the way it is used in the Zephyr driver?

If it turns out that it is an expected condition (i.e. DCMI was starved for too long), then maybe this needs some preventive measure to be implemented in video_stm32_dcmi.c so that the video API can be used without having to i.e. restart a driver that fails.

Also linking the Arduino-side issue where HAL_DCMI_ErrorCallback is also triggered:

• Portenta H7: Trying to add the Camera/GC2145 support arduino/ArduinoCore-zephyr#78 (comment)

Thank you for all the communication!

[KurtE]

Thanks @josuah,

I totally agree that it would be better to avoid having to restart the stream!

Current setup:

Current sketch:

#include <ST77XX_zephyr.h>
#include "camera.h"

Camera cam;
#include <SPI.h>
#include <ST77XX_zephyr.h>
#include <ST77XX_zephyr_font_Arial.h>

#ifdef ARDUINO_PORTENTA_H7 
#define TFT_DC   4 
#define TFT_CS   2  
#define TFT_RST  3
#define TFT_SPI  SPI

#else
#define TFT_CS   10  // CS & DC can use pins 2, 6, 9, 10, 15, 20, 21, 22, 23
#define TFT_DC    8  //  but certain pairs must NOT be used: 2+10, 6+9, 20+23, 21+22
#define TFT_RST   9  // RST can use any pin
#define TFT_SPI  SPI1

#endif

#define CAMERA_WIDTH 480 //320
#define CAMERA_HEIGHT 320 // 240
// Use one or the other
//ST7735_zephyr tft = ST7735_zephyr(&SPI, TFT_CS, TFT_DC, TFT_RST);

// For 1.54" TFT with ST7789
//ST7789_zephyr tft = ST7789_zephyr(&SPI, TFT_CS, TFT_DC, TFT_RST);

// For 3.5" or 4.0" TFT with ST7796
ST7796_zephyr tft = ST7796_zephyr(&TFT_SPI, TFT_CS, TFT_DC, TFT_RST);

void fatal_error(const char *msg) {
  Serial.println(msg);
  pinMode(LED_BUILTIN, OUTPUT);
  while (1) {
    digitalWrite(LED_BUILTIN, HIGH);
    delay(100);
    digitalWrite(LED_BUILTIN, LOW);
    delay(100);
  }
}



void setup() {
  pinMode(LED_BUILTIN, OUTPUT);
  Serial.begin(115200);
  while (!Serial && millis() < 5000) {}

  Serial.println("\n*** start display camera image on ST77XX ***");
  // Use this initializer if you're using a 1.8" TFT 128x160 displays
  //tft.initR(INITR_BLACKTAB);

  // Or use this initializer (uncomment) if you're using a 1.44" TFT (128x128)
  //tft.initR(INITR_144GREENTAB);

  // Or use this initializer (uncomment) if you're using a .96" TFT(160x80)
  //tft.initR(INITR_MINI160x80);

  // Or use this initializer (uncomment) for Some 1.44" displays use different memory offsets
  // Try it if yours is not working properly
  // May need to tweek the offsets
  //tft.setRowColStart(32,0);

  // Or use this initializer (uncomment) if you're using a 1.54" 240x240 TFT
  //tft.init(240, 240);   // initialize a ST7789 chip, 240x240 pixels

  // OR use this initializer (uncomment) if using a 2.0" 320x240 TFT:
  //tft.init(240, 320);           // Init ST7789 320x240
  tft.init(320, 480);

  tft.setRotation(1);
  tft.fillScreen(ST77XX_BLACK);
  delay(500);
  tft.fillScreen(ST77XX_RED);
  delay(500);
  tft.fillScreen(ST77XX_GREEN);
  delay(500);
  tft.fillScreen(ST77XX_BLUE);
  delay(500);
  tft.fillScreen(ST77XX_BLACK);
  delay(500);

  Serial.println("call cam.begin");
  if (!cam.begin(CAMERA_WIDTH, CAMERA_HEIGHT, CAMERA_RGB565, true)) {
    fatal_error("Camera begin failed");
  }
  Serial.println("Camera started");
  cam.setVerticalFlip(false);
  cam.setHorizontalMirror(false);
}

volatile bool write_rect_complete = false;
void write_rect_complete_cb(int result) {
  UNUSED(result);
  write_rect_complete = true;

}

bool use_writeRectCB = true;

void loop() {
  // put your main code here, to run repeatedly:
  FrameBuffer fb;
  //Serial.print("Call grabFrame");
  bool frame_received = cam.grabFrame(fb);
  //Serial.println(frame_received? " true" : " false");
  if (frame_received) {
    digitalWrite(LED_BUILTIN, !digitalRead(LED_BUILTIN));
    uint32_t start_time = micros();
    if (use_writeRectCB) {
      write_rect_complete = false;
      tft.writeRectCB(0, 0, CAMERA_WIDTH, CAMERA_HEIGHT, (const uint16_t*)fb.getBuffer(), &write_rect_complete_cb);
      while (!write_rect_complete) delay(1);
    } else {
      tft.writeRect(0, 0, CAMERA_WIDTH, CAMERA_HEIGHT, (const uint16_t*)fb.getBuffer());
    }
    Serial.println(micros()-start_time);
    //tft.writeSubImageRectBytesReversed(0, 0, 320, 240, 0, 0, 320, 240, (const uint16_t*)fb.getBuffer());
    //tft.writeSubImageRect(0, 0, 320, 240, 0, 0, 320, 240, (const uint16_t*)fb.getBuffer());
    cam.releaseFrame(fb);
  }
  if (Serial.available()) {
    while (Serial.read() != -1) {}
    use_writeRectCB = !use_writeRectCB;
  }
}

I have it setup to try to read a 320x480 image using the code in my PR: #91975
It is also using the code I mentioned in issue: #92352

I am using my own ST7796 display library, which can be found at: https://github.com/KurtE/Arduino_GIGA-stuff/tree/main/libraries/ST77XX_zephyr

And there are two forms for writeRect, (write the picture out to the display), one that loops
writing each byte out... SPI.transfer type call, and one that calls:
spi_transceive_cb(_spi_dev, &_config16, &tx_buf_set, nullptr, &spi_callback, (void*)this);
I default to the later, and in the last run, it displayed I believe 19 images before it died:

*** start display camera image on ST77XX ***
call cam.begin
Camera started
453284
446425
...
538126
414601

Timings there are in Micros. With the other approach it takes about 2 seconds.

Error output:

*** Booting Zephyr OS build v3.7.0-17049-g3297c64bfdec ***

$$$ main.cpp called
InitVariant called
calling start_static_threads
static threads start_static_threads
[00:00:06.811,000] <dbg> video_gc2145: gc2145_set_resolution: set_resolution(480 320)
[00:00:06.821,000] <dbg> video_gc2145: gc2145_set_resolution: set resolution(480 320): ratio: 3 3
[00:00:14.815,000] <err> dma_stm32_v1: FiFo error.
[00:00:14.822,000] <inf> dma_stm32_v1: tc: 0, ht: 1, te: 0, dme: 0, fe: 1
[00:00:14.832,000] <err> dma_stm32: Unexpected irq happened.
[00:00:14.840,000] <err> video_stm32_dcmi: DMA callback error with channel 0.
[00:00:14.850,000] <err> dma_stm32_v1: FiFo error.
[00:00:14.858,000] <inf> dma_stm32_v1: tc: 1, ht: 1, te: 0, dme: 0, fe: 1
[00:00:14.867,000] <err> dma_stm32: Unexpected irq happened.
[00:00:14.876,000] <err> video_stm32_dcmi: DMA callback error with channel 0.
[00:00:14.886,000] <err> dma_stm32_v1: FiFo error.
[00:00:14.893,000] <inf> dma_stm32_v1: tc: 1, ht: 1, te: 0, dme: 0, fe: 1
[00:00:14.903,000] <err> dma_stm32: Unexpected irq happened.
[00:00:14.911,000] <err> video_stm32_dcmi: DMA callback error with channel 0.
[00:00:15.229,000] <wrn> video_stm32_dcmi: HAL_DCMI_ErrorCallback

Note: this is probably a torture test for SDRAM. That is there are two DMA type operations going on,
one reading from the camera to SDRAM, and the other writing from SDRAM to display, plus each time
the camera reads a frame in, it then copies the 307200 bytes from on SDRAM buffer to another.

Will keep playing and trying stuff out.

[avolmat-st]

[00:00:12.307,000] <dbg> video_stm32_dcmi: HAL_DCMI_FrameEventCallback: Failed to get buffer from fifo

This seems like something that could be avoided by continuously enqueueing frames to make sure the driver never starves.

I doubt this is due to queueing being done too late since internally the DCMI is always using the 2nd buffer which is kept intern to the driver (the driver always has an internal buffer into which the DCMI is always writing into. What is exposed afterward to the application via dequeued buffer a copy of it). So, even if the application doesn't queue back a buffer, the DCMI will anyway write into the internal buffer held by the dcmi driver.

[00:00:14.851,000] <wrn> video_stm32_dcmi: HAL_DCMI_ErrorCallback

This looks like a bug on the DCMI HAL or the way it is used in the Zephyr driver?

If it turns out that it is an expected condition (i.e. DCMI was starved for too long), then maybe this needs some preventive measure to be implemented in video_stm32_dcmi.c so that the video API can be used without having to i.e. restart a driver that fails.

Also linking the Arduino-side issue where HAL_DCMI_ErrorCallback is also triggered:

• Portenta H7: Trying to add the Camera/GC2145 support arduino/ArduinoCore-zephyr#78 (comment)

Thank you for all the communication!

As pointed in the further comment, this sounds more like a DMA issue or maybe an overrun at DCMI level. For example there could also be cases of overrun in case of the sensor pixel clock is too high compared to the processing clock of the DCMI. My guess is that it is probably not the case here, but just in case do you know the pixel clock frequency of your GC2145 ? It cannot be higher than DCMI processing clock / 2.5 (said differently, DCMI processing clock must be 2.5 higher than the pixel clock of the sensor). For information I had such case of the STM32L4 for which the default OV5640 pixel clock was too high, leading to overruns.

[avolmat-st]

Note: this is probably a torture test for SDRAM. That is there are two DMA type operations going on, one reading from the camera to SDRAM, and the other writing from SDRAM to display, plus each time the camera reads a frame in, it then copies the 307200 bytes from on SDRAM buffer to another.

Will keep playing and trying stuff out.

To check in this area, could you first try to only capture frames without display, with for example sleep between dequeue and enqueue to "simulate" the delay introduced by display and see if this still affect the capture ?