Skip to content

kernel panic RGA_BLIT with yolov5 #133

@MattiaGrigoli

Description

@MattiaGrigoli

i'm trying to build an application that takes a video feed from a camera (with gstreamer) and makes and inference (object detection) with yolov5 using the NPU, i need 4 istances of this application (for 4 camera in total)
the part of gstreamer works flawlessly because with 4 terminal running the gstreamer pipeline (with hardware acceleration) it doesn't run into a kernel panic.
the problem is when i also run the inference:

  • with one inference (one camera) it runs without any problem
  • with two inferences (or more) it goes straight to a kernel panic

the only solution i found on it is to limit the RAM of the board to 4GB, but for me is too little for what i have to do (in the future there will be other things beside the inference)
i'm using rknpu2 to use the NPU.
i know that the issue is generated by the RGA (this explains the weird reason of the 4GB of memory) but i don't know how to fix it. Maybe some kind of dma32 allocation but i have no idea for how to implement it.
is there a fix for my issue?
here are some info:

  • kernel panic
    image

  • error of program
    image

  • code that handles the inference:

`
#pragma once
#include
#include

#include "yolov5.h"
#include "image_utils.h"
#include "file_utils.h"
#include "image_drawing.h"

#include "dma_alloc.hpp"

class YoloModel {

public:
bool loaded = false;
rknn_app_context_t rknn_app_ctx;

YoloModel(std::string model_path) {
int ret;
memset(&rknn_app_ctx, 0, sizeof(rknn_app_context_t));

init_post_process();
ret = init_yolov5_model(model_path.c_str(), &rknn_app_ctx);
if (ret != 0)
{
    printf("init_yolov5_model fail! ret=%d model_path=%s\n", ret, model_path);
    return;
}
loaded = true;

}

bool infer(cv::Mat cvImage) {
if(!loaded)
return false;

  int ret;

  image_buffer_t src_image;
  memset(&src_image, 0, sizeof(image_buffer_t));
  //ret = read_image("../yolov5/model/bus.jpg", &src_image);
  

  /*ret = dma_buf_alloc(DMA_HEAP_DMA32_PATCH, src_image.size, &rknn_app_ctx.img_dma_buf.dma_buf_fd, 
                   (void **) & (rknn_app_ctx.img_dma_buf.dma_buf_virt_addr));
  memcpy(rknn_app_ctx.img_dma_buf.dma_buf_virt_addr, src_image.virt_addr, src_image.size);
  dma_sync_cpu_to_device(rknn_app_ctx.img_dma_buf.dma_buf_fd);
  free(src_image.virt_addr);
  src_image.virt_addr = (unsigned char *)rknn_app_ctx.img_dma_buf.dma_buf_virt_addr;
  src_image.fd = rknn_app_ctx.img_dma_buf.dma_buf_fd;
  rknn_app_ctx.img_dma_buf.size = src_image.size;*/

  src_image.width = cvImage.cols;
  src_image.height = cvImage.rows;
  src_image.format = IMAGE_FORMAT_RGB888;
  src_image.virt_addr = cvImage.data;

  object_detect_result_list od_results;
  ret = inference_yolov5_model(&rknn_app_ctx, &src_image, &od_results);
  if (ret != 0)
  {
      printf("init_yolov5_model fail! ret=%d\n", ret);
      return false;
  }

  // print
  char text[256];
  for (int i = 0; i < od_results.count; i++)
  {
      object_detect_result *det_result = &(od_results.results[i]);
      printf("\t%s @ (%d %d %d %d) %.3f\n", coco_cls_to_name(det_result->cls_id),
            det_result->box.left, det_result->box.top,
            det_result->box.right, det_result->box.bottom,
            det_result->prop);
      int x1 = det_result->box.left;
      int y1 = det_result->box.top;
      int x2 = det_result->box.right;
      int y2 = det_result->box.bottom;

      draw_rectangle(&src_image, x1, y1, x2 - x1, y2 - y1, COLOR_BLUE, 3);
      // sprintf(text, "%s %.1f%%", coco_cls_to_name(det_result->cls_id), det_result->prop * 100);
      // draw_text(&src_image, text, x1, y1 - 20, COLOR_RED, 10);
  }
  /*dma_buf_free(rknn_app_ctx.img_dma_buf.size, &rknn_app_ctx.img_dma_buf.dma_buf_fd, 
            rknn_app_ctx.img_dma_buf.dma_buf_virt_addr);*/
  return true;

}

};
`

  • board = orange pi 5 plus
  • version = ubuntu jammy (joshua)
  • kernel = 5.10 (joshua)

as you can see i've tried to use the dma allocation from the example but it gave me this error:
image
i'm also available to modify the source code of the kernel if needed

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions