Coredump seen in Single Input Multi Interence

[p-mantena]

Hi,

We are facing a memory dump issue with the tiovxmultiscaler plugin in Single Input Multi Inference, When running the testcase in loop.
It crashes after around 43rd iteration when we are using 4 inferences and after ~89th iteration when using 2 inferences.
I am attaching the script that reproduces the issue and the output log file and link for input file.
The same thing works with videoscale.

simi_multiscaler.txt
output.log
https://git.ti.com/cgit/edgeai/edge_ai_apps/

Error log :

MEM: ERROR: Alloc failed with status = 12 !!!
693.842050 s: VX_ZONE_ERROR:[tivxMemBufferAlloc:80] Shared mem ptr allocation failed
693.842073 s: VX_ZONE_ERROR:[ownAllocImageBuffer:309] could not allocate memory
693.842081 s: VX_ZONE_ERROR:[ownCopyAndMapCheckParams:672] image allocation failed
693.842089 s: VX_ZONE_ERROR:[vxUnmapImagePatch:2059] map ID is greater than the maximum image maps
MEM: ERROR: Alloc failed with status = 12 !!!
693.842117 s: VX_ZONE_ERROR:[tivxMemBufferAlloc:80] Shared mem ptr allocation failed
693.842124 s: VX_ZONE_ERROR:[ownAllocImageBuffer:309] could not allocate memory
693.842131 s: VX_ZONE_ERROR:[ownCopyAndMapCheckParams:672] image allocation failed
693.842139 s: VX_ZONE_ERROR:[vxUnmapImagePatch:2059] map ID is greater than the maximum image maps
MEM: ERROR: Alloc failed with status = 12 !!!
693.842194 s: VX_ZONE_ERROR:[tivxMemBufferAlloc:80] Shared mem ptr allocation failed
MEM: ERROR: Alloc failed with status = 12 !!!
693.842211 s: VX_ZONE_ERROR:[tivxMemBufferAlloc:80] Shared mem ptr allocation failed
693.842218 s: VX_ZONE_ERROR:[ownAllocImageBuffer:309] could not allocate memory
693.842224 s: VX_ZONE_ERROR:[ownCopyAndMapCheckParams:672] image allocation failed
693.842230 s: VX_ZONE_ERROR:[vxUnmapImagePatch:2059] map ID is greater than the maximum image maps
MEM: ERROR: Alloc failed with status = 12 !!!
693.842250 s: VX_ZONE_ERROR:[tivxMemBufferAlloc:80] Shared mem ptr allocation failed
693.842256 s: VX_ZONE_ERROR:[ownAllocImageBuffer:309] could not allocate memory
693.842261 s: VX_ZONE_ERROR:[ownCopyAndMapCheckParams:672] image allocation failed
693.842267 s: VX_ZONE_ERROR:[vxUnmapImagePatch:2059] map ID is greater than the maximum image maps
MEM: ERROR: Alloc failed with status = 12 !!!
693.842292 s: VX_ZONE_ERROR:[tivxMemBufferAlloc:80] Shared mem ptr allocation failed
Segmentation fault (core dumped)

Regards,
Prasanth Babu Mantena

[m-herrera]

Hi @p-mantena

After debugging the issue. It seems that the error is related to how python is freeing the pipeline.
Python does not free the pipeline or its elements until the application exits, so tivx memory is not freed in between iterations. The issue disappears If the pipeline is nulled for each iteration, forcing it to be freed.

#!/usr/bin/python3
import gi
gi.require_version('Gst', '1.0')
gi.require_version('GstApp', '1.0')
from gi.repository import Gst, GstApp, GLib
from time import sleep
Gst.init(None)
gst_str = "filesrc location=/opt/edge_ai_apps/data/videos/video_0000_h264.mp4 ! qtdemux ! h264parse ! avdec_h264 ! video/x-raw, format=I420 ! videoconvert ! tee name=tee_split0 tee_split0. ! queue ! tiovxmultiscaler ! kmssink tee_split0. ! queue ! tiovxmultiscaler ! fakesink tee_split0. ! queue ! tiovxmultiscaler ! fakesink tee_split0. ! queue ! tiovxmultiscaler ! fakesink"
i=100
while i>1:
	gst_pipe = Gst.parse_launch(gst_str)
	gst_pipe.set_state(Gst.State.PLAYING)
	sleep(10)
	gst_pipe.set_state(Gst.State.NULL)
	print(100-i)
	i=i-1
+	gst_pipe = None

It is worth mentioning that this is not specifically related to the edgeai-gst-plugins, it also happens with standard gstreamer elements.

Please test the suggested approach and let us know if it solves the issue on your end.