[Python/macOS] malloc: double free crash when assigning numpy arrays to software_video_frame.pixels (Memory Ownership Bug)

[srikar0805]

Issue Description:
When using rs.software_device and injecting offline frames via rs.software_video_frame(), assigning a standard Python/Numpy byte buffer to the .pixels attribute results in a fatal malloc: double free crash on macOS.

This appears to be a memory-ownership conflict between the pyrealsense2 C++ backend and Python's Garbage Collector. When the frame processing concludes, the C++ engine attempts to free the memory buffer, but Python's GC also attempts to free the same numpy array, resulting in a dual-deletion crash.

In the C++ SDK, this is usually handled via a custom deleter callback, but the deleter attribute is not exposed in the pyrealsense2 Python wrapper, leaving no official way to safely manage memory lifecycle for synthetic frames.

Environment:

• OS: macOS (Apple Silicon / Intel)
• Language: Python 3.x
• Library: pyrealsense2
• Camera: No camera connected (Offline processing via software_device)

Steps to Reproduce

Here is a minimal reproducible example (MRE) that instantly triggers the crash:

import pyrealsense2 as rs
import numpy as np

# 1. Initialize Software Device
dev = rs.software_device()
depth_sensor = dev.add_sensor("Depth")
depth_stream = rs.video_stream()
depth_stream.type = rs.stream.depth
depth_stream.width, depth_stream.height = 1280, 720
depth_stream.bpp = 2
depth_stream.fmt = rs.format.z16
depth_profile = depth_sensor.add_video_stream(depth_stream)

# 2. Create Dummy Numpy Array (Simulating an offline raw depth map)
dummy_depth = np.zeros((720, 1280), dtype=np.uint16)

# 3. Inject Frame
d_frame = rs.software_video_frame()
d_frame.pixels = np.ascontiguousarray(dummy_depth).flatten() # <-- Issue originates here
d_frame.stride = 1280 * 2
d_frame.bpp = 2
d_frame.timestamp = 1.0
d_frame.domain = rs.timestamp_domain.system_time
d_frame.frame_number = 1
d_frame.profile = depth_profile.as_video_stream_profile()

depth_sensor.on_video_frame(d_frame)

# 4. Crash occurs shortly after or when script terminates

Expected Behavior

The SDK should accept the python buffer (e.g., Numpy array), process the frame (e.g., via rs.align), and safely release the memory without crashing the Python interpreter. Ideally, software_video_frame should copy the memory internally or the Python wrapper should expose the deleter property so users can tell C++ not to free the memory.

Actual Behavior

The script fatally aborts with a C++ memory exception:

Python(86335,0x1f329b080) malloc: double free for ptr 0x120b78000
Python(86335,0x1f329b080) malloc: * set a breakpoint in malloc_error_break to debug
zsh: abort      python script.py 

[MartyG-RealSense]

Hi @srikar0805 Thanks very much for your report. I will discuss it with my RealSense colleagues.

[srikar0805]

Hi @MartyG-RealSense

I have done some further testing to isolate the root cause, and I can confirm that this crash is entirely independent of the rs.align function. The malloc: double free crash occurs exactly during the frame injection phase inside rs.software_device.

I attempted to bypass rs.align completely and use the official rs.pointcloud() to generate a .ply file from the offline numpy arrays. The script crashes the exact moment the C++ engine processes the injected software_video_frame.

Here is the secondary Minimal Reproducible Example (MRE) proving the crash happens before any processing, alignment, or even frame-syncing takes place:

import pyrealsense2 as rs
import numpy as np

# 1. Initialize Software Device
dev = rs.software_device()
depth_sensor = dev.add_sensor("Depth")

depth_stream = rs.video_stream()
depth_stream.type = rs.stream.depth
depth_stream.uid = 1
depth_stream.width, depth_stream.height = 1280, 720
depth_stream.bpp = 2
depth_stream.fps = 30
depth_stream.fmt = rs.format.z16
# (Intrinsics omitted for brevity)
depth_profile = depth_sensor.add_video_stream(depth_stream)

sync = rs.syncer()
depth_sensor.open(depth_profile)
depth_sensor.start(sync)

# 2. Create Dummy Array
dummy_depth = np.zeros((720, 1280), dtype=np.uint16)
depth_array = np.ascontiguousarray(dummy_depth).flatten()

# 3. Inject Frame
d_frame = rs.software_video_frame()
d_frame.pixels = depth_array  
d_frame.stride = 1280 * 2
d_frame.bpp = 2
d_frame.timestamp = 1.0
d_frame.domain = rs.timestamp_domain.system_time
d_frame.frame_number = 1
d_frame.profile = depth_profile.as_video_stream_profile()

print("* Before injection")
depth_sensor.on_video_frame(d_frame) # <--- CRASH OCCURS EXACTLY HERE
print("* After injection")           # <--- This never prints

# The script never even reaches the syncer or the pointcloud generation:
frames = sync.wait_for_frames()
pc = rs.pointcloud()

Terminal Output:

* Before injection
Python(66822,0x1f329b080) malloc: double free for ptr 0x128960000
Python(66822,0x1f329b080) malloc: *** set a breakpoint in malloc_error_break to debug
zsh: abort      python script.py

Conclusion:
Because the software_video_frame python wrapper on macOS attempts to free() memory that is still managed by Python's Garbage Collector, all offline processing on macOS using pure Python arrays is completely blocked (whether using rs.align, rs.pointcloud, or custom filters). The only viable workaround remains bypassing the wrapper's memory management using ctypes.

I hope this helps narrow down the bug in the wrapper's bindings!.

[MartyG-RealSense]

Thanks so much @srikar0805 for the additional detailed findings!

[hsuys]

I'd wondering if the problem is specific to the OSx as the crash (with the MRE) doesn't occur on the Ubuntu nor Windows OS. When possible, please give them a try.

[srikar0805]

Hi, thank you so much for testing the MRE on Ubuntu and Windows!

That perfectly aligns with my debugging. Because the crash throws a malloc: double free specifically on macOS (Apple Silicon), it strongly points to a platform-specific bug in how the macOS Python bindings handle the memory lifecycle/garbage collection when software_video_frame.pixels is assigned.

I have a colleague currently running this exact MRE on their Windows machine to verify. I will update this thread as soon as they confirm it executes without the memory abort.

In the meantime, since I am locked to macOS for this project, I am currently blocked from using rs.software_device to load my offline frames. Is there any other recommended method within the pyrealsense2 API that I could use to bypass this issue and still utilize rs.align? I have my offline frames saved as rgb.png images, depth.raw arrays, and ir.raw arrays, and my ultimate goal is simply to align them using the official camera intrinsics/extrinsics.

[MartyG-RealSense]

Hi @srikar0805 Regarding an alternative to software_device, if you were able to record all the frames to a bag file (which is like a video recording of camera data) then you could load the offline recorded bag with your pyrealsense2 script and it would treat the data as though it were from a live camera and you would be able to perform rs.align on the data.

If your project requires using the mix of formats that you are currently using then you will need to use software_device to feed the frames into the SDK so they can be aligned.