Streaming of RGB and Depth Image From the Headset

[boardd]

Hi,

Thank you for open sourcing such a great project. I am thinking of using your pipeline to do some human data collection (human wears headset and does stuff).

We are wondering if you've experimented with the depth or passthrough camera APIs such that you can stream those images through ROS as well. If not, any tips on where I can get started.

I'm pretty new to this Unity/ XR development type stuff.

Thanks in advance for the help!

[maxwildersmith]

Hi there, great use case for this project!

Yes we have looked at it, unfortunately the passthrough API in past was difficult to get access to directly (privacy issues), but that has been recently updated and is something we can access now. If this is something you would like to add to our project I would be happy to help with any issues you encounter along the way, for getting started I would recomend checking out our old VR Streamer component this would access the main camera view as a render texture I believe and send it over ROS.

For accessing the camera data, I would recomend starting with the passthrough samples and see if you can get one of these to work in a new component folder of our dev/unity6 branch.

Hope this helps!

[boardd]

wow thanks for the quick reply. Yes I'll have a go at it and let you know if I get it working!