Skip to content

Commit 1853975

Browse files
Erol444Erol444
authored
Merge pull request #853 from luxonis/tof_docs
Added ToF docs
2 parents 4e68c1b + 11162d3 commit 1853975

File tree

5 files changed

+243
-0
lines changed

5 files changed

+243
-0
lines changed

‎docs/source/components/messages/tof_config.rst

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
ToFConfig
2+
=========
3+
4+
This message is used to configure the :ref:`ToF` node.
5+
6+
Examples of functionality
7+
#########################
8+
9+
- :ref:`ToF depth`
10+
11+
Reference
12+
#########
13+
14+
.. tabs::
15+
16+
.. tab:: Python
17+
18+
.. autoclass:: depthai.ToFConfig
19+
:members:
20+
:inherited-members:
21+
:noindex:
22+
23+
.. tab:: C++
24+
25+
.. doxygenclass:: dai::ToFConfig
26+
:project: depthai-core
27+
:members:
28+
:private-members:
29+
:undoc-members:
30+
31+
.. include:: ../../includes/footer-short.rst

‎docs/source/components/nodes/tof.rst

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
ToF
2+
===
3+
4+
**ToF node** is used for converting the raw data from the ToF sensor into a depth map. Currently, these 2 products contain a ToF sensor:
5+
6+
- `OAK-D SR PoE <https://docs.luxonis.com/projects/hardware/en/latest/pages/OAK-D-SR-POE/>`__ - integrated 33D ToF sensor, together with a stereo camera pair
7+
- `OAK-FFC ToF 33D <https://docs.luxonis.com/projects/hardware/en/latest/pages/DM0256/>`__ - standalone FFC module with a 33D ToF sensor
8+
9+
ToF's ``depth`` output can be used instead of :ref:`StereoDepth`'s - so you can link ToF.depth to :ref:`MobileNetSpatialDetectionNetwork`/:ref:`YoloSpatialDetectionNetwork` or
10+
:ref:`SpatialLocationCalculator` directly.
11+
12+
How to place it
13+
###############
14+
15+
.. tabs::
16+
17+
.. code-tab:: py
18+
19+
pipeline = dai.Pipeline()
20+
warp = pipeline.create(dai.node.ToF)
21+
22+
.. code-tab:: c++
23+
24+
dai::Pipeline pipeline;
25+
auto warp = pipeline.create<dai::node::ToF>();
26+
27+
Inputs and Outputs
28+
##################
29+
30+
.. code-block::
31+
32+
┌───────────┐ depth
33+
inputConfig | ├────────►
34+
───────────►│ | amplitude
35+
input | ToF ├────────►
36+
───────────►│ │ error
37+
│ ├────────►
38+
└───────────┘
39+
40+
**Message types**
41+
42+
- ``inputConfig`` - :ref:`ToFConfig`
43+
- ``input`` - :ref:`ImgFrame`
44+
- ``depth`` - :ref:`ImgFrame`
45+
- ``amplitude`` - :ref:`ImgFrame`
46+
- ``error`` - :ref:`ImgFrame`
47+
48+
Usage
49+
#####
50+
51+
.. tabs::
52+
53+
.. code-tab:: py
54+
55+
pipeline = dai.Pipeline()
56+
57+
tof_cam = pipeline.create(dai.node.Camera)
58+
# We assume the ToF camera sensor is on port CAM_A
59+
tof_cam.setBoardSocket(dai.CameraBoardSocket.CAM_A)
60+
61+
tof = pipeline.create(dai.node.ToF)
62+
# ToF node converts raw sensor frames into depth
63+
tof_cam.raw.link(tof.input)
64+
65+
# Send ToF depth output to the host, or perhaps to SLC / Spatial Detection Network
66+
tof.depth.link(xout.input)
67+
68+
.. code-tab:: c++
69+
70+
dai::Pipeline pipeline;
71+
72+
auto tofCam = pipeline.create<dai::node::MonoCamera>();
73+
// We assume the ToF camera sensor is on port CAM_A
74+
tofCam->setBoardSocket(dai::CameraBoardSocket::AUTO);
75+
76+
auto tof = pipeline.create<dai::node::ToF>();
77+
// ToF node converts raw sensor frames into depth
78+
tofCam->raw.link(tof->input);
79+
80+
auto xout = pipeline.create<dai::node::XLinkOut>();
81+
xout->setStreamName("depth");
82+
// Send ToF depth output to the host
83+
tof->depth.link(xout->input);
84+
85+
Examples of functionality
86+
#########################
87+
88+
- :ref:`ToF depth`
89+
90+
Reference
91+
#########
92+
93+
.. tabs::
94+
95+
.. tab:: Python
96+
97+
.. autoclass:: depthai.node.ToF
98+
:members:
99+
:inherited-members:
100+
:noindex:
101+
102+
.. tab:: C++
103+
104+
.. doxygenclass:: dai::node::ToF
105+
:project: depthai-core
106+
:members:
107+
:private-members:
108+
:undoc-members:
109+
110+
.. include:: ../../includes/footer-short.rst

‎docs/source/samples/ToF/tof_depth.rst

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
ToF depth
2+
=========
3+
4+
This is a sample code that showcases how to use the ToF sensor. The :ref:`ToF node <ToF>` converts raw data from the ToF sensor into a depth map.
5+
6+
Demo
7+
####
8+
9+
This demo was recorded using the `OAK-D SR PoE <https://docs.luxonis.com/projects/hardware/en/latest/pages/OAK-D-SR-POE/>`__, that's why we selected CAM_A port
10+
on the ToF sensor.
11+
12+
.. raw:: html
13+
14+
<div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; height: auto;">
15+
<iframe src="https://www.youtube.com/embed/D2MnnyxdsMA" frameborder="0" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe>
16+
</div>
17+
18+
Setup
19+
#####
20+
21+
.. include:: /includes/install_from_pypi.rst
22+
23+
Source code
24+
###########
25+
26+
.. tabs::
27+
28+
.. tab:: Python
29+
30+
Also `available on GitHub <https://github.com/luxonis/depthai-python/blob/main/examples/ToF/tof_depth.py>`__
31+
32+
.. literalinclude:: ../../../../examples/ToF/tof_depth.py
33+
:language: python
34+
:linenos:
35+
36+
.. tab:: C++
37+
38+
..
39+
Also `available on GitHub <https://github.com/luxonis/depthai-core/blob/main/examples/ToF/tof_depth.cpp>`__
40+
41+
.. literalinclude:: ../../../../depthai-core/examples/ToF/tof_depth.cpp
42+
:language: cpp
43+
:linenos:
44+
45+
.. include:: /includes/footer-short.rst

‎docs/source/tutorials/code_samples.rst

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ Code Samples
2323
../samples/SpatialDetection/*
2424
../samples/StereoDepth/*
2525
../samples/SystemLogger/*
26+
../samples/ToF/*
2627
../samples/VideoEncoder/*
2728
../samples/Warp/*
2829
../samples/Yolo/*
@@ -152,6 +153,10 @@ are presented with code.
152153

153154
- :ref:`System information` - Displays device system information (memory/cpu usage, temperature)
154155

156+
.. rubric:: ToF
157+
158+
- :ref:`ToF depth` - Displays colorized ToF depth frames
159+
155160
.. rubric:: VideoEncoder
156161

157162
- :ref:`Disparity encoding` - Encodes stereo disparity into :code:`.mjpeg`

‎examples/ToF/tof_depth.py

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
#!/usr/bin/env python3
2+
3+
import cv2
4+
import depthai as dai
5+
import numpy as np
6+
7+
pipeline = dai.Pipeline()
8+
9+
cam_a = pipeline.create(dai.node.Camera)
10+
# We assume the ToF camera sensor is on port CAM_A
11+
cam_a.setBoardSocket(dai.CameraBoardSocket.CAM_A)
12+
13+
tof = pipeline.create(dai.node.ToF)
14+
15+
# Configure the ToF node
16+
tofConfig = tof.initialConfig.get()
17+
# tofConfig.depthParams.freqModUsed = dai.RawToFConfig.DepthParams.TypeFMod.MIN
18+
tofConfig.depthParams.freqModUsed = dai.RawToFConfig.DepthParams.TypeFMod.MAX
19+
tofConfig.depthParams.avgPhaseShuffle = False
20+
tofConfig.depthParams.minimumAmplitude = 3.0
21+
tof.initialConfig.set(tofConfig)
22+
# Link the ToF sensor to the ToF node
23+
cam_a.raw.link(tof.input)
24+
25+
xout = pipeline.create(dai.node.XLinkOut)
26+
xout.setStreamName("depth")
27+
tof.depth.link(xout.input)
28+
29+
# Connect to device and start pipeline
30+
with dai.Device(pipeline) as device:
31+
print('Connected cameras:', device.getConnectedCameraFeatures())
32+
q = device.getOutputQueue(name="depth")
33+
34+
while True:
35+
imgFrame = q.get() # blocking call, will wait until a new data has arrived
36+
depth_map = imgFrame.getFrame()
37+
38+
# Colorize the depth frame to jet colormap
39+
depth_downscaled = depth_map[::4]
40+
non_zero_depth = depth_downscaled[depth_downscaled != 0] # Remove invalid depth values
41+
if len(non_zero_depth) == 0:
42+
min_depth, max_depth = 0, 0
43+
else:
44+
min_depth = np.percentile(non_zero_depth, 3)
45+
max_depth = np.percentile(non_zero_depth, 97)
46+
depth_colorized = np.interp(depth_map, (min_depth, max_depth), (0, 255)).astype(np.uint8)
47+
depth_colorized = cv2.applyColorMap(depth_colorized, cv2.COLORMAP_JET)
48+
49+
cv2.imshow("Colorized depth", depth_colorized)
50+
51+
if cv2.waitKey(1) == ord('q'):
52+
break

0 commit comments

Comments
 (0)