Merge pull request #486 from luxonis/depth_filters

Depth filters

Author: Erol444

docs/source/_static/images/components/depth_diagram.jpeg

@@ -73,7 +73,7 @@ Inputs and Outputs
 Internal block diagram of StereoDepth node
 ##########################################
 
-.. image:: /_static/images/components/depth_diagram.jpeg
+.. image:: /_static/images/components/depth_diagram.png
    :target: https://whimsical.com/stereo-node-EKcfcXGjGpNL6cwRPV6NPv
 
 On the diagram, red rectangle are firmware settings that are configurable via the API. Gray rectangles are settings that that are not yet
@@ -126,9 +126,15 @@ Currently configurable blocks
 
         For comparison of normal disparity vs. subpixel disparity images, click `here <https://github.com/luxonis/depthai/issues/184>`__.
 
-  .. tab:: Mesh file / Homography matrix
+  .. tab:: Depth Filters
 
-    Mesh files are generated using the camera intrinsics, distortion coeffs, and rectification rotations.
+    **Depth Filtering** / **Depth Post-Processing** is performed at the end of the depth pipeline. It helps with noise reduction and overall depth quality.
+
+    .. include::  ../../includes/depth-filters.rst
+
+  .. tab:: Mesh files
+
+    Mesh files (homography matrix) are generated using the camera intrinsics, distortion coeffs, and rectification rotations.
     These files helps in overcoming the distortions in the camera increasing the accuracy and also help in when `wide FOV <https://docs.luxonis.com/projects/hardware/en/latest/pages/arducam.html#arducam-compatible-cameras>`__ lens are used.
 
     .. note::
@@ -148,7 +154,7 @@ Currently configurable blocks
 
   .. tab:: Confidence Threshold
 
-    - **Confidence threshold**: Stereo depth algorithm searches for the matching feature from right camera point to the left image (along the 96 disparity levels). During this process it computes the cost for each disparity level and choses the minimal cost between two disparities and uses it to compute the confidence at each pixel. Stereo node will output disparity/depth pixels only where depth confidence is below the **confidence threshold** (lower the confidence value means better depth accuracy). Note: This threshold only applies to Normal stereo mode as of now.
+    - **Confidence threshold**: Stereo depth algorithm searches for the matching feature from right camera point to the left image (along the 96 disparity levels). During this process it computes the cost for each disparity level and chooses the minimal cost between two disparities and uses it to compute the confidence at each pixel. Stereo node will output disparity/depth pixels only where depth confidence is below the **confidence threshold** (lower the confidence value means better depth accuracy).
     - **LR check threshold**: Disparity is considered for the output when the difference between LR and RL disparities is smaller than the LR check threshold.
 
     .. doxygenfunction:: dai::StereoDepthConfig::setConfidenceThreshold
@@ -159,8 +165,8 @@ Currently configurable blocks
         :project: depthai-core
         :no-link:
 
-Current limitations
-###################
+Limitations
+###########
 
 - Median filtering is disabled when subpixel mode is set to 4 or 5 bits.
 

docs/source/_static/images/components/depth_diagram.png

@@ -0,0 +1,65 @@
+.. tabs::
+
+    .. tab:: Median
+
+        This is a non-edge preserving `Median filter <https://en.wikipedia.org/wiki/Median_filter>`__, which can be used
+        to reduce noise and smoothen the depth map. Median filter is implemented in hardware, so it's the fastest filter.
+
+        .. doxygenenum:: dai::MedianFilter
+            :project: depthai-core
+            :no-link:
+
+    .. tab:: Speckle
+
+        **Speckle Filter** is used to reduce the speckle noise. Speckle noise is a region with huge
+        variance between neighboring disparity/depth pixels, and speckle filter tries to filter this region.
+
+        .. doxygenstruct:: dai::RawStereoDepthConfig::PostProcessing::SpeckleFilter
+            :project: depthai-core
+            :no-link:
+            :members:
+
+    .. tab:: Temporal
+
+        **Temporal Filter** is intended to improve the depth data persistency by manipulating per-pixel
+        values based on previous frames. The filter performs a single pass on the data, adjusting the depth
+        values while also updating the tracking history. In cases where the pixel data is missing or invalid,
+        the filter uses a user-defined persistency mode to decide whether the missing value should be
+        rectified with stored data. Note that due to its reliance on historic data the filter may introduce
+        visible blurring/smearing artifacts, and therefore is best-suited for static scenes.
+
+        .. doxygenstruct:: dai::RawStereoDepthConfig::PostProcessing::TemporalFilter
+            :project: depthai-core
+            :no-link:
+            :members:
+
+    .. tab:: Spatial
+
+        **Spatial Edge-Preserving Filter** will fill invalid depth pixels with valid neighboring depth pixels.
+        It performs a series of 1D horizontal and vertical passes or iterations, to enhance the smoothness of
+        the reconstructed data. It is based on `this research paper <https://www.inf.ufrgs.br/~eslgastal/DomainTransform/>`__.
+
+        .. doxygenstruct:: dai::RawStereoDepthConfig::PostProcessing::SpatialFilter
+            :project: depthai-core
+            :no-link:
+            :members:
+
+    .. tab:: Threshold
+
+        **Threshold Filter** filters out all disparity/depth pixels outside the configured min/max
+        threshold values.
+
+        .. autoclass:: depthai.RawStereoDepthConfig.PostProcessing.ThresholdFilter
+            :members:
+            :inherited-members:
+            :noindex:
+
+    .. tab:: Decimation
+
+        **Decimation Filter** will sub-samples the depth map, which means it reduces the depth scene complexity and allows
+        other filters to run faster. Setting :code:`decimationFactor` to 2 will downscale 1280x800 depth map to 640x400.
+
+        .. doxygenstruct:: dai::RawStereoDepthConfig::PostProcessing::DecimationFilter
+            :members:
+            :project: depthai-core
+            :no-link:

docs/source/_static/images/examples/depth_comparison.png

@@ -0,0 +1,49 @@
+Depth Post-Processing
+=====================
+
+This example shows how you can run depth post-processing filters on the device itself to reduce noise,
+smooth the depth map and overall improve the depth map quality. Post-processing can be added to :ref:`StereoDepth` node.
+
+
+Demo
+####
+
+.. image:: /_static/images/examples/depth_comparison.png
+
+Depth filters
+#############
+
+.. include::  /includes/depth-filters.rst
+
+.. rubric:: Similar samples:
+
+- :ref:`Depth Preview`
+- :ref:`Stereo Depth from host`
+
+Setup
+#####
+
+.. include::  /includes/install_from_pypi.rst
+
+Source code
+###########
+
+.. tabs::
+
+    .. tab:: Python
+
+        Also `available on GitHub <https://github.com/luxonis/depthai-python/blob/main/examples/StereoDepth/depth_post_processing.py>`__
+
+        .. literalinclude:: ../../../../examples/StereoDepth/depth_post_processing.py
+           :language: python
+           :linenos:
+
+    .. tab:: C++
+
+        Also `available on GitHub <https://github.com/luxonis/depthai-core/blob/main/examples/StereoDepth/depth_post_processing.cpp>`__
+
+        .. literalinclude:: ../../../../depthai-core/examples/StereoDepth/depth_post_processing.cpp
+           :language: cpp
+           :linenos:
+
+.. include::  /includes/footer-short.rst

docs/source/components/nodes/stereo_depth.rst

@@ -122,6 +122,7 @@ are presented with code.
 .. rubric:: StereoDepth
 
 - :ref:`Depth Crop Control` - Demonstrates how to control cropping of depth frames from the host
+- :ref:`Depth Post-Processing` - Depth post-processing filters
 - :ref:`Depth Preview` - Displays colorized stereo disparity
 - :ref:`Stereo Depth from host` - Generates stereo depth frame from a set of mono images from the host
 - :ref:`Stereo Depth Video` - An extended version of **Depth Preview**