Skip to content

Commit 1c23e84

Browse files
MaticToninMaticTonin
committed
Adding dynamic calibration examples
1 parent f3a818f commit 1c23e84

File tree

11 files changed

+1879
-0
lines changed

11 files changed

+1879
-0
lines changed

‎README.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -141,3 +141,18 @@ Afterwards, navigate to desired application as listed below
141141
</td>
142142
</tr>
143143
</table>
144+
145+
### [🔄 Dynamic Calibration](dynamic-calibration/)
146+
147+
<table>
148+
<tr>
149+
<td width="50%" valign="top">
150+
<img src="dynamic-calibration/media/dcl.gif" alt="Dynamic Calibration">
151+
</td>
152+
<td width="50%" valign="middle" align="center">
153+
Example of showing off the dynamic recalibration in action
154+
<br><br>
155+
<a href="dynamic-calibration/">🔄 Dynamic calibration</a>
156+
</td>
157+
</tr>
158+
</table>

‎dynamic-calibration/README.md‎

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
# Stereo Dynamic Calibration
2+
3+
This example demonstrates **runtime stereo camera calibration** with the `DynamicCalibration` node, plus a host-side controller/visualizer that overlays helpful UI (help panel, coverage bar, quality/recalibration modals, and a depth ROI HUD). It integrates with the [RemoteConnection](https://rvc4.docs.luxonis.com/software/depthai-components/tools/remote_connection/) service to visualize streams in real time.
4+
5+
> Works in **peripheral mode**: the device performs calibration; the host sends commands and renders overlays.
6+
7+
## Features
8+
9+
- **Interactive commands**: start/force recalibration, load images, run quality checks, apply/rollback calibrations, and **flash** (EEPROM) new/previous/factory calibration.
10+
- **Coverage bar**: centered, large progress bar while collecting frames (or briefly after `l`).
11+
- **Quality modal**: big 3-color bar (GOOD / COULD BE IMPROVED / NEEDS RECALIBRATION) with a pointer based on rotation change and a summary of depth-error deltas.
12+
- **Recalibration modal**: summary with Euler angle deltas and depth-error deltas; prompts to flash if there is a significant change.
13+
- **Depth HUD**: optional, shows depth/disp at a movable ROI (center + mean), with a small “tiny box” indicator.
14+
- **Always-on help panel** (toggleable).
15+
16+
## Demo
17+
18+
<p align="center">
19+
<img src="media/dcl.gif" alt="demo" />
20+
</p>
21+
22+
## Requirements
23+
24+
- A **Luxonis device** connected via USB/Ethernet.
25+
- Python **3.10+** (tested with 3.12).
26+
- Packages:
27+
- `depthai`
28+
- `depthai-nodes`
29+
- `opencv-python`
30+
- `numpy`
31+
32+
Install via:
33+
```bash
34+
pip install -r requirements.txt
35+
```
36+
37+
## Run
38+
39+
```bash
40+
python3 main.py
41+
# or
42+
python3 main.py --fps_limit 10
43+
# or
44+
python3 main.py --device 18443010C1BA9D1200
45+
```
46+
47+
When launched, the app starts a RemoteConnection server. Open the visualizer at:
48+
```
49+
http://localhost:8082
50+
```
51+
Replace `localhost` with your host IP if viewing from another machine.
52+
53+
## Wiring Notes (snippet)
54+
55+
Make sure to:
56+
1) Link a preview stream for timestamped overlays (e.g., colormapped disparity/depth),
57+
2) Link a raw depth OR disparity stream to the controller (for the ROI HUD),
58+
3) Provide the **device** handle to enable flashing.
59+
60+
```python
61+
visualizer = dai.RemoteConnection(httpPort=8082)
62+
device = pipeline.getDefaultDevice()
63+
64+
stereo = pipeline.create(dai.node.StereoDepth)
65+
# ... set stereo configs, link left/right, etc.
66+
67+
# Colormap for display
68+
depth_color = pipeline.create(ApplyColormap).build(stereo.disparity) # or stereo.depth
69+
depth_color.setColormap(cv2.COLORMAP_JET)
70+
71+
# Dynamic calibration node
72+
dyn_calib = pipeline.create(dai.node.DynamicCalibration)
73+
left_out.link(dyn_calib.left)
74+
right_out.link(dyn_calib.right)
75+
76+
# Controller
77+
dyn_ctrl = pipeline.create(DynamicCalibrationControler).build(
78+
preview=depth_color.out, # used for overlay timing
79+
depth=stereo.depth # or .disparity; call set_depth_units_is_mm(False) if disparity
80+
)
81+
dyn_ctrl.set_command_input(dyn_calib.inputControl.createInputQueue())
82+
dyn_ctrl.set_quality_output(dyn_calib.qualityOutput.createOutputQueue())
83+
dyn_ctrl.set_calibration_output(dyn_calib.calibrationOutput.createOutputQueue())
84+
dyn_ctrl.set_coverage_output(dyn_calib.coverageOutput.createOutputQueue())
85+
dyn_ctrl.set_device(device) # enables flashing p/k/f
86+
```
87+
88+
## Controls
89+
90+
Use these keys while the app is running (focus the browser visualizer window):
91+
92+
| Key | Action |
93+
| --- | ------ |
94+
| `q` | Quit the app |
95+
| `h` | Toggle help panel |
96+
| `g` | Toggle Depth HUD (ROI readout) |
97+
| `r` | Start recalibration |
98+
| `d` | **Force** recalibration |
99+
| `l` | Load image(s) for calibration (shows coverage bar for ~2s) |
100+
| `c` | Calibration quality check |
101+
| `v` | **Force** calibration quality check |
102+
| `n` | Apply **NEW** calibration (when available) |
103+
| `o` | Apply **PREVIOUS** calibration (rollback) |
104+
| `p` | **Flash NEW/current** calibration to EEPROM |
105+
| `k` | **Flash PREVIOUS** calibration to EEPROM |
106+
| `f` | **Flash FACTORY** calibration to EEPROM |
107+
| `w / a / s` | Move ROI up/left/down (Depth HUD).<br>**Note:** `d` is reserved for *Force recalibrate*. |
108+
| `z / x` | ROI size − / + |
109+
110+
> **Status banners** appear in the **center** after critical actions (e.g., applying/ flashing calibration) and auto-hide after ~2s.
111+
> **Modals** (quality/recalibration) also appear centered and auto-hide after ~3.5s or on any key press.
112+
113+
## On‑screen UI Cheat Sheet
114+
115+
- **Help panel** (top-left): quick reference of all keys (toggle with `h`).
116+
- **Coverage bar** (center): big progress bar while collecting frames; also shown briefly (≈2s) after pressing `l`.
117+
- **Quality modal** (center): three colored segments (green/yellow/red) with a **downward** pointer (`▼`) indicating rotation-change severity; optional line with depth-error deltas (@1m/2m/5m/10m).
118+
- **Recalibration modal** (center): “Recalibration complete”, significant-axis warning (if any), Euler angles, and depth-error deltas; suggests flashing if the change is significant.
119+
- **Depth HUD** (inline): shows depth/disp at the ROI center and mean within a tiny box; move with `w/a/s` (and resize with `z/x`).
120+
121+
## Output (console)
122+
123+
- **Coverage**: per-cell coverage and acquisition status when emitted by the device.
124+
- **Calibration results**: prints when a new calibration is produced and shows deltas:
125+
- Rotation delta `|| r_current - r_new ||` in degrees,
126+
- Mean Sampson error (new vs. current),
127+
- Theoretical **Depth Error Difference** at 1/2/5/10 meters.
128+
- **Quality checks**: same metrics as above without actually applying a new calibration.
129+
130+
## Tips & Notes
131+
132+
- To **flash** (EEPROM) from the UI you must pass the `device` into the controller (`dyn_ctrl.set_device(device)`).
133+
- If you link **disparity** instead of **depth** to the controller, call `dyn_ctrl.set_depth_units_is_mm(False)` so the HUD labels use “Disp” instead of meters.
134+
- The coverage percentage accepts either `[0..1]` or `[0..100]` from the device; the controller auto-detects and normalizes.
135+
- The **Collecting frames** bar hides automatically 2s after pressing `l`; during active recalibration (`r`/`d`) it stays up until calibration finishes.
136+
137+
## Installation (dev quick start)
138+
139+
```bash
140+
python3 -m venv .venv
141+
source .venv/bin/activate
142+
pip install -U pip
143+
pip install -r requirements.txt
144+
python3 main.py
145+
```
146+
147+
---
148+
149+
If you use this as a base for your own app, the heart of the UX is `utils/dynamic_controler.py` — it wires `DynamicCalibration` queues and renders all overlays via `ImgAnnotations` so you don’t need `cv2.imshow()`.

‎dynamic-calibration/main.py‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
#!/usr/bin/env python3
2+
3+
import depthai as dai
4+
from depthai_nodes.node import ApplyColormap
5+
from utils.arguments import initialize_argparser
6+
from utils.dynamic_controler import DynamicCalibrationControler
7+
import cv2
8+
import numpy as np
9+
print(dai.__file__)
10+
_, args = initialize_argparser()
11+
12+
visualizer = dai.RemoteConnection(httpPort=8082)
13+
# ---------- Pipeline definition ----------
14+
pipeline = dai.Pipeline()
15+
16+
# Create camera nodes
17+
cam_left = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_B)
18+
cam_right = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_C)
19+
20+
# Request full resolution NV12 outputs
21+
left_out = cam_left.requestFullResolutionOutput(dai.ImgFrame.Type.NV12, fps=args.fps_limit)
22+
right_out = cam_right.requestFullResolutionOutput(dai.ImgFrame.Type.NV12, fps=args.fps_limit)
23+
24+
# Stereo node
25+
stereo = pipeline.create(dai.node.StereoDepth)
26+
left_out.link(stereo.left)
27+
right_out.link(stereo.right)
28+
29+
# Dynamic calibration node
30+
dyn_calib = pipeline.create(dai.node.DynamicCalibration)
31+
left_out.link(dyn_calib.left)
32+
right_out.link(dyn_calib.right)
33+
34+
# Output queues
35+
depth_parser = pipeline.create(ApplyColormap).build(stereo.disparity)
36+
# depth_parser.setMaxValue(int(stereo.initialConfig.getMaxDisparity())) # NOTE: Uncomment when DAI fixes a bug
37+
depth_parser.setColormap(cv2.COLORMAP_JET)
38+
39+
device = pipeline.getDefaultDevice()
40+
41+
calibration = device.readCalibration()
42+
new_calibration = None
43+
old_calibration = None
44+
# --- existing ---
45+
calibration_output = dyn_calib.calibrationOutput.createOutputQueue()
46+
coverage_output = dyn_calib.coverageOutput.createOutputQueue()
47+
quality_output = dyn_calib.qualityOutput.createOutputQueue()
48+
input_control = dyn_calib.inputControl.createInputQueue()
49+
device.setCalibration(calibration)
50+
# ---------- Visualizer setup ----------
51+
visualizer.addTopic("Left", stereo.syncedLeft, "images")
52+
visualizer.addTopic("Right", stereo.syncedRight, "images")
53+
visualizer.addTopic("Depth", depth_parser.out, "images")
54+
55+
dyn_ctrl = pipeline.create(DynamicCalibrationControler).build(
56+
preview=depth_parser.out, # for timestamped overlays
57+
depth=stereo.depth # raw uint16 depth in mm
58+
)
59+
visualizer.addTopic("DynCalib HUD", dyn_ctrl.out_annotations, "images")
60+
61+
pipeline.start()
62+
visualizer.registerPipeline(pipeline)
63+
64+
# give it queues
65+
dyn_ctrl.set_coverage_output(coverage_output)
66+
dyn_ctrl.set_calibration_output(calibration_output)
67+
dyn_ctrl.set_command_input(input_control)
68+
dyn_ctrl.set_quality_output(quality_output)
69+
dyn_ctrl.set_depth_units_is_mm(True) # True for stereo.depth, False for disparity
70+
dyn_ctrl.set_device(device)
71+
72+
# (optional) seed current calibration
73+
try:
74+
dyn_ctrl.set_current_calibration(device.readCalibration())
75+
except Exception:
76+
pass
77+
78+
while pipeline.isRunning():
79+
key = visualizer.waitKey(1)
80+
if key != -1:
81+
dyn_ctrl.handle_key_press(key)
82+
if dyn_ctrl.wants_quit:
83+
break

‎dynamic-calibration/media/dcl.gif‎

5.81 MB
Loading

‎dynamic-calibration/oakapp.toml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
identifier = "com.example.dynamic-calibration"
2+
app_version = "1.0.0"
3+
4+
prepare_container = [
5+
{ type = "RUN", command = "apt-get update" },
6+
{ type = "RUN", command = "apt-get install -y python3-pip" },
7+
{ type = "COPY", source = "requirements.txt", target = "requirements.txt" },
8+
{ type = "RUN", command = "pip3 install -r /app/requirements.txt --break-system-packages" },
9+
]
10+
11+
prepare_build_container = []
12+
13+
build_steps = []
14+
15+
entrypoint = ["bash", "-c", "python3 -u /app/main.py"]

‎dynamic-calibration/requirements.txt‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
--extra-index-url https://artifacts.luxonis.com/artifactory/luxonis-python-snapshot-local
2+
depthai==3.0.0rc4.dev0+61682670ac5f02e7e6cf2e306fddce998e228720
3+
depthai-nodes==0.3.1
4+
numpy>=1.22
5+
open3d~=0.18
6+
opencv-python==4.10.0.84
7+
opencv-contrib-python==4.10.0.84

‎dynamic-calibration/utils/__init__.py‎

Whitespace-only changes.

‎dynamic-calibration/utils/arguments.py‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
import argparse
2+
3+
4+
def initialize_argparser():
5+
"""Initialize the argument parser for the script."""
6+
parser = argparse.ArgumentParser(
7+
formatter_class=argparse.ArgumentDefaultsHelpFormatter
8+
)
9+
parser.description = "This example showcases one possible approach for measuring the size of a box using DepthAI."
10+
11+
parser.add_argument(
12+
"-d",
13+
"--device",
14+
help="Optional name, DeviceID or IP of the camera to connect to.",
15+
required=False,
16+
default=None,
17+
type=str,
18+
)
19+
20+
parser.add_argument(
21+
"-fps",
22+
"--fps_limit",
23+
help="FPS limit for the model runtime.",
24+
required=False,
25+
default=10,
26+
type=int,
27+
)
28+
29+
args = parser.parse_args()
30+
31+
return parser, args

0 commit comments

Comments
 (0)