Skip to content

Commit 9fb5e6a

Browse files
committed
Refactor coordinate systems documentation for clarity and consistency in English and Chinese
1 parent 94169ab commit 9fb5e6a

File tree

2 files changed

+453
-78
lines changed

2 files changed

+453
-78
lines changed
Lines changed: 238 additions & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,29 @@
1-
### ROS2 Robot vs Camera Optical Coordination Systems
1+
## ROS2 Robot Coordinate System vs Camera Optical Coordinate System
22

3-
* Point Of View:
4-
* Imagine we are standing behind of the camera, and looking forward.
5-
* Always use this point of view when talking about coordinates, left vs right IRs, position of sensor, etc..
3+
* Point of View:
4+
* Imagine standing behind the camera and looking forward.
5+
* Always use this point of view when discussing coordinates, left vs right IR, sensor positions, etc.
66

77
![ROS2 and Camera Coordinate System](../image/application_guide/image0.png)
88

9-
* ROS2 Coordinate System: (X: Forward, Y:Left, Z: Up)
9+
* ROS2 Coordinate System: (X: Forward, Y: Left, Z: Up)
1010
* Camera Optical Coordinate System: (X: Right, Y: Down, Z: Forward)
11-
* All data published in our wrapper topics is optical data taken directly from our camera sensors.
12-
* static and dynamic TF topics publish optical CS and ROS CS to give the user the ability to move from one CS to other CS.
11+
* All data published in the wrapper topics is optical data taken directly from the camera sensors.
12+
* Static and dynamic TF topics publish optical and ROS coordinate systems so users can transform between them.
1313

1414
### Using ROS2 TF Tools
1515

16-
#### Viewing the TF Tree Structure
16+
#### View TF Tree Structure
1717

18-
You can use the following ROS2 commands to print and visualize the TF tree published by the camera package:
18+
Use the following ROS2 commands to print and visualize the TF tree published by the camera package:
1919

2020
**Print all TF relationships:**
2121

2222
```bash
2323
ros2 run tf2_tools view_frames
2424
```
2525

26-
This command generates a `frames.pdf` file that displays the hierarchical relationships between all frames.
26+
This command generates a `frames.pdf` file showing the hierarchy between all frames.
2727

2828
![image-20251027111351870](../image/application_guide/image4.png)
2929

@@ -33,9 +33,45 @@ This command generates a `frames.pdf` file that displays the hierarchical relati
3333
ros2 topic echo /tf_static
3434
```
3535

36-
#### Visualizing TF Tree with rviz2
36+
**View the TF transform between two specified frames:**
3737

38-
You can use rviz2 to visualize the TF tree structure and relative positions of coordinate systems in real-time:
38+
Use the following command to view the transform between two specific frames:
39+
40+
```bash
41+
ros2 run tf2_ros tf2_echo [source_frame] [target_frame]
42+
```
43+
44+
Example, view transform from `camera_link` to `camera_depth_optical_frame`:
45+
46+
```bash
47+
ros2 run tf2_ros tf2_echo camera_link camera_depth_optical_frame
48+
```
49+
50+
The command continuously outputs the real-time transform between the two frames, including:
51+
52+
- Translation: x, y, z (meters)
53+
- Rotation (Quaternion): x, y, z, w
54+
- Rotation (RPY): roll, pitch, yaw (radians and degrees)
55+
- Transform Matrix: 4×4 matrix with rotation and translation
56+
57+
Sample output:
58+
59+
```
60+
At time 0.0
61+
- Translation: [0.000, 0.000, 0.000]
62+
- Rotation: in Quaternion [-0.500, 0.500, -0.500, 0.500]
63+
- Rotation: in RPY (radian) [-1.571, -0.000, -1.571]
64+
- Rotation: in RPY (degree) [-90.000, -0.000, -90.000]
65+
- Matrix:
66+
0.000 0.000 1.000 0.000
67+
-1.000 0.000 0.000 0.000
68+
0.000 -1.000 0.000 0.000
69+
0.000 0.000 0.000 1.000
70+
```
71+
72+
#### Visualize TF Tree in rviz2
73+
74+
Use rviz2 to visualize the TF tree and relative frame poses in real time:
3975

4076
```bash
4177
rviz2
@@ -44,100 +80,247 @@ rviz2
4480
In rviz2:
4581

4682
- Add the `TF` display plugin
47-
- Configure the Fixed Frame to `camera_link` or `camera_depth_optical_frame`
48-
- Select the TF frames to display
83+
- Set the Fixed Frame to `camera_link` or `camera_depth_optical_frame`
84+
- Select which TF frames to display
4985

5086
![image-20251027140652727](../image/application_guide/image5.png)
5187

5288
### Camera TF Calculation and Publishing Mechanism
5389

5490
#### Core Function: `OBCameraNode::calcAndPublishStaticTransform()`
5591

56-
The camera node calculates and publishes static transformation relationships between all sensors through this function. Below is a detailed explanation of the code:
92+
The camera node uses this function to calculate and publish all static transforms between sensors.
93+
94+
```cpp
95+
void OBCameraNode::calcAndPublishStaticTransform() {
96+
tf2::Quaternion quaternion_optical, zero_rot;
97+
zero_rot.setRPY(0.0, 0.0, 0.0);
98+
quaternion_optical.setRPY(-M_PI / 2, 0.0, -M_PI / 2);
99+
tf2::Vector3 zero_trans(0, 0, 0);
100+
auto base_stream_profile = stream_profile_[base_stream_];
101+
auto device_info = device_->getDeviceInfo();
102+
CHECK_NOTNULL(device_info);
103+
auto pid = device_info->getPid();
104+
if (!base_stream_profile) {
105+
RCLCPP_ERROR_STREAM(logger_, "Failed to get base stream profile");
106+
return;
107+
}
108+
CHECK_NOTNULL(base_stream_profile.get());
109+
for (const auto &item : stream_profile_) {
110+
auto stream_index = item.first;
57111

58-
#### Quaternion Initialization and Coordinate System Transformation
112+
auto stream_profile = item.second;
113+
if (!stream_profile) {
114+
continue;
115+
}
116+
OBExtrinsic ex;
117+
try {
118+
ex = stream_profile->getExtrinsicTo(base_stream_profile);
119+
} catch (const ob::Error &e) {
120+
RCLCPP_ERROR_STREAM(logger_, "Failed to get " << stream_name_[stream_index]
121+
<< " extrinsic: " << e.getMessage());
122+
ex = OBExtrinsic({{1, 0, 0, 0, 1, 0, 0, 0, 1}, {0, 0, 0}});
123+
}
124+
125+
auto Q = rotationMatrixToQuaternion(ex.rot);
126+
Q = quaternion_optical * Q * quaternion_optical.inverse();
127+
tf2::Vector3 trans(ex.trans[0], ex.trans[1], ex.trans[2]);
128+
auto timestamp = node_->now();
129+
if (stream_index.first != base_stream_.first) {
130+
if (stream_index.first == OB_STREAM_IR_RIGHT && base_stream_.first == OB_STREAM_DEPTH) {
131+
trans[0] = std::abs(trans[0]); // because left and right ir calibration is error
132+
}
133+
publishStaticTF(timestamp, trans, Q, frame_id_[base_stream_], frame_id_[stream_index]);
134+
}
135+
publishStaticTF(timestamp, zero_trans, quaternion_optical, frame_id_[stream_index],
136+
optical_frame_id_[stream_index]);
137+
RCLCPP_INFO_STREAM(logger_, "Publishing static transform from " << stream_name_[stream_index]
138+
<< " to "
139+
<< stream_name_[base_stream_]);
140+
RCLCPP_INFO_STREAM(logger_, "Translation " << trans[0] << ", " << trans[1] << ", " << trans[2]);
141+
RCLCPP_INFO_STREAM(logger_, "Rotation " << Q.getX() << ", " << Q.getY() << ", " << Q.getZ()
142+
<< ", " << Q.getW());
143+
}
144+
145+
if ((pid == FEMTO_BOLT_PID || pid == FEMTO_MEGA_PID) && enable_stream_[DEPTH] &&
146+
enable_stream_[COLOR] && enable_publish_extrinsic_) {
147+
// calc depth to color
148+
149+
CHECK_NOTNULL(stream_profile_[COLOR]);
150+
auto depth_to_color_extrinsics = base_stream_profile->getExtrinsicTo(stream_profile_[COLOR]);
151+
auto Q = rotationMatrixToQuaternion(depth_to_color_extrinsics.rot);
152+
Q = quaternion_optical * Q * quaternion_optical.inverse();
153+
publishStaticTF(node_->now(), zero_trans, Q, camera_link_frame_id_, frame_id_[base_stream_]);
154+
} else {
155+
publishStaticTF(node_->now(), zero_trans, zero_rot, camera_link_frame_id_,
156+
frame_id_[base_stream_]);
157+
}
158+
159+
if (enable_stream_[DEPTH] && enable_stream_[COLOR] && enable_publish_extrinsic_) {
160+
static const char *frame_id = "depth_to_color_extrinsics";
161+
OBExtrinsic ex;
162+
try {
163+
ex = base_stream_profile->getExtrinsicTo(stream_profile_[COLOR]);
164+
} catch (const ob::Error &e) {
165+
RCLCPP_ERROR_STREAM(logger_,
166+
"Failed to get " << frame_id << " extrinsic: " << e.getMessage());
167+
ex = OBExtrinsic({{1, 0, 0, 0, 1, 0, 0, 0, 1}, {0, 0, 0}});
168+
}
169+
depth_to_other_extrinsics_[COLOR] = ex;
170+
auto ex_msg = obExtrinsicsToMsg(ex, frame_id);
171+
CHECK_NOTNULL(depth_to_other_extrinsics_publishers_[COLOR]);
172+
depth_to_other_extrinsics_publishers_[COLOR]->publish(ex_msg);
173+
}
174+
175+
if (enable_stream_[DEPTH] && enable_stream_[INFRA0] && enable_publish_extrinsic_) {
176+
static const char *frame_id = "depth_to_ir_extrinsics";
177+
OBExtrinsic ex;
178+
try {
179+
ex = base_stream_profile->getExtrinsicTo(stream_profile_[INFRA0]);
180+
} catch (const ob::Error &e) {
181+
RCLCPP_ERROR_STREAM(logger_,
182+
"Failed to get " << frame_id << " extrinsic: " << e.getMessage());
183+
ex = OBExtrinsic({{1, 0, 0, 0, 1, 0, 0, 0, 1}, {0, 0, 0}});
184+
}
185+
depth_to_other_extrinsics_[INFRA0] = ex;
186+
auto ex_msg = obExtrinsicsToMsg(ex, frame_id);
187+
CHECK_NOTNULL(depth_to_other_extrinsics_publishers_[INFRA0]);
188+
depth_to_other_extrinsics_publishers_[INFRA0]->publish(ex_msg);
189+
}
190+
if (enable_stream_[DEPTH] && enable_stream_[INFRA1] && enable_publish_extrinsic_) {
191+
static const char *frame_id = "depth_to_left_ir_extrinsics";
192+
OBExtrinsic ex;
193+
try {
194+
ex = base_stream_profile->getExtrinsicTo(stream_profile_[INFRA1]);
195+
} catch (const ob::Error &e) {
196+
RCLCPP_ERROR_STREAM(logger_,
197+
"Failed to get " << frame_id << " extrinsic: " << e.getMessage());
198+
ex = OBExtrinsic({{1, 0, 0, 0, 1, 0, 0, 0, 1}, {0, 0, 0}});
199+
}
200+
depth_to_other_extrinsics_[INFRA1] = ex;
201+
auto ex_msg = obExtrinsicsToMsg(ex, frame_id);
202+
CHECK_NOTNULL(depth_to_other_extrinsics_publishers_[INFRA1]);
203+
depth_to_other_extrinsics_publishers_[INFRA1]->publish(ex_msg);
204+
}
205+
if (enable_stream_[DEPTH] && enable_stream_[INFRA2] && enable_publish_extrinsic_) {
206+
static const char *frame_id = "depth_to_right_ir_extrinsics";
207+
OBExtrinsic ex;
208+
try {
209+
ex = base_stream_profile->getExtrinsicTo(stream_profile_[INFRA2]);
210+
} catch (const ob::Error &e) {
211+
RCLCPP_ERROR_STREAM(logger_,
212+
"Failed to get " << frame_id << " extrinsic: " << e.getMessage());
213+
ex = OBExtrinsic({{1, 0, 0, 0, 1, 0, 0, 0, 1}, {0, 0, 0}});
214+
}
215+
ex.trans[0] = -std::abs(ex.trans[0]);
216+
depth_to_other_extrinsics_[INFRA2] = ex;
217+
auto ex_msg = obExtrinsicsToMsg(ex, frame_id);
218+
CHECK_NOTNULL(depth_to_other_extrinsics_publishers_[INFRA2]);
219+
depth_to_other_extrinsics_publishers_[INFRA2]->publish(ex_msg);
220+
}
221+
if (enable_stream_[DEPTH] && enable_stream_[ACCEL] && enable_publish_extrinsic_) {
222+
static const char *frame_id = "depth_to_accel_extrinsics";
223+
OBExtrinsic ex;
224+
try {
225+
ex = base_stream_profile->getExtrinsicTo(stream_profile_[ACCEL]);
226+
} catch (const ob::Error &e) {
227+
RCLCPP_ERROR_STREAM(logger_,
228+
"Failed to get " << frame_id << " extrinsic: " << e.getMessage());
229+
ex = OBExtrinsic({{1, 0, 0, 0, 1, 0, 0, 0, 1}, {0, 0, 0}});
230+
}
231+
depth_to_other_extrinsics_[ACCEL] = ex;
232+
auto ex_msg = obExtrinsicsToMsg(ex, frame_id);
233+
CHECK_NOTNULL(depth_to_other_extrinsics_publishers_[ACCEL]);
234+
depth_to_other_extrinsics_publishers_[ACCEL]->publish(ex_msg);
235+
}
236+
if (enable_stream_[DEPTH] && enable_stream_[GYRO] && enable_publish_extrinsic_) {
237+
static const char *frame_id = "depth_to_gyro_extrinsics";
238+
OBExtrinsic ex;
239+
try {
240+
ex = base_stream_profile->getExtrinsicTo(stream_profile_[GYRO]);
241+
} catch (const ob::Error &e) {
242+
RCLCPP_ERROR_STREAM(logger_,
243+
"Failed to get " << frame_id << " extrinsic: " << e.getMessage());
244+
ex = OBExtrinsic({{1, 0, 0, 0, 1, 0, 0, 0, 1}, {0, 0, 0}});
245+
}
246+
depth_to_other_extrinsics_[GYRO] = ex;
247+
auto ex_msg = obExtrinsicsToMsg(ex, frame_id);
248+
CHECK_NOTNULL(depth_to_other_extrinsics_publishers_[GYRO]);
249+
depth_to_other_extrinsics_publishers_[GYRO]->publish(ex_msg);
250+
}
251+
if (enable_sync_output_accel_gyro_) {
252+
tf2::Quaternion zero_rot;
253+
zero_rot.setRPY(0.0, 0.0, 0.0);
254+
tf2::Vector3 zero_trans(0, 0, 0);
255+
publishStaticTF(node_->now(), zero_trans, zero_rot, optical_frame_id_[GYRO],
256+
accel_gyro_frame_id_);
257+
}
258+
}
259+
```
260+
261+
#### Function Breakdown
262+
263+
Detailed explanation of the code:
264+
265+
**Quaternion Initialization and Coordinate Transform**
59266

60267
```cpp
61268
tf2::Quaternion quaternion_optical, zero_rot;
62269
zero_rot.setRPY(0.0, 0.0, 0.0);
63270
quaternion_optical.setRPY(-M_PI / 2, 0.0, -M_PI / 2);
64271
```
65272

66-
**Explanation:**
67-
68-
- `quaternion_optical`: Defines the rotation transformation from the optical coordinate system to the ROS standard coordinate system (90-degree rotation)
69-
- This rotation converts the camera optical coordinate system (X right, Y down, Z forward) to the ROS standard coordinate system (X forward, Y left, Z up)
273+
- `quaternion_optical`: Defines the rotation from optical coordinates to ROS standard (90° rotation)
274+
- Converts camera optical CS (X right, Y down, Z forward) to ROS CS (X forward, Y left, Z up)
70275

71-
#### Obtaining Device Information and Base Stream
276+
**Get Device Info and Base Stream**
72277

73278
```cpp
74279
auto base_stream_profile = stream_profile_[base_stream_];
75280
auto device_info = device_->getDeviceInfo();
76-
// The base stream is typically the DEPTH stream
281+
// Base stream usually DEPTH
77282
```
78283

79-
**Explanation:**
80-
81-
- A base stream (typically the depth stream) is selected, and all other sensor transformations are calculated relative to this base stream
284+
- Choose a base stream (usually depth); all other transforms are relative to it
82285

83-
#### Iterating Through All Streams and Calculating Relative Transformations
286+
**Iterate Streams and Compute Relative Transforms**
84287

85288
```cpp
86289
for (const auto &item : stream_profile_) {
87290
auto stream_index = item.first;
88291
auto stream_profile = item.second;
89-
90-
// Get the extrinsics of this stream relative to the base stream
91292
OBExtrinsic ex;
92293
ex = stream_profile->getExtrinsicTo(base_stream_profile);
93-
94-
// Convert rotation matrix to quaternion
95294
auto Q = rotationMatrixToQuaternion(ex.rot);
96-
97-
// Apply optical coordinate system transformation: Q_new = quaternion_optical * Q * quaternion_optical.inverse()
98295
Q = quaternion_optical * Q * quaternion_optical.inverse();
99-
100296
tf2::Vector3 trans(ex.trans[0], ex.trans[1], ex.trans[2]);
101297
```
102298
103-
**Explanation:**
104-
105-
- `OBExtrinsic` contains the rotation matrix (`rot`) and translation vector (`trans`) between two sensors
106-
- Quaternion multiplication applies the optical coordinate system transformation to each sensor's rotation relationship
107-
- This transformation converts the camera's native optical coordinate system to the ROS standard coordinate system
299+
- `OBExtrinsic` holds rotation matrix (`rot`) and translation vector (`trans`)
300+
- Apply optical-to-ROS rotation via quaternion multiplication
108301
109-
#### Publishing TF Transformations
302+
**Publish TF Transforms**
110303
111304
```cpp
112-
// Publish the transformation from sensor to base stream (in ROS coordinate system)
113305
publishStaticTF(timestamp, trans, Q, frame_id_[base_stream_], frame_id_[stream_index]);
114-
115-
// Publish the transformation from physical frame to its optical frame
116306
publishStaticTF(timestamp, zero_trans, quaternion_optical, frame_id_[stream_index],
117307
optical_frame_id_[stream_index]);
118308
```
119309

120-
**Explanation:**
310+
- First: base stream to sensor (translation + rotation)
311+
- Second: sensor frame to optical frame (pure rotation)
121312

122-
- First `publishStaticTF`: Publishes the transformation from the base stream to the current sensor (translation + rotation)
123-
- Second `publishStaticTF`: Publishes the transformation from physical frame to optical frame (pure rotation, no translation)
124-
- `frame_id_[stream_index]`: Physical coordinate system frame name (e.g., `camera_depth_frame`)
125-
- `optical_frame_id_[stream_index]`: Optical coordinate system frame name (e.g., `camera_depth_optical_frame`)
126-
127-
#### Special Handling for Left and Right IR Cameras
313+
**Special Handling for Left/Right IR**
128314

129315
```cpp
130316
if (stream_index.first == OB_STREAM_IR_RIGHT && base_stream_.first == OB_STREAM_DEPTH) {
131317
trans[0] = std::abs(trans[0]);
132318
}
133319
```
134320

135-
**Explanation:**
136-
137-
- Left and right IR cameras are symmetric about the center plane in the device coordinate system
138-
- Using `abs()` ensures the X-axis offset is positive, maintaining geometric consistency
321+
- Ensures symmetry consistency between left/right IR cameras
139322

140-
#### Publishing Extrinsics from Depth to Other Sensors
323+
**Publish Depth-to-Other Extrinsics**
141324

142325
```cpp
143326
if (enable_stream_[DEPTH] && enable_stream_[COLOR] && enable_publish_extrinsic_) {
@@ -147,7 +330,4 @@ if (enable_stream_[DEPTH] && enable_stream_[COLOR] && enable_publish_extrinsic_)
147330
}
148331
```
149332

150-
**Explanation:**
151-
152-
- In addition to publishing transformation relationships through TF, raw extrinsic parameters are also published through custom topics
153-
- This allows users to directly access the camera's intrinsic and extrinsic parameters for high-precision point cloud alignment and depth-color registration
333+
- Publishes raw extrinsics via topic for advanced alignment and registration

0 commit comments

Comments
 (0)