Skip to content

Feature/eskf landmark valve#667

Open
AntTra wants to merge 128 commits intodev/eskf_fuse_depthfrom
feature/eskf-landmark-valve
Open

Feature/eskf landmark valve#667
AntTra wants to merge 128 commits intodev/eskf_fuse_depthfrom
feature/eskf-landmark-valve

Conversation

@AntTra
Copy link
Copy Markdown

@AntTra AntTra commented Feb 15, 2026

Visual egomotion feature in ESKF

The current ESKF struggles with drift over time during the mission, especially when the DVL loses bottom lock when the AUV is pitched to look at the bottom valve with the front camera. This project aims to use landmark-based visual odometry as an additional measurement to be fused to the ESKF as 6-dof pose and derived velocity measurements alongside the IMU and DVL.

It works by placing an anchor on the first valid landmark detection and calculates the relative positioning, computing a transform from the VO frame to the navigation frame. All subsequent marker measurements are mapped through this anchor to produce position and orientation in the navigation frame. Velocity is derived from consecutive position measurements. NIS gating on both pose and velocity updates are based on chi-squared thresholds.

Features:

  • Optional sliding window module for robust quaternion averaging using log/exp maps, Huber weighting for linear downweighting less likely measurements, and angular gating to reject outliers before they reach the filter
  • Smart anchor reset which automatically re-anchors after a set amout of configurable consecutive NIS rejections or dropout

For testing:

colcon build --symlink-install --packages-select eskf aruco_detector
source install/setup.bash

In one terminal run

ros2 launch eskf eskf.launch.py

and in another

ros2 launch aruco_detector aruco_detector.launch.py

together with the simulator and direct the AUV towards an AruCo-marker.
Test anchor reset on marker dropout, when the marker ID gets switched, and the performance during movement.

Note: Another important test would be to test the performance of the sliding window as the ESKF already acts as a filter for outliers
TODO: Change weighted quaternion averaging method for estimation. Replace Huber weighting with ref(?): (https://ntrs.nasa.gov/api/citations/20070017872/downloads/20070017872.pdf). Filter buffer with gating before passing to average_quaternions

Talhanc and others added 30 commits February 9, 2025 19:47
…ses msg imu/data_raw and /orca/pose as the dvl info
@AntTra AntTra force-pushed the feature/eskf-landmark-valve branch from 12dc7d0 to 91ec174 Compare March 2, 2026 19:20
@AntTra AntTra force-pushed the feature/eskf-landmark-valve branch 2 times, most recently from b395d9b to 752d7f7 Compare March 17, 2026 07:43
@AntTra AntTra force-pushed the feature/eskf-landmark-valve branch from 752d7f7 to 024dc22 Compare March 17, 2026 08:24
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 17, 2026

Codecov Report

❌ Patch coverage is 0% with 254 lines in your changes missing coverage. Please review.
✅ Project coverage is 35.33%. Comparing base (95b70dc) to head (bbb983c).

Files with missing lines Patch % Lines
.../landmark_egomotion/src/lib/landmark_egomotion.cpp 0.00% 149 Missing ⚠️
navigation/eskf/src/ros/eskf_ros.cpp 0.00% 89 Missing ⚠️
...ission/landmark_server/src/landmark_server_ros.cpp 0.00% 7 Missing ⚠️
...andmark_server/src/landmark_server_convergence.cpp 0.00% 5 Missing ⚠️
...lude/landmark_egomotion/lib/landmark_egomotion.hpp 0.00% 2 Missing ⚠️
navigation/eskf/include/eskf/lib/eskf.hpp 0.00% 1 Missing ⚠️
...clude/landmark_egomotion/lib/landmark_typedefs.hpp 0.00% 1 Missing ⚠️
Additional details and impacted files
@@                   Coverage Diff                   @@
##           dev/eskf_fuse_depth     #667      +/-   ##
=======================================================
- Coverage                36.78%   35.33%   -1.45%     
=======================================================
  Files                       86       90       +4     
  Lines                     5755     5991     +236     
  Branches                  1749     1779      +30     
=======================================================
  Hits                      2117     2117              
- Misses                    3192     3428     +236     
  Partials                   446      446              
Flag Coverage Δ
unittests 35.33% <0.00%> (-1.45%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...er/include/landmark_server/landmark_server_ros.hpp 0.00% <ø> (ø)
navigation/eskf/include/eskf/lib/eskf.tpp 0.00% <ø> (ø)
navigation/eskf/include/eskf/lib/typedefs.hpp 0.00% <ø> (ø)
navigation/eskf/include/eskf/ros/eskf_ros.hpp 0.00% <ø> (ø)
navigation/eskf/src/lib/eskf.cpp 0.00% <ø> (ø)
navigation/eskf/include/eskf/lib/eskf.hpp 0.00% <0.00%> (ø)
...clude/landmark_egomotion/lib/landmark_typedefs.hpp 0.00% <0.00%> (ø)
...lude/landmark_egomotion/lib/landmark_egomotion.hpp 0.00% <0.00%> (ø)
...andmark_server/src/landmark_server_convergence.cpp 0.00% <0.00%> (ø)
...ission/landmark_server/src/landmark_server_ros.cpp 0.00% <0.00%> (ø)
... and 2 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@AntTra AntTra force-pushed the feature/eskf-landmark-valve branch from 7f48bf3 to 6af0de2 Compare March 17, 2026 11:58
Copy link
Copy Markdown
Contributor

@kluge7 kluge7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, can do a more thorough review once we merge it into main

Comment on lines +11 to +12
nis_gate_pose: 16.812
nis_gate_velocity: 11.345
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Magic numbers, explaing how you got these numbers

Copy link
Copy Markdown
Member

@chrstrom chrstrom Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be a good fit for an eskf wiki-page? Like for future reference, instead of adding "value from P(X ≤ chi^2) = 0.99 with 6dof / 3dof" in comments, having a bit more space to explain why this is done.

(And hopefully to avoid having the next N+1 years teams re-implement yet another ESKF 😭)

Comment on lines +27 to +31
#ifdef HAVE_LANDMARK_EGOMOTION
#include <vortex_msgs/msg/landmark_array.hpp>
#include "landmark_egomotion/lib/landmark_egomotion.hpp"
#include "landmark_egomotion/lib/landmark_typedefs.hpp"
#endif
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unsure about this, but probably fine for now

Comment on lines +76 to +78
bool have_last_marker_{false};
uint16_t last_marker_id_{0};
int vo_rejects_limit_{0};
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be consistent with either using {} or just setting =, but can fix when merging to main

this->declare_parameter<std::string>("vo.landmarks_topic");
this->declare_parameter<std::string>("vo.base_frame");
this->declare_parameter<std::string>("vo.camera_frame");
this->declare_parameter<bool>("vo.disable_gating", false);
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that a rare default value in the wild? can be removed since it is already defined in config

vo_cfg.sw_huber_deg = this->get_parameter("vo.huber_deg").as_double();
vo_cfg.sw_gate_deg = this->get_parameter("vo.gate_deg").as_double();

landmark_eskf_->set_vo_enabled(true);
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

look at this mess

Comment on lines +213 to +217
for (const auto& lm : msg->landmarks) {
if (lm.type.value == vortex_msgs::msg::LandmarkType::ARUCO_MARKER) {
markers.push_back(&lm);
}
}
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only handles the LandmarkType being ARUCO_MARKER. should add a default case for handling other types (currently just failing gracefully)

Comment on lines +240 to +241
tf_msg = tf_buffer_->lookupTransform(vo_base_frame_, vo_cam_frame_,
tf2::TimePointZero);
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Get from recent message instead of the first?? Unsure, might be fine?? Idk?? msg filter in ros which makes it so that the callback only runs if the transformation is available? more effective but this is probably fine

Comment on lines +273 to +278
LandmarkMeasurement meas;
meas.pos = t_base;
meas.quat = q_base;
meas.stamp = rclcpp::Time(msg->header.stamp).seconds();
meas.R = Eigen::Map<const Eigen::Matrix<double, 6, 6, Eigen::RowMajor>>(
chosen->pose.covariance.data());
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unsure about this, if you should either pass in the different measurements and then create a LandmarkMeasurement in the function, or as it is now.

@AntTra AntTra force-pushed the feature/eskf-landmark-valve branch from db9668f to d3b0b14 Compare March 23, 2026 19:12
…s from ESKF directly. Fixed quaternion error to use right perturbation
@AntTra AntTra force-pushed the feature/eskf-landmark-valve branch from d3b0b14 to dc82fb7 Compare March 23, 2026 19:21
@AntTra AntTra force-pushed the feature/eskf-landmark-valve branch from cdb6a8d to bbb983c Compare April 4, 2026 22:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorporate landmark_pose into eskf

8 participants