[WIP] Tracker service v0.3.2: robot vision integration#968
Draft
jdanieck wants to merge 39 commits intotracker-service-v0.3.1from
Draft
[WIP] Tracker service v0.3.2: robot vision integration#968jdanieck wants to merge 39 commits intotracker-service-v0.3.1from
jdanieck wants to merge 39 commits intotracker-service-v0.3.1from
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR integrates RobotVision's Kalman filter tracking library into the tracker service, replacing the previous stub tracking implementation. The change upgrades track IDs from UUID strings to integer IDs produced by RobotVision's tracking system.
Changes:
- Integrated RobotVision TrackTracker with Kalman filter for multi-object tracking
- Changed track ID type from string to int32_t to match RobotVision's output
- Added camera calibration support for pixel-to-world coordinate transformation
- Updated tests to reflect real tracking behavior (tracks require multiple detections for reliability)
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tracker/src/tracking_worker.cpp | Replaced stub tracking with RobotVision TrackTracker integration, added camera calibration handling |
| tracker/inc/tracking_worker.hpp | Added RobotVision headers and member variables for tracker and camera parameters |
| tracker/inc/tracking_types.hpp | Changed Track::id from string to int32_t |
| tracker/src/track_publisher.cpp | Updated JSON serialization to use SetInt() instead of SetString() for track ID |
| tracker/src/time_chunk_scheduler.cpp | Refactored worker creation to build camera map and pass tracking config |
| tracker/schema/scene-data.schema.json | Updated schema to enforce integer track IDs with minimum 0 |
| tracker/test/unit/tracking_worker_test.cpp | Added helper functions, new tests for edge cases, updated expectations for RobotVision behavior |
| tracker/test/unit/track_publisher_test.cpp | Updated test data to use integer track IDs |
| tracker/test/unit/time_chunk_scheduler_test.cpp | Added camera intrinsics/distortion to test fixtures |
| tracker/test/unit/message_handler_test.cpp | Added test for lagged message handling |
| tracker/test/service/test_mqtt.py | Added detection sequence helper and new tracking reliability test |
Comments suppressed due to low confidence (5)
tracker/test/service/test_mqtt.py:1
- Test code contains timestamps in January 2026, which is in the future relative to the current date (February 3, 2026). Consider using timestamps that are clearly test data or relative to the current time to avoid confusion.
#!/usr/bin/env python3
tracker/test/service/test_mqtt.py:1
- Test code contains timestamps in January 2026, which is in the future relative to the current date (February 3, 2026). Consider using timestamps that are clearly test data or relative to the current time to avoid confusion.
#!/usr/bin/env python3
tracker/test/service/test_mqtt.py:1
- Test code contains timestamps in January 2026, which is in the future relative to the current date (February 3, 2026). Consider using timestamps that are clearly test data or relative to the current time to avoid confusion.
#!/usr/bin/env python3
tracker/test/service/test_mqtt.py:1
- Test code contains timestamps in January 2026, which is in the future relative to the current date (February 3, 2026). Consider using timestamps that are clearly test data or relative to the current time to avoid confusion.
#!/usr/bin/env python3
tracker/test/service/test_mqtt.py:1
- Test code contains timestamps in January 2026, which is in the future relative to the current date (February 3, 2026). Consider using timestamps that are clearly test data or relative to the current time to avoid confusion.
#!/usr/bin/env python3
jdanieck
changed the title
15 tasks
jdanieck
force-pushed
the
tracker-service-v0.3.1
branch
from
d186ad4 to
6fd555a
Compare
jdanieck
force-pushed
the
tracker-service-v0.3.2
branch
from
fc4b99c to
70db2a4
Compare
jdanieck
force-pushed
the
tracker-service-v0.3.1
branch
2 times, most recently
from
7aaba31 to
5fee258
Compare
jdanieck
force-pushed
the
tracker-service-v0.3.2
branch
from
70db2a4 to
d527ff8
Compare
jdanieck
force-pushed
the
tracker-service-v0.3.2
branch
from
d527ff8 to
d2031d9
Compare
Replace hardcoded 2020 timestamp with dynamically generated past timestamp using generate_past_iso_timestamp() helper. This makes the test's intent clearer and more maintainable. Addresses review comment about explaining timestamp choice.
Replace std::mktime (local time) with C++20 calendar types (sys_days, year_month_day) for portable UTC timestamp conversion. Changes: - Use std::from_chars for fast integer parsing without exceptions - Use std::all_of for digit validation - Add comprehensive input validation with logging - Use std::format for fallback timestamp generation - Remove iomanip/sstream dependencies Addresses review comment about mktime interpreting 'Z' timestamps as local time instead of UTC.
Change obj.width from world_bbox.width to world_bbox.height to use the correct transformed bounding box dimension. The previous code was a copy/paste error that assigned the same value to both length and width.
Add early return for empty camera_batches to prevent run_tracking from querying getReliableTracks() with no new input, which could republish stale tracks from the tracker's internal state. Empty chunks now publish an empty tracks vector with current timestamp.
Correct the docstring to accurately reflect the function's behavior: - Returns ground plane intersection for downward rays - Returns horizon-projected point for upward/parallel rays (horizon culling) - Never returns nullopt despite the optional return type
Use C++20 <numbers> header for portable pi constant instead of platform-specific M_PI macro from cmath.
Remove trailing comma from environment array that made the JSON invalid per strict JSON specification.
std::stod can throw std::out_of_range for values that overflow double. Handle this case alongside the existing std::invalid_argument catch.
The cv.wait_for without a predicate can return early on spurious wakeups. Add callback_called flag and use it as the wait predicate.
Match pixelToWorld docs - returns horizon point for upward rays, not nullopt. Both methods now have consistent documentation.
Use polling loop on in_callback flag instead of fixed 50ms sleep. This makes the test more reliable on slow/loaded CI runners.
These headers are used directly (std::optional<int> parse_int, chrono types in parse_timestamp/process_chunk) but were only included transitively. Explicit includes follow the include-what-you-use principle. Addresses PR #968 review comment about missing includes.
Use CoordinateTransformer (full intrinsics + extrinsics pipeline) instead of rv::computePixelsToMeterPlane (undistort-only). This changes published Track.translation from camera-relative normalized coords to actual world coordinates via the bboxFootToWorld() method (undistort -> pose -> ray-plane intersection with ground plane z=0). Changes: - TrackingWorker stores CoordinateTransformer per camera instead of raw intrinsics/distortion matrices - run_tracking() uses bboxFootToWorld() for pixel-to-world transform - Remove get_camera_params() method (no longer needed) - Update test cameras to include extrinsics Addresses PR #968 review comment about computePixelsToMeterPlane lacking extrinsics.
Replace unbounded while loops with deadline-bounded loops that fail with ASSERT_LT after 5 seconds. Prevents CI from hanging forever if the worker thread deadlocks. Addresses PR #968 review comment about unbounded polling in QueueDepth_ReturnsCorrectSize.
Replace unbounded while loop with deadline-bounded loop that fails with ASSERT_LT after 5 seconds. Prevents CI from hanging forever if the worker thread deadlocks. Addresses PR #968 review comment about unbounded polling in QueueFull_IncrementsDroppedCount.
The cv.wait_for() return value was discarded, so the test would silently pass even if the callback was never invoked. Wrap in ASSERT_TRUE to fail explicitly on timeout. Addresses PR #968 review comment about discarded wait_for result.
The 1e-9 tolerance leaves very little margin for differences in sin/cos implementations across compilers, libm versions, and floating-point optimization flags. 1e-8 is still strict enough to catch real bugs (wrong rotation order differs by ~0.7) while being robust across build configurations. Addresses PR #968 review comment about strict rotation tolerance.
Use std::format("{:%Y-%m-%dT%H:%M:%S}.000Z", ...) with chrono time_point
instead of POSIX-only gmtime_r + std::put_time. This is consistent with
the approach used in tracking_worker.cpp and uses only standard C++20.
Remove unused <iomanip> and <sstream> includes.
Addresses PR #968 review comment about gmtime_r portability.
The run/run-debug targets export TRACKER_MQTT_HOST, TRACKER_MQTT_PORT, TRACKER_MQTT_INSECURE, and TRACKER_MQTT_TLS_CA_CERT. If a developer runs 'make run' then 'make test-service' in the same shell, those env vars would leak into the pytest process. Add explicit unset to isolate the test environment. Addresses PR #968 review comment about removed env var isolation.
…comments - Add hour/minute/second range validation in parse_timestamp() to reject out-of-range values like T99:99:99Z instead of silently producing shifted time_points - Fix path comment in generate_reference_data.py: correct directory name (tools, not transformations) and traversal count (4 levels, not 3)
jdanieck
force-pushed
the
tracker-service-v0.3.2
branch
from
26444de to
8bbf97a
Compare
jdanieck
commented
Feb 10, 2026
jdanieck
commented
Feb 10, 2026
jdanieck
commented
Feb 10, 2026
Object size transformation: - Add bboxSizeToWorld() to CoordinateTransformer projecting bbox corners to ground plane and computing width/height in meters - Match Python controller's projectBounds() elevation angle geometry - Size convention: [width, width, height] per moving_object.py - Skip detection (no fallback) if size projection fails - Add Python reference data generator and 3 C++ unit tests Yaw-to-quaternion rotation: - Add static yawToQuaternion() converting RobotVision's Z-axis yaw (radians) to [x, y, z, w] quaternion: [0, 0, sin(yaw/2), cos(yaw/2)] - Replace identity quaternion placeholder in tracking_worker - Add Python reference data generator and 4 C++ unit tests Container hardening: - Add read_only, cap_drop: [ALL], security_opt: [no-new-privileges:true] to tracker service in sample_data and test docker-compose files
Extract shared parseTimestamp() into time_utils.hpp/cpp, replacing duplicate implementations in message_handler and tracking_worker. - sscanf-based ISO 8601 parser with C++20 calendar validation - Store parsed time_point in DetectionBatch (single parse per message) - 21 parameterized unit tests (valid, invalid, round-trip) - Remove ~110 lines of duplicated parsing code
Remove detection filter in message_handler — empty detection lists are now buffered so the tracker sees zero-detection frames for track aging. - Remove empty-chunk early return in tracking_worker (all chunks flow through run_tracking) - Add formatTimestamp() to time_utils for fallback timestamp generation - Update malformed detection tests with expect_buffered flag - Rename EmptyChunk test to verify flow-through behavior
- Replace BoundingBoxPx with cv::Rect2f in Detection, eliminating manual static_cast<float> conversions in tracking_worker - Narrow Detection::id from int64_t to int32_t to match rv::tracking::Id (int32_t) - Change pixelToWorld/bboxFootToWorld returns from Point2f to Point2d, removing 8 narrowing static_cast<float> calls in coordinate_transformer since internal math is all double - Change getCameraOrigin return from Point3f to Point3d, returning camera_origin_ directly (already stored as Point3d)
10 tasks
jdanieck
changed the title
15 tasks
- Replaced individual detection transformations with a batch processing method in TrackingWorker::run_tracking. - Updated unit tests for CoordinateTransformer to validate the new batch API. - Added tests for detection ID preservation and handling of missing IDs. - Improved test coverage for various transformation scenarios, including size and position checks. - Ensured that the new implementation maintains performance and correctness with large batches of detections.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📝 Description
Integrates Intel RobotVision's Kalman filter tracking library into the tracker service, replacing the stub tracking implementation. This enables production-quality multi-object tracking with proper state estimation and track lifecycle management.
Key Changes
RobotVision Integration:
rv::tracking::TrackTrackerfor Kalman filter-based multi-object trackingCoordinateTransformerclass for pixel-to-world coordinate transformation (matching Python controller's transform.py)Track ID Format Change:
string(UUID) toint32_t(RobotVision's native format)Configuration:
TrackingConfigstruct:max_unreliable_time_s- time before track becomes reliablenon_measurement_time_dynamic_s- timeout for moving objectsnon_measurement_time_static_s- timeout for static objectsDeveloper Experience:
run/run-debugtargets with proper MQTT TLS settingsTesting:
✨ Type of Change
🧪 Testing Scenarios
✅ Checklist