Anti-UAV Hybrid Tracker : YOLOv5 + SiamFC with GMC

This repository contains an Infrared (IR) based small Unmanned Aerial Vehicle (UAV) tracking pipeline, specifically developed for the CVPR 2025 Anti-UAV Workshop & Challenge (Track 1 : Single UAV Tracking) dataset.

Inspired by the core logic of the paper "A Simple Detector with Frame Dynamics is a Strong Tracker", this project integrates a globl object detector (YOLO) with a local tracker (SiamFC). It maximizes tracking robustness against extremely small drones by applying Optical Flow-based Global Motion Compensation (GMC) and Motion Mask-based Trajectory Constraint Filtering (TC-Filtering).

Key Features

• Hybrid Tracking Pipeline : Invokes YOLOv5 for global detection when the target is initially searched or lost, and utilizes the lightweigth SiamFC for high-speed local tracking during continuous frames.
• Optical Flow-based GMC : Utilizes 'Lucas-Kanade Optical Flow' and the 'RANSAC' algorithm to track background feature points and derive a 2D Affine transformation matrix. This mathematically cancels out severe camera shake (ego-motion).
• Robust TC-Filtering : Extracts purely moving objects by thresholding the difference image between the GMC-warped previous frame and the current frame. It calculates the area ratio of moving pixels within the predicted bounding box to filter out fake targets. (background noise) and seamlessly trigger YOLO re-detection.

Repository Structure

.
├── Baseline_code/                    # SiamFC model weights (model.pth) and core tracker modules
├── FD with GMC/                      # Visualization result of GMC, TC-Filtering, Optical Flow and Frame Difference
      ├── README.md
      ├── Residual_Vis.py             # GMC, FD Visualization code
├── Optical Flow Implementation/      # Implementation of Optical Flow to understand
      ├── opticalflow.py              # Optical Flow code
      ├── tennis_with_optical_flow-2.mp4 
├── video_To_be_Improved/             # Results that should be improved
├── video_good/                       # Good Results
├── README.md
├── [TCF]test_detection_tracking.py   # Main execution script with TC-Filtering (Tested for all cases - AOA : 0.6067)
├── test_detection_tracking           # Main execution script without TC-Filtering

Results

Baseline (YOLO + SiamFC)

Experienced drifting where the tracking box gets stuck on the screen edges by mistaking background noise for the drone, and easily lost the target during severe camera shaking.

Before.mp4

Before.mp4

Before.mp4

Ours (+ GMC & TC-Filtering)

Effectively compensates for severe video shaking and robustly ignores stationary false positives. (e.g., Buildings, Clouds, and Trees) in the background. This approach significantly improved tracking success rates and Intersection over Union (IoU) scores.

After.mp4

After.mp4

After.mp4

After.mp4

Visualization : TC-Filtering based on GMC (Residual Analysis)

This visualization demonstrates the core mechanism of our pipeline : Global Motion Compensation (GMC) combined with Binarization Filtering. It effectively isolates the target even in challenging scenarios with severe camera moving and thermal clutter.

The extracted debug image illustrates the following three stages, from left to right:

1. Origianl (GrayScale) : the raw recurrent frame where the tiny drone target is mixed with complex background noise.
2. Raw Residual (Pre-thresholding) : The absolute difference between the current frame and the affine-warped previous frame. The static background is mostly canceld out, leaving only the physically moving target and strong structural edges.
3. Thresholded Residual (Post-thresholding) : The final result after applying a fixed threhold. Minor background residual noise is masked out, cleanly isolating the target candidates with distinct movement as pure white.

Description : Motion-Compensated Redisual Analysis

• Optical Flow : Tracks robust background feature points between consecutive frames to accurately estimate the camera's ego-motion (panning and tilitng).
• Frame Difference : Utilizes the estimated motion matrix to align (warp) the previous frame to the current frame's perspective, effectively neutralizing camera movement.
• GMC : Suubtracts the GMC-aligned background from the current frame. This eliminates static thermal clutter and isolates only the physically moving target (UAV, drone) as the distinct residual.

References

• CVPR 2025 Anti-UAV Workshop & Challenge
• Paper : "A Simple Detector with Frame Dynamics is a Strong Tracker"
• Codebase adapted from SiamFC and YOLOv5 official repositories.

Appendix

• Datasets for Implementing Optical Flow (Lucas-Kanade)