ID Reassignment in BoT-SORT and ByteTrack Trackers with Webcam Input

[Tixtor710]

I've been experimenting with the BoT-SORT and ByteTrack trackers in Ultralytics YOLO, using my webcam as the video source. I've observed the following behavior:

When I remain within the webcam's field of view, moving around or staying still, the tracker maintains the same ID for me.

However, if I exit the field of view completely and then re-enter, I'm assigned a new ID number.

I've tried this with both the default BoT-SORT and ByteTrack configurations, as well as a modified BoT-SORT configuration. The behavior remains consistent across all setups.

My questions are:

Is this the expected behavior for these trackers?

What factors determine ID assignment and reassignment in these tracking algorithms?

Are there any configuration parameters I can adjust to maintain ID consistency even when an object temporarily leaves the frame?

I'd appreciate any insights into the underlying mechanics of ID assignment in these trackers and potential ways to optimize for consistent ID tracking across brief disappearances from the frame. Thank you!

This is my modified yaml file.

tracker_type: botsort # tracker type, ['botsort', 'bytetrack']
track_high_thresh: 0.25 # threshold for the first association
track_low_thresh: 0.05 # threshold for the second association
new_track_thresh: 1 # threshold for init new track if the detection does not match any tracks
track_buffer: 300 # buffer to calculate the time when to remove tracks
match_thresh: 0.8 # threshold for matching tracks
fuse_score: True # Whether to fuse confidence scores with the iou distances before matching
# min_box_area: 10  # threshold for min box areas(for tracker evaluation, not used for now)

# BoT-SORT settings
gmc_method: sparseOptFlow # method of global motion compensation
# ReID model related thresh (not supported yet)
proximity_thresh: 0.5
appearance_thresh: 0.25
with_reid: False

[UltralyticsAssistant]

👋 Hello @Tixtor710, thank you for sharing your detailed observations and YAML configuration! 🚀 We appreciate your interest in Ultralytics and the BoT-SORT and ByteTrack trackers.

To address your questions and assist further:

• For insights into the underlying mechanics of tracker behavior and ID assignment, we recommend exploring the Docs, particularly the sections on trackers and configuration. These resources provide an excellent starting point.
• For configuration troubleshooting, ensure you are following the Tips for Best Training Results and verify your changes align with tracker-specific parameters.

If this is a 🐛 Bug Report, please share a minimum reproducible example to help us replicate and analyze the issue. This includes code, logs, and a detailed description of your webcam setup and environment.

Upgrade

To ensure you are working with the latest features and fixes, upgrade the ultralytics package along with all requirements in a Python>=3.8 environment with PyTorch>=1.8:

pip install -U ultralytics

Environments

YOLO can run in any of the following environments with dependencies pre-installed:

• Notebooks with free GPU:
• Google Cloud Deep Learning VM: GCP Quickstart Guide
• Amazon Deep Learning AMI: AWS Quickstart Guide
• Docker Image: Docker Quickstart Guide

Community

You can also join the Ultralytics community to discuss tracking algorithms or share your findings:

• Discord 🎧 for real-time chat
• Discourse for in-depth discussions
• Reddit for community threads

An Ultralytics engineer will review your discussion soon to provide further assistance. Let us know if there are any additional details you'd like to share! 😊

[glenn-jocher]

@Tixtor710 this is expected behavior in both trackers. ID reassignment occurs when objects leave the frame due to how track buffers and matching thresholds work. The key parameters to adjust are track_buffer (extends how long tracks are kept without detection) and match_thresh (lower values increase re-identification flexibility). Your modified config already uses an extended 300-frame buffer - you might try slightly lowering match_thresh to 0.6-0.7 to improve re-ID matching after brief absences. The tracking mechanics are explained in our BoT-SORT and ByteTrack documentation references.

[Tixtor710]

Thank you for the detailed explanation and suggestions! I've implemented the adjustments you recommended, including increasing track_buffer to 300 and lowering match_thresh to 0.6. However, I'm still experiencing ID reassignments when objects temporarily leave the frame.

I'd like to delve deeper into the mechanics of re-identification in BoT-SORT and ByteTrack:

ReID Integration: You mentioned ReID integration isn't currently supported. Are there plans to incorporate ReID models into Ultralytics for enhanced re-identification capabilities?

Motion and Velocity Models: How do these trackers handle abrupt motion or occlusions? Are there specific parameters or techniques to improve tracking under such conditions?

Practical Limits of track_buffer: You mentioned potential performance degradation with very high track_buffer values. What are the practical limits for this parameter in terms of frames or seconds, considering typical video frame rates (e.g., 30 FPS)?

Custom Logic for Edge Cases: Are there any examples or guidelines for implementing custom logic to handle specific edge cases, such as brief exits from the frame or sudden changes in object appearance?

Advanced Tuning Strategies: What advanced tuning strategies or parameter combinations have been successful in similar scenarios? Are there any community resources or examples that demonstrate effective use of these trackers in real-world applications?

[glenn-jocher]

@Tixtor710 thanks for following up! Here's a concise breakdown:

1. ReID Integration: While not currently supported in our trackers, we welcome community contributions through GitHub Pull Requests to explore this enhancement.

2. Motion Handling: Both trackers use Kalman filtering for motion prediction. For abrupt movements, consider increasing track_buffer while monitoring performance tradeoffs.

3. track_buffer Limits: At 30 FPS, values up to 300-600 frames (10-20 seconds) are practical for most systems. Monitor memory usage with higher values.

4. Custom Logic: While we don't provide custom implementation examples, you could extend the tracker classes to add position/appearance matching logic for re-identified objects.

5. Advanced Tuning: Focus on balancing track_high_thresh/track_low_thresh pairs and experiment with proximity_thresh in BoT-SORT. Community examples can be found in GitHub Discussions.

For specific implementation details, check our BOTSORT and BYTETracker class references in the docs. Let us know if you need clarification on particular parameters!

[Tixtor710]

tracker_type: botsort  # Tracker type, ['botsort', 'bytetrack']

# Thresholds for association
track_high_thresh: 0.3  # Increased slightly to improve initial association robustness
track_low_thresh: 0.1  # Lower threshold for secondary association remains unchanged
new_track_thresh: 0.25  # Threshold for initializing new tracks

# Track buffer and matching thresholds
track_buffer: 600  # Extended buffer to retain tracks longer (20 seconds at 30 FPS)
match_thresh: 0.65  # Slightly reduced to improve re-identification flexibility

# Score fusion and minimum box area
fuse_score: True  # Fuse confidence scores with IoU distances before matching
# min_box_area: 10  # Uncomment if needed for filtering small detections

# BoT-SORT-specific settings
gmc_method: sparseOptFlow  # Global motion compensation method
proximity_thresh: 0.5  # Increased for better spatial matching of objects
appearance_thresh: 0.3  # Slightly increased to improve appearance-based matching
with_reid: False  # ReID model remains disabled (can be enabled if supported)

# Additional parameters (if needed)
max_age: 70  # Maximum age of a track without updates (optional, adjust as needed)

Despite these adjustments, the issue of ID reassignment persists. When an object leaves the frame and returns after some time, it is often assigned a new ID instead of retaining its original ID.

Follow-Up Question on Tracker Internals
I'm still facing persistent ID reassignment issues with BoT-SORT even after implementing the suggested configuration adjustments.

To better understand how the tracker makes its decisions, could you elaborate on the following:

When an object is detected and tracked, what characteristics or features does the tracker extract and store? Is it similar to generating an embedding of the object's appearance, or are other features like size, shape, or color also considered?

When an object re-enters the frame after being absent, how does the tracker attempt to match it with previously tracked objects? Does it compare the newly extracted features with the stored information from previous frames? If so, what matching algorithms or distance metrics are used?

How do proximity_thresh and appearance_thresh exactly contribute in this re-identification matching?

[glenn-jocher]

Thank you for the follow-up. Let's clarify BoT-SORT's internal mechanics:

1. Feature Storage: BoT-SORT primarily stores motion characteristics (Kalman filter states) and maintains a rolling buffer of appearance features (50-frame history by default) when with_reid=True. These features are normalized embeddings from the object's visual appearance.

2. Re-identification Logic: When objects reappear, matching combines:

   • Motion similarity (IoU distance via Kalman predictions)
   • Optional appearance similarity (cosine distance of ReID embeddings)
   • The proximity_thresh (0.5 in your config) sets the maximum IoU distance for valid matches
   • appearance_thresh (0.3) acts as the cosine similarity threshold when ReID is active

Since you have with_reid=False, matching relies solely on motion prediction and IoU spatial relationships. You might consider reducing proximity_thresh further (e.g., 0.4) to allow more flexible spatial matches for returning objects. The BoT-SORT reference shows how these thresholds interact in the matching process.

[Y-T-G]

It's normal. The ID is based on where the object is every consecutive frame compared to the last frame. It's very loosely based on how the object looks, if at all. Leaving the frame would result in a new ID.

For a consistent ID, you would need at least reidentification if not advanced techniques like facial recognition, both of which makes inference slower.

[Tixtor710]

Title: Model Assigning New IDs Without Proper ReID Similarity Check

Description:
I am experiencing an issue with my YOLO + BoT-SORT tracking pipeline that includes re-identification (ReID). The model is assigning new IDs to objects (persons) even when they appear to be the same individuals based on their visual features. This suggests that the model might not be performing a thorough similarity check against existing embeddings during the ReID process.

Observed Issue:

From the attached video and CSV data:

• A person initially assigned ID_No 1 at 16:30:30 is later given a new ID 15 at 16:31:45, despite being visually the same individual.
• Similar issues occur with other individuals, such as person 17 and person 86, who are assigned new IDs while still in the frame.
• This behavior indicates that the model is not consistently applying the ReID logic to prevent premature ID reassignment.

CSV Sample:

Object_Class, ID_No, In_Time, Out_Time
person, 1, 16:30:30,
person, 2, 16:30:30,
person, 3, 16:30:48,
person, 10, 16:30:55,
person, 11, 16:30:55,
person, 19, 16:31:01,
person, 20, 16:31:04,
person, 1, 16:31:42,
person, 15, 16:31:45,  ⬅️  Appears to be the same person as ID 1
person, 17, 16:31:46,
person, 86, 16:32:01,

Question:

1. Why is the model assigning new IDs before running a proper similarity check against existing embeddings?
2. Are there specific parameters or thresholds (e.g., match_thresh, track_buffer, or appearance_thresh) that need adjustment to ensure consistent application of the ReID logic?
3. Could this behavior be caused by the model failing to persistently store embeddings across frames?

I would appreciate any insights into resolving this premature ID reassignment issue.

---

Script Used:
The script used for this task is attached, which includes YOLO detection and BoT-SORT tracking with ReID. The key functions involve extracting dense vectors from detected objects and matching them against stored embeddings to maintain consistent IDs. However, the current implementation seems to be assigning new IDs without properly checking against existing embeddings in the interim time between checking the similarity, once its checked it assigns the original id as shown in video.

Screen.Recording.2025-03-25.163231.mp4

Code Snippet for ReID Logic:

def match_embedding(dense_vector, threshold=0.8):
    """Matches a dense vector against stored embeddings for ReID."""
    best_match = None
    best_score = float("-inf")

    for obj_id, stored_vector in embedding_store.items():
        similarity = np.dot(dense_vector, stored_vector) / (
            np.linalg.norm(dense_vector) * np.linalg.norm(stored_vector) + 1e-8
        )
        if similarity > best_score and similarity >= threshold:
            best_match = obj_id
            best_score = similarity

    return best_match

[glenn-jocher]

@Tixtor710 thank you for your detailed report. The observed ID reassignment is expected behavior in both trackers when objects leave the frame, as they maintain tracks through a buffer period. For BoT-SORT specifically, while our implementation currently doesn't fully support ReID (the with_reid flag is experimental), you can try adjusting these parameters:

1. Increase track_buffer to extend how long disappeared tracks are retained
2. Lower appearance_thresh for stricter ReID matching (though note our ReID implementation is limited)
3. Ensure proximity_thresh and match_thresh are properly balanced for your use case

The current architecture doesn't maintain long-term embeddings between full disappearances. For persistent tracking across long occlusions, you'd need to implement a custom ReID system with external memory beyond the trackers' native capabilities. We're actively working on improving ReID integration in future updates.

[Tixtor710]

"Thank you for the clarification. I understand the BoT-SORT ReID implementation is experimental, and I appreciate the parameter suggestions. My script uses an embedding_store and cosine similarity for ReID. However, I'm encountering an issue where objects, upon re-entry into the frame, are initially assigned a new ID before being corrected to their original ID after a brief period. This creates a temporary ID "swap" that I'd like to eliminate.

Given my existing ReID setup, how can I ensure BoT-SORT prioritizes a faster and more consistent re-identification process upon an object's reappearance? Specifically:

Parameter Influence on Re-entry: Could you elaborate on how parameters like track_buffer, appearance_thresh, proximity_thresh, and match_thresh specifically influence ID assignment when a previously tracked object re-enters the scene? What values would encourage immediate ReID and prevent the brief assignment of a new, incorrect ID?

Early ReID Integration: Is there a specific point in the BoT-SORT code I could modify to force an immediate check against my embedding_store upon a new detection to minimize the delay in re-assigning the correct, pre-existing ID?

Embedding Handling: Is there anything about how BoT-SORT handles embeddings of temporarily disappeared objects that might interfere with my match_embedding function? In other words, am I potentially fighting against some internal logic that is causing the temporary ID swap?"

@glenn-jocher

[glenn-jocher]

@Tixtor710 this is expected behavior due to how trackers manage 'lost' states. To improve ID consistency:

1. Parameters: Increase track_buffer (currently 300) to extend how long a track is retained after disappearance. Lower appearance_thresh (currently 0.25) to make ReID matching more lenient. These adjustments help the tracker wait longer and accept weaker ReID matches before creating new IDs.

2. Code Modification: Implement your ReID matching earlier in the track reactivation logic by modifying the re_activate() method in the BOTrack class (see bot_sort.py reference). Compare new detections against your embedding_store before standard matching occurs.

3. Embedding Handling: BoT-SORT maintains a feature history deque (default 50 frames). Ensure your match_embedding function compares against all historical features in tracks, not just the last seen embedding, to account for appearance changes during absence.

For immediate testing, try these parameter adjustments:

track_buffer: 90  # ~3sec at 30FPS
appearance_thresh: 0.2
proximity_thresh: 0.7
match_thresh: 0.7

This configuration prioritizes ReID matches while allowing reasonable disappearance time. Monitor performance with your specific use case.

[Tixtor710]

Follow-Up Clarification

Thank you for the detailed suggestions. I've implemented your recommended parameters and modified the re_activate() method, but I'm still observing temporary ID swaps during object re-entry. Let me clarify my current setup and remaining issue:

---

Observed Behavior

CSV Log Example:

person,1,22:07:41,  # Original ID assignment  
person,1,22:07:42,  
person,1,22:07:43,  
person,61,22:07:43,  # Temporary new ID  
person,62,22:07:43,  # Temporary new ID  
person,63,22:07:48,  # Temporary new ID  
person,1,22:07:49,   # Correct re-identification  

The object disappears/re-enters and receives 3 temporary IDs before reverting to ID 1.

---

Current Implementation

Parameters:

track_buffer = 90       # ~3s buffer at 30 FPS  
appearance_thresh = 0.2  
proximity_thresh = 0.7  
match_thresh = 0.7  

ReID Logic Flow:

1. Feature Extraction (extract_dense_vector):

def extract_dense_vector(frame, bbox, model):  
    # Crops object region, resizes to 640x640  
    # Uses YOLO to extract 4-element dense vector (xyxy coordinates)  
    return np.array([x1, y1, x2, y2])  # Simplified representation  

2. Embedding Matching (match_embedding):

python
def match_embedding(dense_vector, threshold=0.8):  
    # Compares using cosine similarity  
    # Returns matching ID if similarity >= 0.8  

---

Specific Questions

Parameter Conflict:

• Could the coordinate-based dense vectors (xyxy) be less effective for ReID than appearance-based features?
• Is there a recommended minimum track_buffer value that would prevent new ID assignment during brief disappearances?

Confidence Thresholds:

• With appearance_thresh=0.2, could this allow false matches that interfere with correct re-identification?
• Should I adjust the similarity threshold in my match_embedding function (0.8) to align with BoT-SORT's thresholds?

@glenn-jocher

[glenn-jocher]

@Tixtor710 this behavior occurs because coordinate-based features (xyxy) lack persistent appearance information needed for reliable ReID. For improved consistency:

1. Feature Enhancement: Replace coordinate vectors with appearance features using our ReID models (see Ultralytics HUB integration examples). Bounding box positions alone can't maintain identity across exits.

2. Threshold Alignment: Your embedding threshold (0.8) conflicts with appearance_thresh=0.2. For coordinate features, set appearance_thresh=0.8 to match your similarity threshold.

3. Buffer Adjustment: With 30 FPS, track_buffer=90 gives 3s grace period. Increase to 150-300 (5-10s) if objects regularly exit longer than 3s.

For optimal results, combine these adjustments with proper appearance feature extraction through our built-in ReID support.

[Tixtor710]

Hi @glenn-jocher and the Ultralytics team, thanks for the feedback on coordinate-based ReID. I understand that appearance-based features are crucial for reliable re-identification. Following up on my earlier question about accessing intermediate layers in YOLO12n for Centroid ReID:

Is there a recommended or supported way to extract appearance features directly from YOLO12n's intermediate layers to improve ReID performance? If intermediate layer access isn't feasible, are there alternative methods within the Ultralytics framework for generating robust appearance embeddings from YOLO12n detections that can then be integrated with a Centroid ReID pipeline, as you mentioned in your reply?

[glenn-jocher]

@Tixtor710 hi! For ReID features with YOLO models, we recommend using YOLO11 instead of YOLO12 as it's more stable and performant. While direct intermediate layer access isn't supported, you can use YOLO11's built-in ReID capabilities by enabling with_reid=True in your BoT-SORT config and providing a ReID model. The Ultralytics framework allows integration of custom ReID models through the tracker configuration - check out our tracking documentation for implementation details and example configs.

For ReID model options, you might explore our pre-trained models on Ultralytics HUB or train a custom ReID model using our classification training tools.

[Tixtor710]

I'm working on a YOLO-based model to detect surgical instruments during surgery, where the tools are often overlapping, clustered, and partially occluded. My goal is to not only detect all present instruments but also retain consistent detection throughout the procedure.

I'm currently training on annotated surgical videos/images, but I'm noticing the model sometimes misses tools when they overlap or when motion blur/occlusion occurs.

My main questions are:

Will training the model for more epochs (with the same dataset) significantly help in learning to detect instruments in such cluttered scenes?

How much does performance in these kinds of dense environments depend on more training vs better/more diverse training data (e.g., more occlusion/angle variation)?

Are there any best practices or YOLO-specific tips (augmentations, anchors, model size, etc.) that help improve performance in these scenarios?

Any advice or insights from people who have worked on similar medical/industrial use cases would be greatly appreciated!

Thanks in advance.

[glenn-jocher]

@Tixtor710 Your question about surgical instrument detection is unrelated to the tracker discussion, but I'm happy to help.

For detecting overlapping surgical instruments:

1. Additional epochs alone won't solve occlusion/overlap issues. After initial convergence, returns diminish significantly without new data variety.

2. Data quality and variety are far more important than simply training longer. Focus on collecting examples with different lighting conditions, angles, occlusion patterns, and motion blur scenarios.

3. YOLO-specific recommendations:

   • Use mosaic/mixup augmentations to simulate occlusion
   • Try a larger model (YOLO11m/l) for better feature extraction
   • Consider instance segmentation (YOLO11-seg) instead of detection for overlapping instruments
   • Implement test-time augmentation during inference for challenging cases

For tracking consistency specifically, you might want to combine detection with optical flow or specialized surgical instrument tracking approaches. Consider posting your tracking question as a separate discussion for more focused feedback.

[KiraTcheishvili]

hello
i am using Yolo11n model to detect people from frames which is takes in every 5 seconds from video. my problem is that when yolo names one person ID 1 for example, new person who came later in the frame also become ID 1, so yolo reuse the ID number. based on that according to yolo i have for example 5 person in the frames withing 5 minutes and not 15 (which is real case), so i can not calculate average wait time of people. does yolo have anything to solve this problem?

thank you!

[glenn-jocher]

Yes, that’s expected for both ByteTrack and BoT-SORT: IDs are only guaranteed to be consistent while a track stays “alive”, and once a person fully leaves the view the track typically becomes Lost then gets Removed after track_buffer frames, so a re-entry is treated as a new track (new ID); matching is primarily IoU/motion-based (track_high_thresh, track_low_thresh, match_thresh, fuse_score), and track_buffer only controls how long a lost track is kept around, not true re-identification across exits (see the tracking mode docs and the default configs like BoT-SORT tracker YAML and ByteTrack tracker YAML); also note your new_track_thresh: 1 is effectively “never start new tracks” unless confidence is 1.0, so I’d revert that closer to the defaults.

For the “frame every 5 seconds” case, ID reuse usually happens because the tracker is being re-initialized each time (or not run with persistence), so keep one YOLO instance and call track() with persist=True on each new frame (trackers assume consecutive frames, long gaps will increase ID switches no matter what):

import cv2
from ultralytics import YOLO

model = YOLO("yolo26n.pt")  # or your YOLO11n model
cap = cv2.VideoCapture("video.mp4")

while True:
    ok, frame = cap.read()
    if not ok:
        break
    results = model.track(frame, persist=True, tracker="bytetrack.yaml")  # or "botsort.yaml"
    # use results[0].boxes.id here