Trackers 2.2.0 release #247
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Remove DeepSORT from trackers
# Conflicts: # docs/trackers/core/deepsort/tracker.md # docs/trackers/core/reid/reid.md
Feat/update examples with rf detr
Bumps [torch](https://github.com/pytorch/pytorch) from 2.6.0+cu124 to 2.8.0. - [Release notes](https://github.com/pytorch/pytorch/releases) - [Changelog](https://github.com/pytorch/pytorch/blob/main/RELEASE.md) - [Commits](https://github.com/pytorch/pytorch/commits/v2.8.0) --- updated-dependencies: - dependency-name: torch dependency-version: 2.8.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 3.1.3 to 3.1.5. - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@3.1.3...3.1.5) --- updated-dependencies: - dependency-name: werkzeug dependency-version: 3.1.5 dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [protobuf](https://github.com/protocolbuffers/protobuf) from 6.30.2 to 6.31.1. - [Release notes](https://github.com/protocolbuffers/protobuf/releases) - [Commits](protocolbuffers/protobuf@v6.30.2...v6.31.1) --- updated-dependencies: - dependency-name: protobuf dependency-version: 6.31.1 dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Added bytetrack without REID * Added lines to install bytetrack and keyword * added docs and type traacker type to sort_utils for using get_iou * fixed spelling mistakes and improved docs * improved doc overview based on Tomasz feedback and fixed pre-commit * added bytetrack to index in mkdocs * renamed high_conf_boxes_threshold to high_conf_det_threshold
--- updated-dependencies: - dependency-name: aiohttp dependency-version: 3.13.3 dependency-type: direct:production dependency-group: uv - dependency-name: uv dependency-version: 0.9.6 dependency-type: direct:development dependency-group: uv - dependency-name: filelock dependency-version: 3.20.3 dependency-type: indirect dependency-group: uv - dependency-name: fonttools dependency-version: 4.60.2 dependency-type: indirect dependency-group: uv - dependency-name: jupyterlab dependency-version: 4.4.8 dependency-type: indirect dependency-group: uv - dependency-name: pillow dependency-version: 11.3.0 dependency-type: indirect dependency-group: uv - dependency-name: pymdown-extensions dependency-version: 10.16.1 dependency-type: indirect dependency-group: uv - dependency-name: requests dependency-version: 2.32.4 dependency-type: indirect dependency-group: uv - dependency-name: tornado dependency-version: '6.5' dependency-type: indirect dependency-group: uv - dependency-name: urllib3 dependency-version: 2.6.3 dependency-type: indirect dependency-group: uv - dependency-name: virtualenv dependency-version: 20.36.1 dependency-type: indirect dependency-group: uv ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the uv group with 1 update in the / directory: [wheel](https://github.com/pypa/wheel). Updates `wheel` from 0.45.1 to 0.46.2 - [Release notes](https://github.com/pypa/wheel/releases) - [Changelog](https://github.com/pypa/wheel/blob/main/docs/news.rst) - [Commits](pypa/wheel@0.45.1...0.46.2) --- updated-dependencies: - dependency-name: wheel dependency-version: 0.46.2 dependency-type: direct:development dependency-group: uv ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* ⬆️ Bump pypa/gh-action-pypi-publish from 1.12.4 to 1.13.0 Bumps [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) from 1.12.4 to 1.13.0. - [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases) - [Commits](pypa/gh-action-pypi-publish@76f52bc...ed0c539) --- updated-dependencies: - dependency-name: pypa/gh-action-pypi-publish dependency-version: 1.13.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * chore(pre_commit): ⬆ pre_commit autoupdate updates: - [github.com/pre-commit/pre-commit-hooks: v5.0.0 → v6.0.0](pre-commit/pre-commit-hooks@v5.0.0...v6.0.0) - [github.com/PyCQA/bandit: 1.8.6 → 1.9.3](PyCQA/bandit@1.8.6...1.9.3) - [github.com/astral-sh/ruff-pre-commit: v0.12.2 → v0.14.13](astral-sh/ruff-pre-commit@v0.12.2...v0.14.13) - [github.com/pre-commit/mirrors-mypy: v1.16.1 → v1.19.1](pre-commit/mirrors-mypy@v1.16.1...v1.19.1) * Update .pre-commit-config.yaml --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Skalski <[email protected]> Co-authored-by: Jirka Borovec <[email protected]>
Bumps [torchvision](https://github.com/pytorch/vision) from 0.21.0+cu124 to 0.23.0. - [Release notes](https://github.com/pytorch/vision/releases) - [Commits](https://github.com/pytorch/vision/commits/0.23.0) --- updated-dependencies: - dependency-name: torchvision dependency-version: 0.23.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [torch](https://github.com/pytorch/pytorch) from 2.6.0+cu124 to 2.8.0. - [Release notes](https://github.com/pytorch/pytorch/releases) - [Changelog](https://github.com/pytorch/pytorch/blob/main/RELEASE.md) - [Commits](https://github.com/pytorch/pytorch/commits/v2.8.0) --- updated-dependencies: - dependency-name: torch dependency-version: 2.8.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the github-actions group with 3 updates: [actions/checkout](https://github.com/actions/checkout), [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) and [actions/create-github-app-token](https://github.com/actions/create-github-app-token). Updates `actions/checkout` from 4.2.2 to 6.0.2 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@11bd719...de0fac2) Updates `astral-sh/setup-uv` from 6.3.1 to 7.2.0 - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](astral-sh/setup-uv@bd01e18...61cb8a9) Updates `actions/create-github-app-token` from 2.0.6 to 2.2.1 - [Release notes](https://github.com/actions/create-github-app-token/releases) - [Commits](actions/create-github-app-token@df432ce...29824e6) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: 6.0.2 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions - dependency-name: astral-sh/setup-uv dependency-version: 7.2.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions - dependency-name: actions/create-github-app-token dependency-version: 2.2.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [numpy](https://github.com/numpy/numpy) from 2.2.5 to 2.2.6. - [Release notes](https://github.com/numpy/numpy/releases) - [Changelog](https://github.com/numpy/numpy/blob/main/doc/RELEASE_WALKTHROUGH.rst) - [Commits](numpy/numpy@v2.2.5...v2.2.6) --- updated-dependencies: - dependency-name: numpy dependency-version: 2.2.6 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Enhance publish-docs workflow to allow custom ref deployments via workflow_dispatch * Update .github/workflows/publish-docs.yml --------- Co-authored-by: Copilot <[email protected]>
Bumps the github-actions group with 1 update: [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv). Updates `astral-sh/setup-uv` from 7.2.0 to 7.2.1 - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](astral-sh/setup-uv@61cb8a9...803947b) --- updated-dependencies: - dependency-name: astral-sh/setup-uv dependency-version: 7.2.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [twine](https://github.com/pypa/twine) from 6.1.0 to 6.2.0. - [Release notes](https://github.com/pypa/twine/releases) - [Changelog](https://github.com/pypa/twine/blob/main/docs/changelog.rst) - [Commits](pypa/twine@6.1.0...6.2.0) --- updated-dependencies: - dependency-name: twine dependency-version: 6.2.0 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.6.13 to 9.7.1. - [Release notes](https://github.com/squidfunk/mkdocs-material/releases) - [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG) - [Commits](squidfunk/mkdocs-material@9.6.13...9.7.1) --- updated-dependencies: - dependency-name: mkdocs-material dependency-version: 9.7.1 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 4.2.0 to 4.5.1. - [Release notes](https://github.com/pre-commit/pre-commit/releases) - [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md) - [Commits](pre-commit/pre-commit@v4.2.0...v4.5.1) --- updated-dependencies: - dependency-name: pre-commit dependency-version: 4.5.1 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [build](https://github.com/pypa/build) from 1.2.2.post1 to 1.4.0. - [Release notes](https://github.com/pypa/build/releases) - [Changelog](https://github.com/pypa/build/blob/main/CHANGELOG.rst) - [Commits](pypa/build@1.2.2.post1...1.4.0) --- updated-dependencies: - dependency-name: build dependency-version: 1.4.0 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [mkdocs-glightbox](https://github.com/blueswen/mkdocs-glightbox) from 0.4.0 to 0.5.2. - [Release notes](https://github.com/blueswen/mkdocs-glightbox/releases) - [Changelog](https://github.com/blueswen/mkdocs-glightbox/blob/main/CHANGELOG) - [Commits](blueswen/mkdocs-glightbox@v0.4.0...v0.5.2) --- updated-dependencies: - dependency-name: mkdocs-glightbox dependency-version: 0.5.2 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
#210) * Add box IoU/IoA calculation module from TrackEval - Add box_iou() for Intersection over Union calculation - Add box_ioa() for Intersection over Area calculation - Support both xyxy and xywh box formats - Include 32 unit tests covering edge cases and floating point precision - Adapted from TrackEval with MIT license attribution * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix ruff and mypy linting errors - Add S101 ignore for test files in pyproject.toml - Split error test cases into separate test functions - Remove type: ignore comments by simplifying type hints - Clean up unused imports --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…rt 2) (#211) * Add box IoU/IoA calculation module from TrackEval - Add box_iou() for Intersection over Union calculation - Add box_ioa() for Intersection over Area calculation - Support both xyxy and xywh box formats - Include 32 unit tests covering edge cases and floating point precision - Adapted from TrackEval with MIT license attribution * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix ruff and mypy linting errors - Add S101 ignore for test files in pyproject.toml - Split error test cases into separate test functions - Remove type: ignore comments by simplifying type hints - Clean up unused imports * Add Hungarian matching utility for detection assignment - Add match_detections() using Jonker-Volgenant algorithm (scipy) - Maximize similarity with threshold filtering - Return matched/unmatched indices for GT and tracker detections - Include 17 unit tests covering edge cases and scipy consistency - Adapted from TrackEval matching patterns * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix unused variable warnings in tests * Fix remaining unused variable warning * Add diverse test cases for match_detections - Off-diagonal matches (GT0->TR2, GT1->TR0, etc.) - Swapped/reversed matches - Sparse non-sequential matches - Very unbalanced sizes (1 GT vs 5 trackers, 5 GTs vs 1 tracker) - Only middle/corner elements matching - Threshold edge cases - Optimal vs greedy assignment tests * Remove redundant tests from test_matching.py --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
… 3) (#212) * Add box IoU/IoA calculation module from TrackEval - Add box_iou() for Intersection over Union calculation - Add box_ioa() for Intersection over Area calculation - Support both xyxy and xywh box formats - Include 32 unit tests covering edge cases and floating point precision - Adapted from TrackEval with MIT license attribution * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix ruff and mypy linting errors - Add S101 ignore for test files in pyproject.toml - Split error test cases into separate test functions - Remove type: ignore comments by simplifying type hints - Clean up unused imports * Add Hungarian matching utility for detection assignment - Add match_detections() using Jonker-Volgenant algorithm (scipy) - Maximize similarity with threshold filtering - Return matched/unmatched indices for GT and tracker detections - Include 17 unit tests covering edge cases and scipy consistency - Adapted from TrackEval matching patterns * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix unused variable warnings in tests * Fix remaining unused variable warning * Add diverse test cases for match_detections - Off-diagonal matches (GT0->TR2, GT1->TR0, etc.) - Swapped/reversed matches - Sparse non-sequential matches - Very unbalanced sizes (1 GT vs 5 trackers, 5 GTs vs 1 tracker) - Only middle/corner elements matching - Threshold edge cases - Optimal vs greedy assignment tests * Remove redundant tests from test_matching.py * Add CLEAR metrics module for multi-object tracking evaluation Implement compute_clear_metrics function adapted from TrackEval with exact numerical parity. The function computes standard CLEAR metrics including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML). Key implementation details: - Score matrix construction with IDSW prioritization (1000x bonus) - Hungarian matching with proper threshold filtering - ID switch detection based on previous tracker associations - MT/PT/ML thresholds (>80%, >=20%, <20%) - Fragmentation counting for track interruptions - Vectorized GT ID mapping using np.searchsorted Tests are fully parametrized and only test the public API. Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method) * fix(pre_commit): 🎨 auto format pre-commit hooks * Add CLEAR metrics module for multi-object tracking evaluation Implement compute_clear_metrics function adapted from TrackEval with exact numerical parity. The function computes standard CLEAR metrics including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML). Key implementation details: - Score matrix construction with IDSW prioritization (1000x bonus) - Hungarian matching with proper threshold filtering - ID switch detection based on previous tracker associations - MT/PT/ML thresholds (>80%, >=20%, <20%) - Fragmentation counting for track interruptions - Vectorized GT ID mapping using np.searchsorted Tests are fully parametrized and only test the public API. Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method) * fix(pre_commit): 🎨 auto format pre-commit hooks * fix(pre_commit): 🎨 auto format pre-commit hooks --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…TrackEval Part 4) (#214) * Add box IoU/IoA calculation module from TrackEval - Add box_iou() for Intersection over Union calculation - Add box_ioa() for Intersection over Area calculation - Support both xyxy and xywh box formats - Include 32 unit tests covering edge cases and floating point precision - Adapted from TrackEval with MIT license attribution * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix ruff and mypy linting errors - Add S101 ignore for test files in pyproject.toml - Split error test cases into separate test functions - Remove type: ignore comments by simplifying type hints - Clean up unused imports * Add Hungarian matching utility for detection assignment - Add match_detections() using Jonker-Volgenant algorithm (scipy) - Maximize similarity with threshold filtering - Return matched/unmatched indices for GT and tracker detections - Include 17 unit tests covering edge cases and scipy consistency - Adapted from TrackEval matching patterns * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix unused variable warnings in tests * Fix remaining unused variable warning * Add diverse test cases for match_detections - Off-diagonal matches (GT0->TR2, GT1->TR0, etc.) - Swapped/reversed matches - Sparse non-sequential matches - Very unbalanced sizes (1 GT vs 5 trackers, 5 GTs vs 1 tracker) - Only middle/corner elements matching - Threshold edge cases - Optimal vs greedy assignment tests * Remove redundant tests from test_matching.py * Add CLEAR metrics module for multi-object tracking evaluation Implement compute_clear_metrics function adapted from TrackEval with exact numerical parity. The function computes standard CLEAR metrics including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML). Key implementation details: - Score matrix construction with IDSW prioritization (1000x bonus) - Hungarian matching with proper threshold filtering - ID switch detection based on previous tracker associations - MT/PT/ML thresholds (>80%, >=20%, <20%) - Fragmentation counting for track interruptions - Vectorized GT ID mapping using np.searchsorted Tests are fully parametrized and only test the public API. Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method) * fix(pre_commit): 🎨 auto format pre-commit hooks * Add CLEAR metrics module for multi-object tracking evaluation Implement compute_clear_metrics function adapted from TrackEval with exact numerical parity. The function computes standard CLEAR metrics including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML). Key implementation details: - Score matrix construction with IDSW prioritization (1000x bonus) - Hungarian matching with proper threshold filtering - ID switch detection based on previous tracker associations - MT/PT/ML thresholds (>80%, >=20%, <20%) - Fragmentation counting for track interruptions - Vectorized GT ID mapping using np.searchsorted Tests are fully parametrized and only test the public API. Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method) * fix(pre_commit): 🎨 auto format pre-commit hooks * Add MOT format file loading and sequence preparation for E2E evaluation - Add MOTFrameData dataclass for per-frame detection data - Add MOTSequenceData dataclass for prepared sequence data ready for metrics - Add load_mot_file() to parse MOT Challenge format files - Add prepare_mot_sequence() to compute IoU and remap IDs for evaluation - Update __init__.py to export new types and functions - Add comprehensive unit tests (20 tests for io module) * fix(pre_commit): 🎨 auto format pre-commit hooks * Add integration tests for CLEAR metrics validation against TrackEval - Add test/conftest.py with fixture to download and cache SoccerNet test data - Add test/eval/test_integration.py with 49 parametrized tests for all sequences - Add ci-integration-tests.yml workflow that runs only on eval code changes - Update ci-tests.yml to exclude integration tests from regular CI - Add integration marker to pyproject.toml pytest config All 49 sequences pass with exact numerical parity to TrackEval for: - Integer metrics: CLR_TP, CLR_FN, CLR_FP, IDSW, MT, PT, ML, Frag - Float metrics: MOTA, MOTP, MTR, PTR, MLR * Refactor integration tests and improve code quality - Rename CI job to 'TrackEval Parity Validation' for clarity - Derive sequence names dynamically from expected_results.json - Simplify test_io.py from 21 to 10 test cases while maintaining coverage - Restore useful comments in io.py around ID remapping and IoU computation - Clean up conftest.py and test_integration.py docstrings * fix(pre_commit): 🎨 auto format pre-commit hooks * Update integration tests to use SportsMOT and DanceTrack datasets (#221) * Update integration tests to use SportsMOT and DanceTrack data Replace SoccerNet test data with SportsMOT (45 sequences) and DanceTrack (25 sequences) datasets. The tests now validate CLEAR metrics against TrackEval results for 70 total sequences. - Update URLs to new GCS-hosted test data zips - Add multi-dataset fixture support in conftest.py - Parametrize tests across both datasets - Fix metric comparison (new format uses fractions, not percentages) * Fix mypy conftest module name conflict Use a pytest fixture instead of directly importing from conftest.py to avoid mypy seeing the file under two module names ("conftest" and "test.conftest"). * Add type annotation for test_cases variable --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add box IoU/IoA calculation module from TrackEval - Add box_iou() for Intersection over Union calculation - Add box_ioa() for Intersection over Area calculation - Support both xyxy and xywh box formats - Include 32 unit tests covering edge cases and floating point precision - Adapted from TrackEval with MIT license attribution * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix ruff and mypy linting errors - Add S101 ignore for test files in pyproject.toml - Split error test cases into separate test functions - Remove type: ignore comments by simplifying type hints - Clean up unused imports * Add Hungarian matching utility for detection assignment - Add match_detections() using Jonker-Volgenant algorithm (scipy) - Maximize similarity with threshold filtering - Return matched/unmatched indices for GT and tracker detections - Include 17 unit tests covering edge cases and scipy consistency - Adapted from TrackEval matching patterns * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix unused variable warnings in tests * Fix remaining unused variable warning * Add diverse test cases for match_detections - Off-diagonal matches (GT0->TR2, GT1->TR0, etc.) - Swapped/reversed matches - Sparse non-sequential matches - Very unbalanced sizes (1 GT vs 5 trackers, 5 GTs vs 1 tracker) - Only middle/corner elements matching - Threshold edge cases - Optimal vs greedy assignment tests * Remove redundant tests from test_matching.py * Add CLEAR metrics module for multi-object tracking evaluation Implement compute_clear_metrics function adapted from TrackEval with exact numerical parity. The function computes standard CLEAR metrics including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML). Key implementation details: - Score matrix construction with IDSW prioritization (1000x bonus) - Hungarian matching with proper threshold filtering - ID switch detection based on previous tracker associations - MT/PT/ML thresholds (>80%, >=20%, <20%) - Fragmentation counting for track interruptions - Vectorized GT ID mapping using np.searchsorted Tests are fully parametrized and only test the public API. Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method) * fix(pre_commit): 🎨 auto format pre-commit hooks * Add CLEAR metrics module for multi-object tracking evaluation Implement compute_clear_metrics function adapted from TrackEval with exact numerical parity. The function computes standard CLEAR metrics including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML). Key implementation details: - Score matrix construction with IDSW prioritization (1000x bonus) - Hungarian matching with proper threshold filtering - ID switch detection based on previous tracker associations - MT/PT/ML thresholds (>80%, >=20%, <20%) - Fragmentation counting for track interruptions - Vectorized GT ID mapping using np.searchsorted Tests are fully parametrized and only test the public API. Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method) * fix(pre_commit): 🎨 auto format pre-commit hooks * Add MOT format file loading and sequence preparation for E2E evaluation - Add MOTFrameData dataclass for per-frame detection data - Add MOTSequenceData dataclass for prepared sequence data ready for metrics - Add load_mot_file() to parse MOT Challenge format files - Add prepare_mot_sequence() to compute IoU and remap IDs for evaluation - Update __init__.py to export new types and functions - Add comprehensive unit tests (20 tests for io module) * fix(pre_commit): 🎨 auto format pre-commit hooks * Add integration tests for CLEAR metrics validation against TrackEval - Add test/conftest.py with fixture to download and cache SoccerNet test data - Add test/eval/test_integration.py with 49 parametrized tests for all sequences - Add ci-integration-tests.yml workflow that runs only on eval code changes - Update ci-tests.yml to exclude integration tests from regular CI - Add integration marker to pyproject.toml pytest config All 49 sequences pass with exact numerical parity to TrackEval for: - Integer metrics: CLR_TP, CLR_FN, CLR_FP, IDSW, MT, PT, ML, Frag - Float metrics: MOTA, MOTP, MTR, PTR, MLR * Refactor integration tests and improve code quality - Rename CI job to 'TrackEval Parity Validation' for clarity - Derive sequence names dynamically from expected_results.json - Simplify test_io.py from 21 to 10 test cases while maintaining coverage - Restore useful comments in io.py around ID remapping and IoU computation - Clean up conftest.py and test_integration.py docstrings * fix(pre_commit): 🎨 auto format pre-commit hooks * Add high-level evaluation API and CLI for tracker evaluation Introduce SDK functions and CLI for evaluating multi-object tracking results against ground truth using MOT Challenge format data. SDK: - evaluate_mot_sequence(): Evaluate single sequence, returns CLEAR metrics - evaluate_benchmark(): Evaluate multiple sequences with aggregation CLI (beta): - `trackers eval --gt <file> --tracker <file>` for single sequence - `trackers eval --gt-dir <dir> --tracker-dir <dir>` for benchmarks - Optional rich output with `pip install trackers[cli]` - Beta warning displayed on CLI usage * fix(pre_commit): 🎨 auto format pre-commit hooks * Add high-level evaluation API and CLI for tracker evaluation Introduce SDK functions and CLI for evaluating multi-object tracking results against ground truth using MOT Challenge format data. SDK: - evaluate_mot_sequence(): Evaluate single sequence, returns CLEAR metrics - evaluate_benchmark(): Evaluate multiple sequences with aggregation CLI (beta): - `trackers eval --gt <file> --tracker <file>` for single sequence - `trackers eval --gt-dir <dir> --tracker-dir <dir>` for benchmarks - Optional rich output with `pip install trackers[cli]` - Beta warning displayed on CLI usage * fix(pre_commit): 🎨 auto format pre-commit hooks * Update integration tests to use SportsMOT and DanceTrack datasets (#221) * Update integration tests to use SportsMOT and DanceTrack data Replace SoccerNet test data with SportsMOT (45 sequences) and DanceTrack (25 sequences) datasets. The tests now validate CLEAR metrics against TrackEval results for 70 total sequences. - Update URLs to new GCS-hosted test data zips - Add multi-dataset fixture support in conftest.py - Parametrize tests across both datasets - Fix metric comparison (new format uses fractions, not percentages) * Fix mypy conftest module name conflict Use a pytest fixture instead of directly importing from conftest.py to avoid mypy seeing the file under two module names ("conftest" and "test.conftest"). * Add type annotation for test_cases variable * Add smart auto-detection and structured result objects to evaluation API (#222) * Add smart auto-detection and structured result objects to evaluation API - Remove data_format parameter from SDK and --data-format flag from CLI - Add smart detection: auto-detect format (flat vs MOT17), benchmark, split, and tracker name - Log detection results for transparency; error helpfully when ambiguous - Add BenchmarkResult and SequenceResult dataclasses with json/table/save/load methods - Add all CLEAR metrics (MOTA, MOTP, MODA, CLR_Re, CLR_Pr, MTR, PTR, MLR, sMOTA, etc.) - Simplify test fixtures to use auto-detection instead of explicit format parameters * fix(pre_commit): 🎨 auto format pre-commit hooks * Simplify CLI table output to use SDK result.table() method - Remove custom Rich table formatting that truncated columns - Use result.table(columns) for consistent, readable output - Set sensible default columns: MOTA, MOTP, IDSW, CLR_FP, CLR_FN, MT, ML - Remove unused Rich imports and helper functions --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…223) * Add box IoU/IoA calculation module from TrackEval - Add box_iou() for Intersection over Union calculation - Add box_ioa() for Intersection over Area calculation - Support both xyxy and xywh box formats - Include 32 unit tests covering edge cases and floating point precision - Adapted from TrackEval with MIT license attribution * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix ruff and mypy linting errors - Add S101 ignore for test files in pyproject.toml - Split error test cases into separate test functions - Remove type: ignore comments by simplifying type hints - Clean up unused imports * Add Hungarian matching utility for detection assignment - Add match_detections() using Jonker-Volgenant algorithm (scipy) - Maximize similarity with threshold filtering - Return matched/unmatched indices for GT and tracker detections - Include 17 unit tests covering edge cases and scipy consistency - Adapted from TrackEval matching patterns * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix unused variable warnings in tests * Fix remaining unused variable warning * Add diverse test cases for match_detections - Off-diagonal matches (GT0->TR2, GT1->TR0, etc.) - Swapped/reversed matches - Sparse non-sequential matches - Very unbalanced sizes (1 GT vs 5 trackers, 5 GTs vs 1 tracker) - Only middle/corner elements matching - Threshold edge cases - Optimal vs greedy assignment tests * Remove redundant tests from test_matching.py * Add CLEAR metrics module for multi-object tracking evaluation Implement compute_clear_metrics function adapted from TrackEval with exact numerical parity. The function computes standard CLEAR metrics including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML). Key implementation details: - Score matrix construction with IDSW prioritization (1000x bonus) - Hungarian matching with proper threshold filtering - ID switch detection based on previous tracker associations - MT/PT/ML thresholds (>80%, >=20%, <20%) - Fragmentation counting for track interruptions - Vectorized GT ID mapping using np.searchsorted Tests are fully parametrized and only test the public API. Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method) * fix(pre_commit): 🎨 auto format pre-commit hooks * Add CLEAR metrics module for multi-object tracking evaluation Implement compute_clear_metrics function adapted from TrackEval with exact numerical parity. The function computes standard CLEAR metrics including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML). Key implementation details: - Score matrix construction with IDSW prioritization (1000x bonus) - Hungarian matching with proper threshold filtering - ID switch detection based on previous tracker associations - MT/PT/ML thresholds (>80%, >=20%, <20%) - Fragmentation counting for track interruptions - Vectorized GT ID mapping using np.searchsorted Tests are fully parametrized and only test the public API. Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method) * fix(pre_commit): 🎨 auto format pre-commit hooks * Add MOT format file loading and sequence preparation for E2E evaluation - Add MOTFrameData dataclass for per-frame detection data - Add MOTSequenceData dataclass for prepared sequence data ready for metrics - Add load_mot_file() to parse MOT Challenge format files - Add prepare_mot_sequence() to compute IoU and remap IDs for evaluation - Update __init__.py to export new types and functions - Add comprehensive unit tests (20 tests for io module) * fix(pre_commit): 🎨 auto format pre-commit hooks * Add integration tests for CLEAR metrics validation against TrackEval - Add test/conftest.py with fixture to download and cache SoccerNet test data - Add test/eval/test_integration.py with 49 parametrized tests for all sequences - Add ci-integration-tests.yml workflow that runs only on eval code changes - Update ci-tests.yml to exclude integration tests from regular CI - Add integration marker to pyproject.toml pytest config All 49 sequences pass with exact numerical parity to TrackEval for: - Integer metrics: CLR_TP, CLR_FN, CLR_FP, IDSW, MT, PT, ML, Frag - Float metrics: MOTA, MOTP, MTR, PTR, MLR * Refactor integration tests and improve code quality - Rename CI job to 'TrackEval Parity Validation' for clarity - Derive sequence names dynamically from expected_results.json - Simplify test_io.py from 21 to 10 test cases while maintaining coverage - Restore useful comments in io.py around ID remapping and IoU computation - Clean up conftest.py and test_integration.py docstrings * fix(pre_commit): 🎨 auto format pre-commit hooks * Add high-level evaluation API and CLI for tracker evaluation Introduce SDK functions and CLI for evaluating multi-object tracking results against ground truth using MOT Challenge format data. SDK: - evaluate_mot_sequence(): Evaluate single sequence, returns CLEAR metrics - evaluate_benchmark(): Evaluate multiple sequences with aggregation CLI (beta): - `trackers eval --gt <file> --tracker <file>` for single sequence - `trackers eval --gt-dir <dir> --tracker-dir <dir>` for benchmarks - Optional rich output with `pip install trackers[cli]` - Beta warning displayed on CLI usage * fix(pre_commit): 🎨 auto format pre-commit hooks * Add high-level evaluation API and CLI for tracker evaluation Introduce SDK functions and CLI for evaluating multi-object tracking results against ground truth using MOT Challenge format data. SDK: - evaluate_mot_sequence(): Evaluate single sequence, returns CLEAR metrics - evaluate_benchmark(): Evaluate multiple sequences with aggregation CLI (beta): - `trackers eval --gt <file> --tracker <file>` for single sequence - `trackers eval --gt-dir <dir> --tracker-dir <dir>` for benchmarks - Optional rich output with `pip install trackers[cli]` - Beta warning displayed on CLI usage * fix(pre_commit): 🎨 auto format pre-commit hooks * Update integration tests to use SportsMOT and DanceTrack datasets (#221) * Update integration tests to use SportsMOT and DanceTrack data Replace SoccerNet test data with SportsMOT (45 sequences) and DanceTrack (25 sequences) datasets. The tests now validate CLEAR metrics against TrackEval results for 70 total sequences. - Update URLs to new GCS-hosted test data zips - Add multi-dataset fixture support in conftest.py - Parametrize tests across both datasets - Fix metric comparison (new format uses fractions, not percentages) * Fix mypy conftest module name conflict Use a pytest fixture instead of directly importing from conftest.py to avoid mypy seeing the file under two module names ("conftest" and "test.conftest"). * Add type annotation for test_cases variable * Add smart auto-detection and structured result objects to evaluation API (#222) * Add smart auto-detection and structured result objects to evaluation API - Remove data_format parameter from SDK and --data-format flag from CLI - Add smart detection: auto-detect format (flat vs MOT17), benchmark, split, and tracker name - Log detection results for transparency; error helpfully when ambiguous - Add BenchmarkResult and SequenceResult dataclasses with json/table/save/load methods - Add all CLEAR metrics (MOTA, MOTP, MODA, CLR_Re, CLR_Pr, MTR, PTR, MLR, sMOTA, etc.) - Simplify test fixtures to use auto-detection instead of explicit format parameters * fix(pre_commit): 🎨 auto format pre-commit hooks * Simplify CLI table output to use SDK result.table() method - Remove custom Rich table formatting that truncated columns - Use result.table(columns) for consistent, readable output - Set sensible default columns: MOTA, MOTP, IDSW, CLR_FP, CLR_FN, MT, ML - Remove unused Rich imports and helper functions --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Add HOTA (Higher Order Tracking Accuracy) metrics - Create trackers/eval/hota.py with compute_hota_metrics() and aggregate_hota_metrics() functions implementing HOTA algorithm - Add HOTAMetrics dataclass for storing HOTA results - Update SequenceResult and BenchmarkResult to support optional HOTA metrics - Integrate HOTA computation into evaluate_mot_sequence and evaluate_benchmark - Update CLI to support --metrics HOTA option - Add comprehensive unit tests for HOTA computation and aggregation HOTA evaluates tracking at multiple IoU thresholds (0.05-0.95) and computes: - Detection metrics: DetA, DetRe, DetPr - Association metrics: AssA, AssRe, AssPr - Combined HOTA = sqrt(DetA * AssA) - Localization accuracy: LocA * fix(pre_commit): 🎨 auto format pre-commit hooks * Add HOTA (Higher Order Tracking Accuracy) metrics - Create trackers/eval/hota.py with compute_hota_metrics() and aggregate_hota_metrics() functions implementing HOTA algorithm - Add HOTAMetrics dataclass for storing HOTA results - Update SequenceResult and BenchmarkResult to support optional HOTA metrics - Integrate HOTA computation into evaluate_mot_sequence and evaluate_benchmark - Update CLI to support --metrics HOTA option - Add comprehensive unit tests for HOTA computation and aggregation HOTA evaluates tracking at multiple IoU thresholds (0.05-0.95) and computes: - Detection metrics: DetA, DetRe, DetPr - Association metrics: AssA, AssRe, AssPr - Combined HOTA = sqrt(DetA * AssA) - Localization accuracy: LocA * fix(pre_commit): 🎨 auto format pre-commit hooks * Show all metrics by default in table output Remove the column limitation that was originally added as a workaround for Rich table truncation. Now using plain-text tables that don't truncate, so show all available metrics by default: - CLEAR-only: 17 metrics - HOTA+CLEAR: 28 metrics (11 HOTA + 17 CLEAR) Users can still limit columns via the columns parameter if desired. * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix HOTA aggregation TypeError with numpy arrays HOTAMetrics.to_dict() was converting numpy arrays to lists for JSON serialization, but aggregate_hota_metrics() needs numpy arrays for weighted averaging math. Added arrays_as_list parameter to to_dict() to control this behavior: - arrays_as_list=True (default): convert to lists for JSON - arrays_as_list=False: keep as numpy arrays for aggregation * Add HOTA verification to integration tests Update all 4 integration tests to: - Pass metrics=["CLEAR", "HOTA"] to evaluate_benchmark() - Verify HOTA metrics (HOTA, DetA, AssA, LocA) against TrackEval expected results This ensures numerical parity is verified for both CLEAR and HOTA metrics. * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix HOTA expected values in test data Update test data URLs to use regenerated zips with correct HOTA metrics. The previous expected_results.json files incorrectly used HOTA(0) (value at alpha=0.05) instead of the mean HOTA across all alpha thresholds. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
… Part 7) (#224) * Add box IoU/IoA calculation module from TrackEval - Add box_iou() for Intersection over Union calculation - Add box_ioa() for Intersection over Area calculation - Support both xyxy and xywh box formats - Include 32 unit tests covering edge cases and floating point precision - Adapted from TrackEval with MIT license attribution * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix ruff and mypy linting errors - Add S101 ignore for test files in pyproject.toml - Split error test cases into separate test functions - Remove type: ignore comments by simplifying type hints - Clean up unused imports * Add Hungarian matching utility for detection assignment - Add match_detections() using Jonker-Volgenant algorithm (scipy) - Maximize similarity with threshold filtering - Return matched/unmatched indices for GT and tracker detections - Include 17 unit tests covering edge cases and scipy consistency - Adapted from TrackEval matching patterns * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix unused variable warnings in tests * Fix remaining unused variable warning * Add diverse test cases for match_detections - Off-diagonal matches (GT0->TR2, GT1->TR0, etc.) - Swapped/reversed matches - Sparse non-sequential matches - Very unbalanced sizes (1 GT vs 5 trackers, 5 GTs vs 1 tracker) - Only middle/corner elements matching - Threshold edge cases - Optimal vs greedy assignment tests * Remove redundant tests from test_matching.py * Add CLEAR metrics module for multi-object tracking evaluation Implement compute_clear_metrics function adapted from TrackEval with exact numerical parity. The function computes standard CLEAR metrics including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML). Key implementation details: - Score matrix construction with IDSW prioritization (1000x bonus) - Hungarian matching with proper threshold filtering - ID switch detection based on previous tracker associations - MT/PT/ML thresholds (>80%, >=20%, <20%) - Fragmentation counting for track interruptions - Vectorized GT ID mapping using np.searchsorted Tests are fully parametrized and only test the public API. Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method) * fix(pre_commit): 🎨 auto format pre-commit hooks * Add CLEAR metrics module for multi-object tracking evaluation Implement compute_clear_metrics function adapted from TrackEval with exact numerical parity. The function computes standard CLEAR metrics including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML). Key implementation details: - Score matrix construction with IDSW prioritization (1000x bonus) - Hungarian matching with proper threshold filtering - ID switch detection based on previous tracker associations - MT/PT/ML thresholds (>80%, >=20%, <20%) - Fragmentation counting for track interruptions - Vectorized GT ID mapping using np.searchsorted Tests are fully parametrized and only test the public API. Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method) * fix(pre_commit): 🎨 auto format pre-commit hooks * Add MOT format file loading and sequence preparation for E2E evaluation - Add MOTFrameData dataclass for per-frame detection data - Add MOTSequenceData dataclass for prepared sequence data ready for metrics - Add load_mot_file() to parse MOT Challenge format files - Add prepare_mot_sequence() to compute IoU and remap IDs for evaluation - Update __init__.py to export new types and functions - Add comprehensive unit tests (20 tests for io module) * fix(pre_commit): 🎨 auto format pre-commit hooks * Add integration tests for CLEAR metrics validation against TrackEval - Add test/conftest.py with fixture to download and cache SoccerNet test data - Add test/eval/test_integration.py with 49 parametrized tests for all sequences - Add ci-integration-tests.yml workflow that runs only on eval code changes - Update ci-tests.yml to exclude integration tests from regular CI - Add integration marker to pyproject.toml pytest config All 49 sequences pass with exact numerical parity to TrackEval for: - Integer metrics: CLR_TP, CLR_FN, CLR_FP, IDSW, MT, PT, ML, Frag - Float metrics: MOTA, MOTP, MTR, PTR, MLR * Refactor integration tests and improve code quality - Rename CI job to 'TrackEval Parity Validation' for clarity - Derive sequence names dynamically from expected_results.json - Simplify test_io.py from 21 to 10 test cases while maintaining coverage - Restore useful comments in io.py around ID remapping and IoU computation - Clean up conftest.py and test_integration.py docstrings * fix(pre_commit): 🎨 auto format pre-commit hooks * Add high-level evaluation API and CLI for tracker evaluation Introduce SDK functions and CLI for evaluating multi-object tracking results against ground truth using MOT Challenge format data. SDK: - evaluate_mot_sequence(): Evaluate single sequence, returns CLEAR metrics - evaluate_benchmark(): Evaluate multiple sequences with aggregation CLI (beta): - `trackers eval --gt <file> --tracker <file>` for single sequence - `trackers eval --gt-dir <dir> --tracker-dir <dir>` for benchmarks - Optional rich output with `pip install trackers[cli]` - Beta warning displayed on CLI usage * fix(pre_commit): 🎨 auto format pre-commit hooks * Add high-level evaluation API and CLI for tracker evaluation Introduce SDK functions and CLI for evaluating multi-object tracking results against ground truth using MOT Challenge format data. SDK: - evaluate_mot_sequence(): Evaluate single sequence, returns CLEAR metrics - evaluate_benchmark(): Evaluate multiple sequences with aggregation CLI (beta): - `trackers eval --gt <file> --tracker <file>` for single sequence - `trackers eval --gt-dir <dir> --tracker-dir <dir>` for benchmarks - Optional rich output with `pip install trackers[cli]` - Beta warning displayed on CLI usage * fix(pre_commit): 🎨 auto format pre-commit hooks * Update integration tests to use SportsMOT and DanceTrack datasets (#221) * Update integration tests to use SportsMOT and DanceTrack data Replace SoccerNet test data with SportsMOT (45 sequences) and DanceTrack (25 sequences) datasets. The tests now validate CLEAR metrics against TrackEval results for 70 total sequences. - Update URLs to new GCS-hosted test data zips - Add multi-dataset fixture support in conftest.py - Parametrize tests across both datasets - Fix metric comparison (new format uses fractions, not percentages) * Fix mypy conftest module name conflict Use a pytest fixture instead of directly importing from conftest.py to avoid mypy seeing the file under two module names ("conftest" and "test.conftest"). * Add type annotation for test_cases variable * Add smart auto-detection and structured result objects to evaluation API (#222) * Add smart auto-detection and structured result objects to evaluation API - Remove data_format parameter from SDK and --data-format flag from CLI - Add smart detection: auto-detect format (flat vs MOT17), benchmark, split, and tracker name - Log detection results for transparency; error helpfully when ambiguous - Add BenchmarkResult and SequenceResult dataclasses with json/table/save/load methods - Add all CLEAR metrics (MOTA, MOTP, MODA, CLR_Re, CLR_Pr, MTR, PTR, MLR, sMOTA, etc.) - Simplify test fixtures to use auto-detection instead of explicit format parameters * fix(pre_commit): 🎨 auto format pre-commit hooks * Simplify CLI table output to use SDK result.table() method - Remove custom Rich table formatting that truncated columns - Use result.table(columns) for consistent, readable output - Set sensible default columns: MOTA, MOTP, IDSW, CLR_FP, CLR_FN, MT, ML - Remove unused Rich imports and helper functions --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Add HOTA (Higher Order Tracking Accuracy) metrics - Create trackers/eval/hota.py with compute_hota_metrics() and aggregate_hota_metrics() functions implementing HOTA algorithm - Add HOTAMetrics dataclass for storing HOTA results - Update SequenceResult and BenchmarkResult to support optional HOTA metrics - Integrate HOTA computation into evaluate_mot_sequence and evaluate_benchmark - Update CLI to support --metrics HOTA option - Add comprehensive unit tests for HOTA computation and aggregation HOTA evaluates tracking at multiple IoU thresholds (0.05-0.95) and computes: - Detection metrics: DetA, DetRe, DetPr - Association metrics: AssA, AssRe, AssPr - Combined HOTA = sqrt(DetA * AssA) - Localization accuracy: LocA * fix(pre_commit): 🎨 auto format pre-commit hooks * Add HOTA (Higher Order Tracking Accuracy) metrics - Create trackers/eval/hota.py with compute_hota_metrics() and aggregate_hota_metrics() functions implementing HOTA algorithm - Add HOTAMetrics dataclass for storing HOTA results - Update SequenceResult and BenchmarkResult to support optional HOTA metrics - Integrate HOTA computation into evaluate_mot_sequence and evaluate_benchmark - Update CLI to support --metrics HOTA option - Add comprehensive unit tests for HOTA computation and aggregation HOTA evaluates tracking at multiple IoU thresholds (0.05-0.95) and computes: - Detection metrics: DetA, DetRe, DetPr - Association metrics: AssA, AssRe, AssPr - Combined HOTA = sqrt(DetA * AssA) - Localization accuracy: LocA * fix(pre_commit): 🎨 auto format pre-commit hooks * Show all metrics by default in table output Remove the column limitation that was originally added as a workaround for Rich table truncation. Now using plain-text tables that don't truncate, so show all available metrics by default: - CLEAR-only: 17 metrics - HOTA+CLEAR: 28 metrics (11 HOTA + 17 CLEAR) Users can still limit columns via the columns parameter if desired. * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix HOTA aggregation TypeError with numpy arrays HOTAMetrics.to_dict() was converting numpy arrays to lists for JSON serialization, but aggregate_hota_metrics() needs numpy arrays for weighted averaging math. Added arrays_as_list parameter to to_dict() to control this behavior: - arrays_as_list=True (default): convert to lists for JSON - arrays_as_list=False: keep as numpy arrays for aggregation * Add HOTA verification to integration tests Update all 4 integration tests to: - Pass metrics=["CLEAR", "HOTA"] to evaluate_benchmark() - Verify HOTA metrics (HOTA, DetA, AssA, LocA) against TrackEval expected results This ensures numerical parity is verified for both CLEAR and HOTA metrics. * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix HOTA expected values in test data Update test data URLs to use regenerated zips with correct HOTA metrics. The previous expected_results.json files incorrectly used HOTA(0) (value at alpha=0.05) instead of the mean HOTA across all alpha thresholds. * Add Identity metrics (IDF1, IDR, IDP) implementation Implements Identity metrics following TrackEval's algorithm: - compute_identity_metrics() for single sequence evaluation - aggregate_identity_metrics() for benchmark aggregation - IdentityMetrics dataclass for structured results Updates evaluate_mot_sequence and evaluate_benchmark to support metrics=["Identity"] alongside CLEAR and HOTA. Integration tests now verify CLEAR, HOTA, and Identity metrics against TrackEval precomputed values. * Add Identity metrics support to CLI Update --metrics argument to accept Identity alongside CLEAR and HOTA. * code review pt1 * fix(pre_commit): 🎨 auto format pre-commit hooks * code review pt2 * Remove unused match_detections utility The match_detections function was not used by any metric internally. Each metric (CLEAR, HOTA, Identity) implements specialized matching logic that cannot be generalized into a shared utility. Deleted: - trackers/eval/matching.py - test/eval/test_matching.py --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add box IoU/IoA calculation module from TrackEval - Add box_iou() for Intersection over Union calculation - Add box_ioa() for Intersection over Area calculation - Support both xyxy and xywh box formats - Include 32 unit tests covering edge cases and floating point precision - Adapted from TrackEval with MIT license attribution * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix ruff and mypy linting errors - Add S101 ignore for test files in pyproject.toml - Split error test cases into separate test functions - Remove type: ignore comments by simplifying type hints - Clean up unused imports * Add Hungarian matching utility for detection assignment - Add match_detections() using Jonker-Volgenant algorithm (scipy) - Maximize similarity with threshold filtering - Return matched/unmatched indices for GT and tracker detections - Include 17 unit tests covering edge cases and scipy consistency - Adapted from TrackEval matching patterns * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix unused variable warnings in tests * Fix remaining unused variable warning * Add diverse test cases for match_detections - Off-diagonal matches (GT0->TR2, GT1->TR0, etc.) - Swapped/reversed matches - Sparse non-sequential matches - Very unbalanced sizes (1 GT vs 5 trackers, 5 GTs vs 1 tracker) - Only middle/corner elements matching - Threshold edge cases - Optimal vs greedy assignment tests * Remove redundant tests from test_matching.py * Add CLEAR metrics module for multi-object tracking evaluation Implement compute_clear_metrics function adapted from TrackEval with exact numerical parity. The function computes standard CLEAR metrics including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML). Key implementation details: - Score matrix construction with IDSW prioritization (1000x bonus) - Hungarian matching with proper threshold filtering - ID switch detection based on previous tracker associations - MT/PT/ML thresholds (>80%, >=20%, <20%) - Fragmentation counting for track interruptions - Vectorized GT ID mapping using np.searchsorted Tests are fully parametrized and only test the public API. Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method) * fix(pre_commit): 🎨 auto format pre-commit hooks * Add CLEAR metrics module for multi-object tracking evaluation Implement compute_clear_metrics function adapted from TrackEval with exact numerical parity. The function computes standard CLEAR metrics including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML). Key implementation details: - Score matrix construction with IDSW prioritization (1000x bonus) - Hungarian matching with proper threshold filtering - ID switch detection based on previous tracker associations - MT/PT/ML thresholds (>80%, >=20%, <20%) - Fragmentation counting for track interruptions - Vectorized GT ID mapping using np.searchsorted Tests are fully parametrized and only test the public API. Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method) * fix(pre_commit): 🎨 auto format pre-commit hooks * Add MOT format file loading and sequence preparation for E2E evaluation - Add MOTFrameData dataclass for per-frame detection data - Add MOTSequenceData dataclass for prepared sequence data ready for metrics - Add load_mot_file() to parse MOT Challenge format files - Add prepare_mot_sequence() to compute IoU and remap IDs for evaluation - Update __init__.py to export new types and functions - Add comprehensive unit tests (20 tests for io module) * fix(pre_commit): 🎨 auto format pre-commit hooks * Add integration tests for CLEAR metrics validation against TrackEval - Add test/conftest.py with fixture to download and cache SoccerNet test data - Add test/eval/test_integration.py with 49 parametrized tests for all sequences - Add ci-integration-tests.yml workflow that runs only on eval code changes - Update ci-tests.yml to exclude integration tests from regular CI - Add integration marker to pyproject.toml pytest config All 49 sequences pass with exact numerical parity to TrackEval for: - Integer metrics: CLR_TP, CLR_FN, CLR_FP, IDSW, MT, PT, ML, Frag - Float metrics: MOTA, MOTP, MTR, PTR, MLR * Refactor integration tests and improve code quality - Rename CI job to 'TrackEval Parity Validation' for clarity - Derive sequence names dynamically from expected_results.json - Simplify test_io.py from 21 to 10 test cases while maintaining coverage - Restore useful comments in io.py around ID remapping and IoU computation - Clean up conftest.py and test_integration.py docstrings * fix(pre_commit): 🎨 auto format pre-commit hooks * Add high-level evaluation API and CLI for tracker evaluation Introduce SDK functions and CLI for evaluating multi-object tracking results against ground truth using MOT Challenge format data. SDK: - evaluate_mot_sequence(): Evaluate single sequence, returns CLEAR metrics - evaluate_benchmark(): Evaluate multiple sequences with aggregation CLI (beta): - `trackers eval --gt <file> --tracker <file>` for single sequence - `trackers eval --gt-dir <dir> --tracker-dir <dir>` for benchmarks - Optional rich output with `pip install trackers[cli]` - Beta warning displayed on CLI usage * fix(pre_commit): 🎨 auto format pre-commit hooks * Add high-level evaluation API and CLI for tracker evaluation Introduce SDK functions and CLI for evaluating multi-object tracking results against ground truth using MOT Challenge format data. SDK: - evaluate_mot_sequence(): Evaluate single sequence, returns CLEAR metrics - evaluate_benchmark(): Evaluate multiple sequences with aggregation CLI (beta): - `trackers eval --gt <file> --tracker <file>` for single sequence - `trackers eval --gt-dir <dir> --tracker-dir <dir>` for benchmarks - Optional rich output with `pip install trackers[cli]` - Beta warning displayed on CLI usage * fix(pre_commit): 🎨 auto format pre-commit hooks * Update integration tests to use SportsMOT and DanceTrack datasets (#221) * Update integration tests to use SportsMOT and DanceTrack data Replace SoccerNet test data with SportsMOT (45 sequences) and DanceTrack (25 sequences) datasets. The tests now validate CLEAR metrics against TrackEval results for 70 total sequences. - Update URLs to new GCS-hosted test data zips - Add multi-dataset fixture support in conftest.py - Parametrize tests across both datasets - Fix metric comparison (new format uses fractions, not percentages) * Fix mypy conftest module name conflict Use a pytest fixture instead of directly importing from conftest.py to avoid mypy seeing the file under two module names ("conftest" and "test.conftest"). * Add type annotation for test_cases variable * Add smart auto-detection and structured result objects to evaluation API (#222) * Add smart auto-detection and structured result objects to evaluation API - Remove data_format parameter from SDK and --data-format flag from CLI - Add smart detection: auto-detect format (flat vs MOT17), benchmark, split, and tracker name - Log detection results for transparency; error helpfully when ambiguous - Add BenchmarkResult and SequenceResult dataclasses with json/table/save/load methods - Add all CLEAR metrics (MOTA, MOTP, MODA, CLR_Re, CLR_Pr, MTR, PTR, MLR, sMOTA, etc.) - Simplify test fixtures to use auto-detection instead of explicit format parameters * fix(pre_commit): 🎨 auto format pre-commit hooks * Simplify CLI table output to use SDK result.table() method - Remove custom Rich table formatting that truncated columns - Use result.table(columns) for consistent, readable output - Set sensible default columns: MOTA, MOTP, IDSW, CLR_FP, CLR_FN, MT, ML - Remove unused Rich imports and helper functions --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Add HOTA (Higher Order Tracking Accuracy) metrics - Create trackers/eval/hota.py with compute_hota_metrics() and aggregate_hota_metrics() functions implementing HOTA algorithm - Add HOTAMetrics dataclass for storing HOTA results - Update SequenceResult and BenchmarkResult to support optional HOTA metrics - Integrate HOTA computation into evaluate_mot_sequence and evaluate_benchmark - Update CLI to support --metrics HOTA option - Add comprehensive unit tests for HOTA computation and aggregation HOTA evaluates tracking at multiple IoU thresholds (0.05-0.95) and computes: - Detection metrics: DetA, DetRe, DetPr - Association metrics: AssA, AssRe, AssPr - Combined HOTA = sqrt(DetA * AssA) - Localization accuracy: LocA * fix(pre_commit): 🎨 auto format pre-commit hooks * Add HOTA (Higher Order Tracking Accuracy) metrics - Create trackers/eval/hota.py with compute_hota_metrics() and aggregate_hota_metrics() functions implementing HOTA algorithm - Add HOTAMetrics dataclass for storing HOTA results - Update SequenceResult and BenchmarkResult to support optional HOTA metrics - Integrate HOTA computation into evaluate_mot_sequence and evaluate_benchmark - Update CLI to support --metrics HOTA option - Add comprehensive unit tests for HOTA computation and aggregation HOTA evaluates tracking at multiple IoU thresholds (0.05-0.95) and computes: - Detection metrics: DetA, DetRe, DetPr - Association metrics: AssA, AssRe, AssPr - Combined HOTA = sqrt(DetA * AssA) - Localization accuracy: LocA * fix(pre_commit): 🎨 auto format pre-commit hooks * Show all metrics by default in table output Remove the column limitation that was originally added as a workaround for Rich table truncation. Now using plain-text tables that don't truncate, so show all available metrics by default: - CLEAR-only: 17 metrics - HOTA+CLEAR: 28 metrics (11 HOTA + 17 CLEAR) Users can still limit columns via the columns parameter if desired. * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix HOTA aggregation TypeError with numpy arrays HOTAMetrics.to_dict() was converting numpy arrays to lists for JSON serialization, but aggregate_hota_metrics() needs numpy arrays for weighted averaging math. Added arrays_as_list parameter to to_dict() to control this behavior: - arrays_as_list=True (default): convert to lists for JSON - arrays_as_list=False: keep as numpy arrays for aggregation * Add HOTA verification to integration tests Update all 4 integration tests to: - Pass metrics=["CLEAR", "HOTA"] to evaluate_benchmark() - Verify HOTA metrics (HOTA, DetA, AssA, LocA) against TrackEval expected results This ensures numerical parity is verified for both CLEAR and HOTA metrics. * fix(pre_commit): 🎨 auto format pre-commit hooks * Fix HOTA expected values in test data Update test data URLs to use regenerated zips with correct HOTA metrics. The previous expected_results.json files incorrectly used HOTA(0) (value at alpha=0.05) instead of the mean HOTA across all alpha thresholds. * Add Identity metrics (IDF1, IDR, IDP) implementation Implements Identity metrics following TrackEval's algorithm: - compute_identity_metrics() for single sequence evaluation - aggregate_identity_metrics() for benchmark aggregation - IdentityMetrics dataclass for structured results Updates evaluate_mot_sequence and evaluate_benchmark to support metrics=["Identity"] alongside CLEAR and HOTA. Integration tests now verify CLEAR, HOTA, and Identity metrics against TrackEval precomputed values. * Add Identity metrics support to CLI Update --metrics argument to accept Identity alongside CLEAR and HOTA. * code review pt1 * fix(pre_commit): 🎨 auto format pre-commit hooks * code review pt2 * Remove unused match_detections utility The match_detections function was not used by any metric internally. Each metric (CLEAR, HOTA, Identity) implements specialized matching logic that cannot be generalized into a shared utility. Deleted: - trackers/eval/matching.py - test/eval/test_matching.py * Add comprehensive evaluation documentation and API reference (TrackEval Part 8) * fix(pre_commit): 🎨 auto format pre-commit hooks * Refine evaluation docs and unify docstring style - Remove premature CLI and Python quickstart pages - Fix copy-code.js to handle empty continuation lines (>>>) - Remove broken links to cli.md from evaluate.md - Remove duplicate API sections from SORT/ByteTrack pages - Unify docstring style in results.py to match evaluate.py - Remove examples from results.py docstrings for consistency - Standardize attribute documentation format (no backticks on names) * Revamp documentation with Stripe-style formatting - Rewrite evaluate.md with concise intro, "What you'll learn" section, clean tabs for Python/CLI, and collapsible troubleshooting - Apply same style to install.md for consistency - Add full-width table CSS styling - Remove help.md page from navigation * fix(pre_commit): 🎨 auto format pre-commit hooks * Revamp documentation with Stripe-style formatting - Rewrite evaluate.md with concise intro, "What you'll learn" section, clean tabs for Python/CLI, and collapsible troubleshooting - Apply same style to install.md for consistency - Add full-width table CSS styling - Remove help.md page from navigation * Add comprehensive evaluation documentation and API reference (TrackEval Part 8) * fix(pre_commit): 🎨 auto format pre-commit hooks --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add TrackEval-compatible evaluation metrics (CLEAR, HOTA, Identity) with SDK and CLI (#225)
* Add evaluation module with box IoU/IoA calculations (TrackEval Part 1) (#210)
* Add box IoU/IoA calculation module from TrackEval
- Add box_iou() for Intersection over Union calculation
- Add box_ioa() for Intersection over Area calculation
- Support both xyxy and xywh box formats
- Include 32 unit tests covering edge cases and floating point precision
- Adapted from TrackEval with MIT license attribution
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix ruff and mypy linting errors
- Add S101 ignore for test files in pyproject.toml
- Split error test cases into separate test functions
- Remove type: ignore comments by simplifying type hints
- Clean up unused imports
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add Hungarian matching utility for detection assignment (TrackEval Part 2) (#211)
* Add box IoU/IoA calculation module from TrackEval
- Add box_iou() for Intersection over Union calculation
- Add box_ioa() for Intersection over Area calculation
- Support both xyxy and xywh box formats
- Include 32 unit tests covering edge cases and floating point precision
- Adapted from TrackEval with MIT license attribution
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix ruff and mypy linting errors
- Add S101 ignore for test files in pyproject.toml
- Split error test cases into separate test functions
- Remove type: ignore comments by simplifying type hints
- Clean up unused imports
* Add Hungarian matching utility for detection assignment
- Add match_detections() using Jonker-Volgenant algorithm (scipy)
- Maximize similarity with threshold filtering
- Return matched/unmatched indices for GT and tracker detections
- Include 17 unit tests covering edge cases and scipy consistency
- Adapted from TrackEval matching patterns
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix unused variable warnings in tests
* Fix remaining unused variable warning
* Add diverse test cases for match_detections
- Off-diagonal matches (GT0->TR2, GT1->TR0, etc.)
- Swapped/reversed matches
- Sparse non-sequential matches
- Very unbalanced sizes (1 GT vs 5 trackers, 5 GTs vs 1 tracker)
- Only middle/corner elements matching
- Threshold edge cases
- Optimal vs greedy assignment tests
* Remove redundant tests from test_matching.py
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add CLEAR metrics module (MOTA, MOTP, IDSW, MT/PT/ML) (TrackEval Part 3) (#212)
* Add box IoU/IoA calculation module from TrackEval
- Add box_iou() for Intersection over Union calculation
- Add box_ioa() for Intersection over Area calculation
- Support both xyxy and xywh box formats
- Include 32 unit tests covering edge cases and floating point precision
- Adapted from TrackEval with MIT license attribution
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix ruff and mypy linting errors
- Add S101 ignore for test files in pyproject.toml
- Split error test cases into separate test functions
- Remove type: ignore comments by simplifying type hints
- Clean up unused imports
* Add Hungarian matching utility for detection assignment
- Add match_detections() using Jonker-Volgenant algorithm (scipy)
- Maximize similarity with threshold filtering
- Return matched/unmatched indices for GT and tracker detections
- Include 17 unit tests covering edge cases and scipy consistency
- Adapted from TrackEval matching patterns
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix unused variable warnings in tests
* Fix remaining unused variable warning
* Add diverse test cases for match_detections
- Off-diagonal matches (GT0->TR2, GT1->TR0, etc.)
- Swapped/reversed matches
- Sparse non-sequential matches
- Very unbalanced sizes (1 GT vs 5 trackers, 5 GTs vs 1 tracker)
- Only middle/corner elements matching
- Threshold edge cases
- Optimal vs greedy assignment tests
* Remove redundant tests from test_matching.py
* Add CLEAR metrics module for multi-object tracking evaluation
Implement compute_clear_metrics function adapted from TrackEval with
exact numerical parity. The function computes standard CLEAR metrics
including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML).
Key implementation details:
- Score matrix construction with IDSW prioritization (1000x bonus)
- Hungarian matching with proper threshold filtering
- ID switch detection based on previous tracker associations
- MT/PT/ML thresholds (>80%, >=20%, <20%)
- Fragmentation counting for track interruptions
- Vectorized GT ID mapping using np.searchsorted
Tests are fully parametrized and only test the public API.
Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add CLEAR metrics module for multi-object tracking evaluation
Implement compute_clear_metrics function adapted from TrackEval with
exact numerical parity. The function computes standard CLEAR metrics
including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML).
Key implementation details:
- Score matrix construction with IDSW prioritization (1000x bonus)
- Hungarian matching with proper threshold filtering
- ID switch detection based on previous tracker associations
- MT/PT/ML thresholds (>80%, >=20%, <20%)
- Fragmentation counting for track interruptions
- Vectorized GT ID mapping using np.searchsorted
Tests are fully parametrized and only test the public API.
Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* fix(pre_commit): 🎨 auto format pre-commit hooks
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add MOT format file loading and sequence preparation for evaluation (TrackEval Part 4) (#214)
* Add box IoU/IoA calculation module from TrackEval
- Add box_iou() for Intersection over Union calculation
- Add box_ioa() for Intersection over Area calculation
- Support both xyxy and xywh box formats
- Include 32 unit tests covering edge cases and floating point precision
- Adapted from TrackEval with MIT license attribution
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix ruff and mypy linting errors
- Add S101 ignore for test files in pyproject.toml
- Split error test cases into separate test functions
- Remove type: ignore comments by simplifying type hints
- Clean up unused imports
* Add Hungarian matching utility for detection assignment
- Add match_detections() using Jonker-Volgenant algorithm (scipy)
- Maximize similarity with threshold filtering
- Return matched/unmatched indices for GT and tracker detections
- Include 17 unit tests covering edge cases and scipy consistency
- Adapted from TrackEval matching patterns
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix unused variable warnings in tests
* Fix remaining unused variable warning
* Add diverse test cases for match_detections
- Off-diagonal matches (GT0->TR2, GT1->TR0, etc.)
- Swapped/reversed matches
- Sparse non-sequential matches
- Very unbalanced sizes (1 GT vs 5 trackers, 5 GTs vs 1 tracker)
- Only middle/corner elements matching
- Threshold edge cases
- Optimal vs greedy assignment tests
* Remove redundant tests from test_matching.py
* Add CLEAR metrics module for multi-object tracking evaluation
Implement compute_clear_metrics function adapted from TrackEval with
exact numerical parity. The function computes standard CLEAR metrics
including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML).
Key implementation details:
- Score matrix construction with IDSW prioritization (1000x bonus)
- Hungarian matching with proper threshold filtering
- ID switch detection based on previous tracker associations
- MT/PT/ML thresholds (>80%, >=20%, <20%)
- Fragmentation counting for track interruptions
- Vectorized GT ID mapping using np.searchsorted
Tests are fully parametrized and only test the public API.
Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add CLEAR metrics module for multi-object tracking evaluation
Implement compute_clear_metrics function adapted from TrackEval with
exact numerical parity. The function computes standard CLEAR metrics
including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML).
Key implementation details:
- Score matrix construction with IDSW prioritization (1000x bonus)
- Hungarian matching with proper threshold filtering
- ID switch detection based on previous tracker associations
- MT/PT/ML thresholds (>80%, >=20%, <20%)
- Fragmentation counting for track interruptions
- Vectorized GT ID mapping using np.searchsorted
Tests are fully parametrized and only test the public API.
Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add MOT format file loading and sequence preparation for E2E evaluation
- Add MOTFrameData dataclass for per-frame detection data
- Add MOTSequenceData dataclass for prepared sequence data ready for metrics
- Add load_mot_file() to parse MOT Challenge format files
- Add prepare_mot_sequence() to compute IoU and remap IDs for evaluation
- Update __init__.py to export new types and functions
- Add comprehensive unit tests (20 tests for io module)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add integration tests for CLEAR metrics validation against TrackEval
- Add test/conftest.py with fixture to download and cache SoccerNet test data
- Add test/eval/test_integration.py with 49 parametrized tests for all sequences
- Add ci-integration-tests.yml workflow that runs only on eval code changes
- Update ci-tests.yml to exclude integration tests from regular CI
- Add integration marker to pyproject.toml pytest config
All 49 sequences pass with exact numerical parity to TrackEval for:
- Integer metrics: CLR_TP, CLR_FN, CLR_FP, IDSW, MT, PT, ML, Frag
- Float metrics: MOTA, MOTP, MTR, PTR, MLR
* Refactor integration tests and improve code quality
- Rename CI job to 'TrackEval Parity Validation' for clarity
- Derive sequence names dynamically from expected_results.json
- Simplify test_io.py from 21 to 10 test cases while maintaining coverage
- Restore useful comments in io.py around ID remapping and IoU computation
- Clean up conftest.py and test_integration.py docstrings
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Update integration tests to use SportsMOT and DanceTrack datasets (#221)
* Update integration tests to use SportsMOT and DanceTrack data
Replace SoccerNet test data with SportsMOT (45 sequences) and DanceTrack
(25 sequences) datasets. The tests now validate CLEAR metrics against
TrackEval results for 70 total sequences.
- Update URLs to new GCS-hosted test data zips
- Add multi-dataset fixture support in conftest.py
- Parametrize tests across both datasets
- Fix metric comparison (new format uses fractions, not percentages)
* Fix mypy conftest module name conflict
Use a pytest fixture instead of directly importing from conftest.py to avoid
mypy seeing the file under two module names ("conftest" and "test.conftest").
* Add type annotation for test_cases variable
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add high-level evaluation API and CLI (TrackEval Part 5) (#215)
* Add box IoU/IoA calculation module from TrackEval
- Add box_iou() for Intersection over Union calculation
- Add box_ioa() for Intersection over Area calculation
- Support both xyxy and xywh box formats
- Include 32 unit tests covering edge cases and floating point precision
- Adapted from TrackEval with MIT license attribution
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix ruff and mypy linting errors
- Add S101 ignore for test files in pyproject.toml
- Split error test cases into separate test functions
- Remove type: ignore comments by simplifying type hints
- Clean up unused imports
* Add Hungarian matching utility for detection assignment
- Add match_detections() using Jonker-Volgenant algorithm (scipy)
- Maximize similarity with threshold filtering
- Return matched/unmatched indices for GT and tracker detections
- Include 17 unit tests covering edge cases and scipy consistency
- Adapted from TrackEval matching patterns
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix unused variable warnings in tests
* Fix remaining unused variable warning
* Add diverse test cases for match_detections
- Off-diagonal matches (GT0->TR2, GT1->TR0, etc.)
- Swapped/reversed matches
- Sparse non-sequential matches
- Very unbalanced sizes (1 GT vs 5 trackers, 5 GTs vs 1 tracker)
- Only middle/corner elements matching
- Threshold edge cases
- Optimal vs greedy assignment tests
* Remove redundant tests from test_matching.py
* Add CLEAR metrics module for multi-object tracking evaluation
Implement compute_clear_metrics function adapted from TrackEval with
exact numerical parity. The function computes standard CLEAR metrics
including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML).
Key implementation details:
- Score matrix construction with IDSW prioritization (1000x bonus)
- Hungarian matching with proper threshold filtering
- ID switch detection based on previous tracker associations
- MT/PT/ML thresholds (>80%, >=20%, <20%)
- Fragmentation counting for track interruptions
- Vectorized GT ID mapping using np.searchsorted
Tests are fully parametrized and only test the public API.
Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add CLEAR metrics module for multi-object tracking evaluation
Implement compute_clear_metrics function adapted from TrackEval with
exact numerical parity. The function computes standard CLEAR metrics
including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML).
Key implementation details:
- Score matrix construction with IDSW prioritization (1000x bonus)
- Hungarian matching with proper threshold filtering
- ID switch detection based on previous tracker associations
- MT/PT/ML thresholds (>80%, >=20%, <20%)
- Fragmentation counting for track interruptions
- Vectorized GT ID mapping using np.searchsorted
Tests are fully parametrized and only test the public API.
Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add MOT format file loading and sequence preparation for E2E evaluation
- Add MOTFrameData dataclass for per-frame detection data
- Add MOTSequenceData dataclass for prepared sequence data ready for metrics
- Add load_mot_file() to parse MOT Challenge format files
- Add prepare_mot_sequence() to compute IoU and remap IDs for evaluation
- Update __init__.py to export new types and functions
- Add comprehensive unit tests (20 tests for io module)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add integration tests for CLEAR metrics validation against TrackEval
- Add test/conftest.py with fixture to download and cache SoccerNet test data
- Add test/eval/test_integration.py with 49 parametrized tests for all sequences
- Add ci-integration-tests.yml workflow that runs only on eval code changes
- Update ci-tests.yml to exclude integration tests from regular CI
- Add integration marker to pyproject.toml pytest config
All 49 sequences pass with exact numerical parity to TrackEval for:
- Integer metrics: CLR_TP, CLR_FN, CLR_FP, IDSW, MT, PT, ML, Frag
- Float metrics: MOTA, MOTP, MTR, PTR, MLR
* Refactor integration tests and improve code quality
- Rename CI job to 'TrackEval Parity Validation' for clarity
- Derive sequence names dynamically from expected_results.json
- Simplify test_io.py from 21 to 10 test cases while maintaining coverage
- Restore useful comments in io.py around ID remapping and IoU computation
- Clean up conftest.py and test_integration.py docstrings
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add high-level evaluation API and CLI for tracker evaluation
Introduce SDK functions and CLI for evaluating multi-object tracking
results against ground truth using MOT Challenge format data.
SDK:
- evaluate_mot_sequence(): Evaluate single sequence, returns CLEAR metrics
- evaluate_benchmark(): Evaluate multiple sequences with aggregation
CLI (beta):
- `trackers eval --gt <file> --tracker <file>` for single sequence
- `trackers eval --gt-dir <dir> --tracker-dir <dir>` for benchmarks
- Optional rich output with `pip install trackers[cli]`
- Beta warning displayed on CLI usage
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add high-level evaluation API and CLI for tracker evaluation
Introduce SDK functions and CLI for evaluating multi-object tracking
results against ground truth using MOT Challenge format data.
SDK:
- evaluate_mot_sequence(): Evaluate single sequence, returns CLEAR metrics
- evaluate_benchmark(): Evaluate multiple sequences with aggregation
CLI (beta):
- `trackers eval --gt <file> --tracker <file>` for single sequence
- `trackers eval --gt-dir <dir> --tracker-dir <dir>` for benchmarks
- Optional rich output with `pip install trackers[cli]`
- Beta warning displayed on CLI usage
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Update integration tests to use SportsMOT and DanceTrack datasets (#221)
* Update integration tests to use SportsMOT and DanceTrack data
Replace SoccerNet test data with SportsMOT (45 sequences) and DanceTrack
(25 sequences) datasets. The tests now validate CLEAR metrics against
TrackEval results for 70 total sequences.
- Update URLs to new GCS-hosted test data zips
- Add multi-dataset fixture support in conftest.py
- Parametrize tests across both datasets
- Fix metric comparison (new format uses fractions, not percentages)
* Fix mypy conftest module name conflict
Use a pytest fixture instead of directly importing from conftest.py to avoid
mypy seeing the file under two module names ("conftest" and "test.conftest").
* Add type annotation for test_cases variable
* Add smart auto-detection and structured result objects to evaluation API (#222)
* Add smart auto-detection and structured result objects to evaluation API
- Remove data_format parameter from SDK and --data-format flag from CLI
- Add smart detection: auto-detect format (flat vs MOT17), benchmark, split, and tracker name
- Log detection results for transparency; error helpfully when ambiguous
- Add BenchmarkResult and SequenceResult dataclasses with json/table/save/load methods
- Add all CLEAR metrics (MOTA, MOTP, MODA, CLR_Re, CLR_Pr, MTR, PTR, MLR, sMOTA, etc.)
- Simplify test fixtures to use auto-detection instead of explicit format parameters
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Simplify CLI table output to use SDK result.table() method
- Remove custom Rich table formatting that truncated columns
- Use result.table(columns) for consistent, readable output
- Set sensible default columns: MOTA, MOTP, IDSW, CLR_FP, CLR_FN, MT, ML
- Remove unused Rich imports and helper functions
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add HOTA (Higher Order Tracking Accuracy) metrics (TrackEval Part 6) (#223)
* Add box IoU/IoA calculation module from TrackEval
- Add box_iou() for Intersection over Union calculation
- Add box_ioa() for Intersection over Area calculation
- Support both xyxy and xywh box formats
- Include 32 unit tests covering edge cases and floating point precision
- Adapted from TrackEval with MIT license attribution
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix ruff and mypy linting errors
- Add S101 ignore for test files in pyproject.toml
- Split error test cases into separate test functions
- Remove type: ignore comments by simplifying type hints
- Clean up unused imports
* Add Hungarian matching utility for detection assignment
- Add match_detections() using Jonker-Volgenant algorithm (scipy)
- Maximize similarity with threshold filtering
- Return matched/unmatched indices for GT and tracker detections
- Include 17 unit tests covering edge cases and scipy consistency
- Adapted from TrackEval matching patterns
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix unused variable warnings in tests
* Fix remaining unused variable warning
* Add diverse test cases for match_detections
- Off-diagonal matches (GT0->TR2, GT1->TR0, etc.)
- Swapped/reversed matches
- Sparse non-sequential matches
- Very unbalanced sizes (1 GT vs 5 trackers, 5 GTs vs 1 tracker)
- Only middle/corner elements matching
- Threshold edge cases
- Optimal vs greedy assignment tests
* Remove redundant tests from test_matching.py
* Add CLEAR metrics module for multi-object tracking evaluation
Implement compute_clear_metrics function adapted from TrackEval with
exact numerical parity. The function computes standard CLEAR metrics
including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML).
Key implementation details:
- Score matrix construction with IDSW prioritization (1000x bonus)
- Hungarian matching with proper threshold filtering
- ID switch detection based on previous tracker associations
- MT/PT/ML thresholds (>80%, >=20%, <20%)
- Fragmentation counting for track interruptions
- Vectorized GT ID mapping using np.searchsorted
Tests are fully parametrized and only test the public API.
Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add CLEAR metrics module for multi-object tracking evaluation
Implement compute_clear_metrics function adapted from TrackEval with
exact numerical parity. The function computes standard CLEAR metrics
including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML).
Key implementation details:
- Score matrix construction with IDSW prioritization (1000x bonus)
- Hungarian matching with proper threshold filtering
- ID switch detection based on previous tracker associations
- MT/PT/ML thresholds (>80%, >=20%, <20%)
- Fragmentation counting for track interruptions
- Vectorized GT ID mapping using np.searchsorted
Tests are fully parametrized and only test the public API.
Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add MOT format file loading and sequence preparation for E2E evaluation
- Add MOTFrameData dataclass for per-frame detection data
- Add MOTSequenceData dataclass for prepared sequence data ready for metrics
- Add load_mot_file() to parse MOT Challenge format files
- Add prepare_mot_sequence() to compute IoU and remap IDs for evaluation
- Update __init__.py to export new types and functions
- Add comprehensive unit tests (20 tests for io module)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add integration tests for CLEAR metrics validation against TrackEval
- Add test/conftest.py with fixture to download and cache SoccerNet test data
- Add test/eval/test_integration.py with 49 parametrized tests for all sequences
- Add ci-integration-tests.yml workflow that runs only on eval code changes
- Update ci-tests.yml to exclude integration tests from regular CI
- Add integration marker to pyproject.toml pytest config
All 49 sequences pass with exact numerical parity to TrackEval for:
- Integer metrics: CLR_TP, CLR_FN, CLR_FP, IDSW, MT, PT, ML, Frag
- Float metrics: MOTA, MOTP, MTR, PTR, MLR
* Refactor integration tests and improve code quality
- Rename CI job to 'TrackEval Parity Validation' for clarity
- Derive sequence names dynamically from expected_results.json
- Simplify test_io.py from 21 to 10 test cases while maintaining coverage
- Restore useful comments in io.py around ID remapping and IoU computation
- Clean up conftest.py and test_integration.py docstrings
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add high-level evaluation API and CLI for tracker evaluation
Introduce SDK functions and CLI for evaluating multi-object tracking
results against ground truth using MOT Challenge format data.
SDK:
- evaluate_mot_sequence(): Evaluate single sequence, returns CLEAR metrics
- evaluate_benchmark(): Evaluate multiple sequences with aggregation
CLI (beta):
- `trackers eval --gt <file> --tracker <file>` for single sequence
- `trackers eval --gt-dir <dir> --tracker-dir <dir>` for benchmarks
- Optional rich output with `pip install trackers[cli]`
- Beta warning displayed on CLI usage
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add high-level evaluation API and CLI for tracker evaluation
Introduce SDK functions and CLI for evaluating multi-object tracking
results against ground truth using MOT Challenge format data.
SDK:
- evaluate_mot_sequence(): Evaluate single sequence, returns CLEAR metrics
- evaluate_benchmark(): Evaluate multiple sequences with aggregation
CLI (beta):
- `trackers eval --gt <file> --tracker <file>` for single sequence
- `trackers eval --gt-dir <dir> --tracker-dir <dir>` for benchmarks
- Optional rich output with `pip install trackers[cli]`
- Beta warning displayed on CLI usage
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Update integration tests to use SportsMOT and DanceTrack datasets (#221)
* Update integration tests to use SportsMOT and DanceTrack data
Replace SoccerNet test data with SportsMOT (45 sequences) and DanceTrack
(25 sequences) datasets. The tests now validate CLEAR metrics against
TrackEval results for 70 total sequences.
- Update URLs to new GCS-hosted test data zips
- Add multi-dataset fixture support in conftest.py
- Parametrize tests across both datasets
- Fix metric comparison (new format uses fractions, not percentages)
* Fix mypy conftest module name conflict
Use a pytest fixture instead of directly importing from conftest.py to avoid
mypy seeing the file under two module names ("conftest" and "test.conftest").
* Add type annotation for test_cases variable
* Add smart auto-detection and structured result objects to evaluation API (#222)
* Add smart auto-detection and structured result objects to evaluation API
- Remove data_format parameter from SDK and --data-format flag from CLI
- Add smart detection: auto-detect format (flat vs MOT17), benchmark, split, and tracker name
- Log detection results for transparency; error helpfully when ambiguous
- Add BenchmarkResult and SequenceResult dataclasses with json/table/save/load methods
- Add all CLEAR metrics (MOTA, MOTP, MODA, CLR_Re, CLR_Pr, MTR, PTR, MLR, sMOTA, etc.)
- Simplify test fixtures to use auto-detection instead of explicit format parameters
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Simplify CLI table output to use SDK result.table() method
- Remove custom Rich table formatting that truncated columns
- Use result.table(columns) for consistent, readable output
- Set sensible default columns: MOTA, MOTP, IDSW, CLR_FP, CLR_FN, MT, ML
- Remove unused Rich imports and helper functions
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add HOTA (Higher Order Tracking Accuracy) metrics
- Create trackers/eval/hota.py with compute_hota_metrics() and
aggregate_hota_metrics() functions implementing HOTA algorithm
- Add HOTAMetrics dataclass for storing HOTA results
- Update SequenceResult and BenchmarkResult to support optional HOTA metrics
- Integrate HOTA computation into evaluate_mot_sequence and evaluate_benchmark
- Update CLI to support --metrics HOTA option
- Add comprehensive unit tests for HOTA computation and aggregation
HOTA evaluates tracking at multiple IoU thresholds (0.05-0.95) and computes:
- Detection metrics: DetA, DetRe, DetPr
- Association metrics: AssA, AssRe, AssPr
- Combined HOTA = sqrt(DetA * AssA)
- Localization accuracy: LocA
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add HOTA (Higher Order Tracking Accuracy) metrics
- Create trackers/eval/hota.py with compute_hota_metrics() and
aggregate_hota_metrics() functions implementing HOTA algorithm
- Add HOTAMetrics dataclass for storing HOTA results
- Update SequenceResult and BenchmarkResult to support optional HOTA metrics
- Integrate HOTA computation into evaluate_mot_sequence and evaluate_benchmark
- Update CLI to support --metrics HOTA option
- Add comprehensive unit tests for HOTA computation and aggregation
HOTA evaluates tracking at multiple IoU thresholds (0.05-0.95) and computes:
- Detection metrics: DetA, DetRe, DetPr
- Association metrics: AssA, AssRe, AssPr
- Combined HOTA = sqrt(DetA * AssA)
- Localization accuracy: LocA
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Show all metrics by default in table output
Remove the column limitation that was originally added as a workaround
for Rich table truncation. Now using plain-text tables that don't
truncate, so show all available metrics by default:
- CLEAR-only: 17 metrics
- HOTA+CLEAR: 28 metrics (11 HOTA + 17 CLEAR)
Users can still limit columns via the columns parameter if desired.
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix HOTA aggregation TypeError with numpy arrays
HOTAMetrics.to_dict() was converting numpy arrays to lists for JSON
serialization, but aggregate_hota_metrics() needs numpy arrays for
weighted averaging math.
Added arrays_as_list parameter to to_dict() to control this behavior:
- arrays_as_list=True (default): convert to lists for JSON
- arrays_as_list=False: keep as numpy arrays for aggregation
* Add HOTA verification to integration tests
Update all 4 integration tests to:
- Pass metrics=["CLEAR", "HOTA"] to evaluate_benchmark()
- Verify HOTA metrics (HOTA, DetA, AssA, LocA) against TrackEval expected results
This ensures numerical parity is verified for both CLEAR and HOTA metrics.
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix HOTA expected values in test data
Update test data URLs to use regenerated zips with correct HOTA metrics.
The previous expected_results.json files incorrectly used HOTA(0) (value at
alpha=0.05) instead of the mean HOTA across all alpha thresholds.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add Identity metrics (IDF1, IDR, IDP) to evaluation module (TrackEval Part 7) (#224)
* Add box IoU/IoA calculation module from TrackEval
- Add box_iou() for Intersection over Union calculation
- Add box_ioa() for Intersection over Area calculation
- Support both xyxy and xywh box formats
- Include 32 unit tests covering edge cases and floating point precision
- Adapted from TrackEval with MIT license attribution
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix ruff and mypy linting errors
- Add S101 ignore for test files in pyproject.toml
- Split error test cases into separate test functions
- Remove type: ignore comments by simplifying type hints
- Clean up unused imports
* Add Hungarian matching utility for detection assignment
- Add match_detections() using Jonker-Volgenant algorithm (scipy)
- Maximize similarity with threshold filtering
- Return matched/unmatched indices for GT and tracker detections
- Include 17 unit tests covering edge cases and scipy consistency
- Adapted from TrackEval matching patterns
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix unused variable warnings in tests
* Fix remaining unused variable warning
* Add diverse test cases for match_detections
- Off-diagonal matches (GT0->TR2, GT1->TR0, etc.)
- Swapped/reversed matches
- Sparse non-sequential matches
- Very unbalanced sizes (1 GT vs 5 trackers, 5 GTs vs 1 tracker)
- Only middle/corner elements matching
- Threshold edge cases
- Optimal vs greedy assignment tests
* Remove redundant tests from test_matching.py
* Add CLEAR metrics module for multi-object tracking evaluation
Implement compute_clear_metrics function adapted from TrackEval with
exact numerical parity. The function computes standard CLEAR metrics
including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML).
Key implementation details:
- Score matrix construction with IDSW prioritization (1000x bonus)
- Hungarian matching with proper threshold filtering
- ID switch detection based on previous tracker associations
- MT/PT/ML thresholds (>80%, >=20%, <20%)
- Fragmentation counting for track interruptions
- Vectorized GT ID mapping using np.searchsorted
Tests are fully parametrized and only test the public API.
Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add CLEAR metrics module for multi-object tracking evaluation
Implement compute_clear_metrics function adapted from TrackEval with
exact numerical parity. The function computes standard CLEAR metrics
including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML).
Key implementation details:
- Score matrix construction with IDSW prioritization (1000x bonus)
- Hungarian matching with proper threshold filtering
- ID switch detection based on previous tracker associations
- MT/PT/ML thresholds (>80%, >=20%, <20%)
- Fragmentation counting for track interruptions
- Vectorized GT ID mapping using np.searchsorted
Tests are fully parametrized and only test the public API.
Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add MOT format file loading and sequence preparation for E2E evaluation
- Add MOTFrameData dataclass for per-frame detection data
- Add MOTSequenceData dataclass for prepared sequence data ready for metrics
- Add load_mot_file() to parse MOT Challenge format files
- Add prepare_mot_sequence() to compute IoU and remap IDs for evaluation
- Update __init__.py to export new types and functions
- Add comprehensive unit tests (20 tests for io module)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add integration tests for CLEAR metrics validation against TrackEval
- Add test/conftest.py with fixture to download and cache SoccerNet test data
- Add test/eval/test_integration.py with 49 parametrized tests for all sequences
- Add ci-integration-tests.yml workflow that runs only on eval code changes
- Update ci-tests.yml to exclude integration tests from regular CI
- Add integration marker to pyproject.toml pytest config
All 49 sequences pass with exact numerical parity to TrackEval for:
- Integer metrics: CLR_TP, CLR_FN, CLR_FP, IDSW, MT, PT, ML, Frag
- Float metrics: MOTA, MOTP, MTR, PTR, MLR
* Refactor integration tests and improve code quality
- Rename CI job to 'TrackEval Parity Validation' for clarity
- Derive sequence names dynamically from expected_results.json
- Simplify test_io.py from 21 to 10 test cases while maintaining coverage
- Restore useful comments in io.py around ID remapping and IoU computation
- Clean up conftest.py and test_integration.py docstrings
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add high-level evaluation API and CLI for tracker evaluation
Introduce SDK functions and CLI for evaluating multi-object tracking
results against ground truth using MOT Challenge format data.
SDK:
- evaluate_mot_sequence(): Evaluate single sequence, returns CLEAR metrics
- evaluate_benchmark(): Evaluate multiple sequences with aggregation
CLI (beta):
- `trackers eval --gt <file> --tracker <file>` for single sequence
- `trackers eval --gt-dir <dir> --tracker-dir <dir>` for benchmarks
- Optional rich output with `pip install trackers[cli]`
- Beta warning displayed on CLI usage
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add high-level evaluation API and CLI for tracker evaluation
Introduce SDK functions and CLI for evaluating multi-object tracking
results against ground truth using MOT Challenge format data.
SDK:
- evaluate_mot_sequence(): Evaluate single sequence, returns CLEAR metrics
- evaluate_benchmark(): Evaluate multiple sequences with aggregation
CLI (beta):
- `trackers eval --gt <file> --tracker <file>` for single sequence
- `trackers eval --gt-dir <dir> --tracker-dir <dir>` for benchmarks
- Optional rich output with `pip install trackers[cli]`
- Beta warning displayed on CLI usage
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Update integration tests to use SportsMOT and DanceTrack datasets (#221)
* Update integration tests to use SportsMOT and DanceTrack data
Replace SoccerNet test data with SportsMOT (45 sequences) and DanceTrack
(25 sequences) datasets. The tests now validate CLEAR metrics against
TrackEval results for 70 total sequences.
- Update URLs to new GCS-hosted test data zips
- Add multi-dataset fixture support in conftest.py
- Parametrize tests across both datasets
- Fix metric comparison (new format uses fractions, not percentages)
* Fix mypy conftest module name conflict
Use a pytest fixture instead of directly importing from conftest.py to avoid
mypy seeing the file under two module names ("conftest" and "test.conftest").
* Add type annotation for test_cases variable
* Add smart auto-detection and structured result objects to evaluation API (#222)
* Add smart auto-detection and structured result objects to evaluation API
- Remove data_format parameter from SDK and --data-format flag from CLI
- Add smart detection: auto-detect format (flat vs MOT17), benchmark, split, and tracker name
- Log detection results for transparency; error helpfully when ambiguous
- Add BenchmarkResult and SequenceResult dataclasses with json/table/save/load methods
- Add all CLEAR metrics (MOTA, MOTP, MODA, CLR_Re, CLR_Pr, MTR, PTR, MLR, sMOTA, etc.)
- Simplify test fixtures to use auto-detection instead of explicit format parameters
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Simplify CLI table output to use SDK result.table() method
- Remove custom Rich table formatting that truncated columns
- Use result.table(columns) for consistent, readable output
- Set sensible default columns: MOTA, MOTP, IDSW, CLR_FP, CLR_FN, MT, ML
- Remove unused Rich imports and helper functions
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add HOTA (Higher Order Tracking Accuracy) metrics
- Create trackers/eval/hota.py with compute_hota_metrics() and
aggregate_hota_metrics() functions implementing HOTA algorithm
- Add HOTAMetrics dataclass for storing HOTA results
- Update SequenceResult and BenchmarkResult to support optional HOTA metrics
- Integrate HOTA computation into evaluate_mot_sequence and evaluate_benchmark
- Update CLI to support --metrics HOTA option
- Add comprehensive unit tests for HOTA computation and aggregation
HOTA evaluates tracking at multiple IoU thresholds (0.05-0.95) and computes:
- Detection metrics: DetA, DetRe, DetPr
- Association metrics: AssA, AssRe, AssPr
- Combined HOTA = sqrt(DetA * AssA)
- Localization accuracy: LocA
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add HOTA (Higher Order Tracking Accuracy) metrics
- Create trackers/eval/hota.py with compute_hota_metrics() and
aggregate_hota_metrics() functions implementing HOTA algorithm
- Add HOTAMetrics dataclass for storing HOTA results
- Update SequenceResult and BenchmarkResult to support optional HOTA metrics
- Integrate HOTA computation into evaluate_mot_sequence and evaluate_benchmark
- Update CLI to support --metrics HOTA option
- Add comprehensive unit tests for HOTA computation and aggregation
HOTA evaluates tracking at multiple IoU thresholds (0.05-0.95) and computes:
- Detection metrics: DetA, DetRe, DetPr
- Association metrics: AssA, AssRe, AssPr
- Combined HOTA = sqrt(DetA * AssA)
- Localization accuracy: LocA
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Show all metrics by default in table output
Remove the column limitation that was originally added as a workaround
for Rich table truncation. Now using plain-text tables that don't
truncate, so show all available metrics by default:
- CLEAR-only: 17 metrics
- HOTA+CLEAR: 28 metrics (11 HOTA + 17 CLEAR)
Users can still limit columns via the columns parameter if desired.
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix HOTA aggregation TypeError with numpy arrays
HOTAMetrics.to_dict() was converting numpy arrays to lists for JSON
serialization, but aggregate_hota_metrics() needs numpy arrays for
weighted averaging math.
Added arrays_as_list parameter to to_dict() to control this behavior:
- arrays_as_list=True (default): convert to lists for JSON
- arrays_as_list=False: keep as numpy arrays for aggregation
* Add HOTA verification to integration tests
Update all 4 integration tests to:
- Pass metrics=["CLEAR", "HOTA"] to evaluate_benchmark()
- Verify HOTA metrics (HOTA, DetA, AssA, LocA) against TrackEval expected results
This ensures numerical parity is verified for both CLEAR and HOTA metrics.
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix HOTA expected values in test data
Update test data URLs to use regenerated zips with correct HOTA metrics.
The previous expected_results.json files incorrectly used HOTA(0) (value at
alpha=0.05) instead of the mean HOTA across all alpha thresholds.
* Add Identity metrics (IDF1, IDR, IDP) implementation
Implements Identity metrics following TrackEval's algorithm:
- compute_identity_metrics() for single sequence evaluation
- aggregate_identity_metrics() for benchmark aggregation
- IdentityMetrics dataclass for structured results
Updates evaluate_mot_sequence and evaluate_benchmark to support
metrics=["Identity"] alongside CLEAR and HOTA.
Integration tests now verify CLEAR, HOTA, and Identity metrics
against TrackEval precomputed values.
* Add Identity metrics support to CLI
Update --metrics argument to accept Identity alongside CLEAR and HOTA.
* code review pt1
* fix(pre_commit): 🎨 auto format pre-commit hooks
* code review pt2
* Remove unused match_detections utility
The match_detections function was not used by any metric internally.
Each metric (CLEAR, HOTA, Identity) implements specialized matching
logic that cannot be generalized into a shared utility.
Deleted:
- trackers/eval/matching.py
- test/eval/test_matching.py
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Trackeval integration part8 (#226)
* Add box IoU/IoA calculation module from TrackEval
- Add box_iou() for Intersection over Union calculation
- Add box_ioa() for Intersection over Area calculation
- Support both xyxy and xywh box formats
- Include 32 unit tests covering edge cases and floating point precision
- Adapted from TrackEval with MIT license attribution
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix ruff and mypy linting errors
- Add S101 ignore for test files in pyproject.toml
- Split error test cases into separate test functions
- Remove type: ignore comments by simplifying type hints
- Clean up unused imports
* Add Hungarian matching utility for detection assignment
- Add match_detections() using Jonker-Volgenant algorithm (scipy)
- Maximize similarity with threshold filtering
- Return matched/unmatched indices for GT and tracker detections
- Include 17 unit tests covering edge cases and scipy consistency
- Adapted from TrackEval matching patterns
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix unused variable warnings in tests
* Fix remaining unused variable warning
* Add diverse test cases for match_detections
- Off-diagonal matches (GT0->TR2, GT1->TR0, etc.)
- Swapped/reversed matches
- Sparse non-sequential matches
- Very unbalanced sizes (1 GT vs 5 trackers, 5 GTs vs 1 tracker)
- Only middle/corner elements matching
- Threshold edge cases
- Optimal vs greedy assignment tests
* Remove redundant tests from test_matching.py
* Add CLEAR metrics module for multi-object tracking evaluation
Implement compute_clear_metrics function adapted from TrackEval with
exact numerical parity. The function computes standard CLEAR metrics
including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML).
Key implementation details:
- Score matrix construction with IDSW prioritization (1000x bonus)
- Hungarian matching with proper threshold filtering
- ID switch detection based on previous tracker associations
- MT/PT/ML thresholds (>80%, >=20%, <20%)
- Fragmentation counting for track interruptions
- Vectorized GT ID mapping using np.searchsorted
Tests are fully parametrized and only test the public API.
Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add CLEAR metrics module for multi-object tracking evaluation
Implement compute_clear_metrics function adapted from TrackEval with
exact numerical parity. The function computes standard CLEAR metrics
including MOTA, MOTP, IDSW, and track quality metrics (MT/PT/ML).
Key implementation details:
- Score matrix construction with IDSW prioritization (1000x bonus)
- Hungarian matching with proper threshold filtering
- ID switch detection based on previous tracker associations
- MT/PT/ML thresholds (>80%, >=20%, <20%)
- Fragmentation counting for track interruptions
- Vectorized GT ID mapping using np.searchsorted
Tests are fully parametrized and only test the public API.
Reference: trackeval/metrics/clear.py:38-129 (eval_sequence method)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add MOT format file loading and sequence preparation for E2E evaluation
- Add MOTFrameData dataclass for per-frame detection data
- Add MOTSequenceData dataclass for prepared sequence data ready for metrics
- Add load_mot_file() to parse MOT Challenge format files
- Add prepare_mot_sequence() to compute IoU and remap IDs for evaluation
- Update __init__.py to export new types and functions
- Add comprehensive unit tests (20 tests for io module)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add integration tests for CLEAR metrics validation against TrackEval
- Add test/conftest.py with fixture to download and cache SoccerNet test data
- Add test/eval/test_integration.py with 49 parametrized tests for all sequences
- Add ci-integration-tests.yml workflow that runs only on eval code changes
- Update ci-tests.yml to exclude integration tests from regular CI
- Add integration marker to pyproject.toml pytest config
All 49 sequences pass with exact numerical parity to TrackEval for:
- Integer metrics: CLR_TP, CLR_FN, CLR_FP, IDSW, MT, PT, ML, Frag
- Float metrics: MOTA, MOTP, MTR, PTR, MLR
* Refactor integration tests and improve code quality
- Rename CI job to 'TrackEval Parity Validation' for clarity
- Derive sequence names dynamically from expected_results.json
- Simplify test_io.py from 21 to 10 test cases while maintaining coverage
- Restore useful comments in io.py around ID remapping and IoU computation
- Clean up conftest.py and test_integration.py docstrings
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add high-level evaluation API and CLI for tracker evaluation
Introduce SDK functions and CLI for evaluating multi-object tracking
results against ground truth using MOT Challenge format data.
SDK:
- evaluate_mot_sequence(): Evaluate single sequence, returns CLEAR metrics
- evaluate_benchmark(): Evaluate multiple sequences with aggregation
CLI (beta):
- `trackers eval --gt <file> --tracker <file>` for single sequence
- `trackers eval --gt-dir <dir> --tracker-dir <dir>` for benchmarks
- Optional rich output with `pip install trackers[cli]`
- Beta warning displayed on CLI usage
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add high-level evaluation API and CLI for tracker evaluation
Introduce SDK functions and CLI for evaluating multi-object tracking
results against ground truth using MOT Challenge format data.
SDK:
- evaluate_mot_sequence(): Evaluate single sequence, returns CLEAR metrics
- evaluate_benchmark(): Evaluate multiple sequences with aggregation
CLI (beta):
- `trackers eval --gt <file> --tracker <file>` for single sequence
- `trackers eval --gt-dir <dir> --tracker-dir <dir>` for benchmarks
- Optional rich output with `pip install trackers[cli]`
- Beta warning displayed on CLI usage
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Update integration tests to use SportsMOT and DanceTrack datasets (#221)
* Update integration tests to use SportsMOT and DanceTrack data
Replace SoccerNet test data with SportsMOT (45 sequences) and DanceTrack
(25 sequences) datasets. The tests now validate CLEAR metrics against
TrackEval results for 70 total sequences.
- Update URLs to new GCS-hosted test data zips
- Add multi-dataset fixture support in conftest.py
- Parametrize tests across both datasets
- Fix metric comparison (new format uses fractions, not percentages)
* Fix mypy conftest module name conflict
Use a pytest fixture instead of directly importing from conftest.py to avoid
mypy seeing the file under two module names ("conftest" and "test.conftest").
* Add type annotation for test_cases variable
* Add smart auto-detection and structured result objects to evaluation API (#222)
* Add smart auto-detection and structured result objects to evaluation API
- Remove data_format parameter from SDK and --data-format flag from CLI
- Add smart detection: auto-detect format (flat vs MOT17), benchmark, split, and tracker name
- Log detection results for transparency; error helpfully when ambiguous
- Add BenchmarkResult and SequenceResult dataclasses with json/table/save/load methods
- Add all CLEAR metrics (MOTA, MOTP, MODA, CLR_Re, CLR_Pr, MTR, PTR, MLR, sMOTA, etc.)
- Simplify test fixtures to use auto-detection instead of explicit format parameters
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Simplify CLI table output to use SDK result.table() method
- Remove custom Rich table formatting that truncated columns
- Use result.table(columns) for consistent, readable output
- Set sensible default columns: MOTA, MOTP, IDSW, CLR_FP, CLR_FN, MT, ML
- Remove unused Rich imports and helper functions
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add HOTA (Higher Order Tracking Accuracy) metrics
- Create trackers/eval/hota.py with compute_hota_metrics() and
aggregate_hota_metrics() functions implementing HOTA algorithm
- Add HOTAMetrics dataclass for storing HOTA results
- Update SequenceResult and BenchmarkResult to support optional HOTA metrics
- Integrate HOTA computation into evaluate_mot_sequence and evaluate_benchmark
- Update CLI to support --metrics HOTA option
- Add comprehensive unit tests for HOTA computation and aggregation
HOTA evaluates tracking at multiple IoU thresholds (0.05-0.95) and computes:
- Detection metrics: DetA, DetRe, DetPr
- Association metrics: AssA, AssRe, AssPr
- Combined HOTA = sqrt(DetA * AssA)
- Localization accuracy: LocA
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Add HOTA (Higher Order Tracking Accuracy) metrics
- Create trackers/eval/hota.py with compute_hota_metrics() and
aggregate_hota_metrics() functions implementing HOTA algorithm
- Add HOTAMetrics dataclass for storing HOTA results
- Update SequenceResult and BenchmarkResult to support optional HOTA metrics
- Integrate HOTA computation into evaluate_mot_sequence and evaluate_benchmark
- Update CLI to support --metrics HOTA option
- Add comprehensive unit tests for HOTA computation and aggregation
HOTA evaluates tracking at multiple IoU thresholds (0.05-0.95) and computes:
- Detection metrics: DetA, DetRe, DetPr
- Association metrics: AssA, AssRe, AssPr
- Combined HOTA = sqrt(DetA * AssA)
- Localization accuracy: LocA
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Show all metrics by default in table output
Remove the column limitation that was originally added as a workaround
for Rich table truncation. Now using plain-text tables that don't
truncate, so show all available metrics by default:
- CLEAR-only: 17 metrics
- HOTA+CLEAR: 28 metrics (11 HOTA + 17 CLEAR)
Users can still limit columns via the columns parameter if desired.
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix HOTA aggregation TypeError with numpy arrays
HOTAMetrics.to_dict() was converting numpy arrays to lists for JSON
serialization, but aggregate_hota_metrics() needs numpy arrays for
weighted averaging math.
Added arrays_as_list parameter to to_dict() to control this behavior:
- arrays_as_list=True (default): convert to lists for JSON
- arrays_as_list=False: keep as numpy arrays for aggregation
* Add HOTA verification to integration tests
Update all 4 integration tests to:
- Pass metrics=["CLEAR", "HOTA"] to evaluate_benchmark()
- Verify HOTA metrics (HOTA, DetA, AssA, LocA) against TrackEval expected results
This ensures numerical parity is verified for both CLEAR and HOTA metrics.
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Fix HOTA expected values in test data
Update test data URLs to use regenerated zips with correct HOTA metrics.
The previous expected_results.json files incorrectly used HOTA(0) (value at
alpha=0.05) instead of the mean HOTA across all alpha thresholds.
* Add Identity metrics (IDF1, IDR, IDP) implementation
Implements Identity metrics following TrackEval's algorithm:
- compute_identity_metrics() for single sequence evaluation
- aggregate_identity_metrics() for benchmark aggregation
- IdentityMetrics dataclass for structured results
Updates evaluate_mot_sequence and evaluate_benchmark to support
metrics=["Identity"] alongside CLEAR and HOTA.
Integration tests now verify CLEAR, HOTA, and Identity metrics
against TrackEval precomputed values.
* Add Identity metrics support to CLI
Update --metrics argument to accept Identity alongside CLEAR and HOTA.
* code review pt1
* fix(pre_commit): 🎨 auto format pre-commit hooks
* code review pt2
* Remove unused match_detections utility
The match_detections function was not used by any metric internally.
Each metric (CLEAR, HOTA, Identity) implements specialized matching
logic that cannot be generalized into a shared utility.
Deleted:
- trackers/eval/matching.py
- test/eval/test_matching.py
* Add comprehensive evaluation documentation and API reference (TrackEval Part 8)
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Refine evaluation docs and unify docstring style
- Remove premature CLI and Python quickstart pages
- Fix copy-code.js to handle empty continuation lines (>>>)
- Remove broken links to cli.md from evaluate.md
- Remove duplicate API sections from SORT/ByteTrack pages
- Unify docstring style in results.py to match evaluate.py
- Remove examples from results.py docstrings for consistency
- Standardize attribute documentation format (no backticks on names)
* Revamp documentation with Stripe-style formatting
- Rewrite evaluate.md with concise intro, "What you'll learn" section,
clean tabs for Python/CLI, and collapsible troubleshooting
- Apply same style to install.md for consistency
- Add full-width table CSS styling
- Remove help.md page from navigation
* fix(pre_commit): 🎨 auto format pre-commit hooks
* Revamp documentation with Stripe-style formatting
- Rewrite evaluate.md with concise intro, "What you'll learn" section,
clean tabs for Python/CLI, and collapsible troubleshooting
- Apply same style to install.md for consistency
- Add full-width table CSS styling
- Remove help.md page from navigation
* Add comprehensive evaluation documentation and API reference (TrackEval Part 8)
* fix(pre_commit): 🎨 auto format pre-commit hooks
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* bump version from `2.1.0` to `2.2.0rc0`
* Enable mdformat and codespell pre-commit hooks
- Add mdformat hook with mdformat-mkdocs and mdformat-ruff extensions
- Add codespell hook for spell checking with 'mot' in ignore list
- Format all markdown files with mdformat
- Exclude docs/reference/ from mdformat (auto-generated)
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* docs: improve doctest support and copy-code functionality - Replace copy-code.js with pycon_copy.js from supervision - Strips >>> and ... prompts when copying - Removes output lines from copied code - Handles triple-quoted strings correctly - Enable pytest doctest support in pyproject.toml - Add --doctest-modules to pytest addopts - Add ELLIPSIS and NORMALIZE_WHITESPACE flags - Convert docstring examples to doctest format - box.py: box_iou, box_ioa (runnable) - clear.py: compute_clear_metrics (runnable) - hota.py: compute_hota_metrics (runnable) - identity.py: compute_identity_metrics (runnable) - evaluate.py: examples with +SKIP (require external files) - io.py: examples with +SKIP (require external files) - Wrap examples in ```python fences for mkdocs rendering - Use empty >>> before closing fence to end doctest blocks * docs: add mkdocs hook to strip doctest directives from rendered docs The hook removes # doctest: +SKIP, # doctest: +ELLIPSIS and similar directives from the HTML output so they don't appear in documentation while still being functional for pytest doctest runs. * docs: clean up docstring examples formatting - Remove markdown fences from doctest examples - Use >>> separators instead of blank lines - Simplify box_iou and box_ioa to single examples - Clean up table formatting in evaluate examples - Match variable names to function arguments in prepare_mot_sequence * docs: add Running Tests section to CONTRIBUTING.md - Document unit tests, doctests, integration tests, and all tests commands - Add noqa for long docstring line in io.py
- Add mike version provider to mkdocs.yml for version selector UI - Add `mike set-default latest` to publish-docs workflow - Fix tag patterns in release/pre-release workflows to support double-digit versions
* Fix ruff and mypy linting errors * fix(pre_commit): 🎨 auto format pre-commit hooks * Clean up registration API: mark internal methods, improve docstrings Mark _lookup_tracker and _registered_trackers as internal for CLI use. Rewrite docstrings, remove BaseTrackerWithFeatures, consolidate tests. * fix(pre_commit): 🎨 auto format pre-commit hooks * Add `trackers track` command for video object tracking (#231) * Add track command for video object tracking Implement `trackers track` CLI command combining inference-models detection with SORT/ByteTrack tracking. Features: - RF-DETR Nano as default model via inference-models - Support for video files, webcams, RTSP streams, image directories - Pre-computed detections via --detections (MOT format) - Dynamic tracker params from auto-registration (--tracker.*) - Visualization: boxes, masks, labels, IDs, confidence, trajectories - MOT format output for evaluation pipeline Files: - trackers/scripts/track.py: Main implementation - test/scripts/test_track.py: 29 unit tests - pyproject.toml: Add cli and inference optional deps * fix(pre_commit): 🎨 auto format pre-commit hooks * Fixed. Ruff check passes now. * Improve track command UX and code testability - Namespace model CLI args consistently (--model.device, --model.api_key) - Add device auto-detection for inference-models (cuda/mps/cpu) - Support directory as --output (saves as output.mp4 inside) - Use sv.xywh_to_xyxy for MOT format conversion - Color annotations by tracker ID with custom color palette - Skip labels for unconfirmed tracks (tracker_id == -1) - Remove unused rich dependency Refactor for testability: - _generate_labels: explicit bool params instead of argparse.Namespace - _check_output_writable: single-path validation function - _mot_frame_to_detections: simplified MOTFrameData converter * fix(pre_commit): 🎨 auto format pre-commit hooks * update PR * fix(pre_commit): 🎨 auto format pre-commit hooks * Make device detection and MOT conversion helpers private - Rename best_device to _best_device with lazy torch import - Extract _mot_frame_to_detections as standalone function - Fix VideoOutput mypy error by passing path explicitly - Remove utils API docs (now empty) * Clean up tests and fix mypy type annotations - Fix mypy errors in VideoOutput by adding type annotations for _writer and _create_writer return type - Rename test/test_io to test/io to avoid Python import conflict - Remove duplicate tests from test_track.py (now in test/io/) - Simplify TestFormatLabels using np.array directly in parametrize with descriptive pytest.param ids - Fix trailing whitespace and end-of-file issues (pre-commit) * Refactor IO module: privatize internals and consolidate exports - Make internal IO helpers private: VideoOutput, DisplayWindow, MOTSequenceData, prepare_mot_sequence, validate_output_path, resolve_video_output_path (all prefixed with underscore) - Move public exports (MOTFrameData, MOTOutput, load_mot_file) from trackers.io to main trackers/__init__.py - Rename test/io to test/io_tests to avoid Python built-in conflict - Remove redundant "Tests for ..." docstrings from all test files - Remove examples from private functions - Update API docs to reflect public-only exports * Test * Improve video I/O classes and clean up docstrings _VideoOutput: - Add configurable fps parameter (default 30.0) - Return bool from write() for error detection - Add isOpened() check with OSError on failure _DisplayWindow: - Add resizable window (WINDOW_NORMAL | WINDOW_KEEPRATIO) - Support ESC key (27) in addition to 'q' for quit - Return quit status from show() - Only destroy its own window, not all windows Also: - Remove verbose Examples from frames_from_source docstring - Remove empty utils docstring * Make MOT I/O classes and functions private - Rename MOTOutput to _MOTOutput - Rename load_mot_file to _load_mot_file - Rename MOTFrameData to _MOTFrameData - Remove MOTOutput, load_mot_file, MOTFrameData from public exports in trackers/__init__.py - Update all internal usages to reference private symbols - Remove entries from API docs (only frames_from_source remains public) * Add Track documentation and improve CLI defaults Documentation: - Add comprehensive docs/learn/track.md guide with CLI and Python examples covering Install, Quickstart, Trackers, Detectors, Visualization, Source, Output, and Integration sections - Add Track page to mkdocs navigation - Simplify README.md and docs/index.md code examples - Add CLI Quickstart section to README.md and docs/index.md - Update benchmark table with BoT-SORT and McByte placeholders - Use backticks for library names (trackers, opencv-python, etc.) - Link to new Track guide from Integration sections CLI improvements: - Make --model optional, defaults to rfdetr-nano - Make --show-ids default to True, add --no-ids to disable - Fix webcam input by converting numeric string "0" to int * Improved CLI args formatting. * Add interactive CLI command builder widget to Track docs - Add reusable terminal-style CLI builder framework (cli_builder_framework.js) with state management, event delegation, numeric hold-to-repeat steppers, and MkDocs style syncing - Refactor command_builder.js into a thin config layer on top of the framework for the `trackers track` command - Add command_builder.css with terminal-themed styling, grid alignment, and responsive layout - Add CLI Command Builder section to track.md with interactive widget for model, tracker, visualization, source, and output configuration - Restore demo video embed above Install section - Wire framework and builder assets in mkdocs.yml * fix(pre_commit): 🎨 auto format pre-commit hooks * Remove Integration page. * fix(pre_commit): 🎨 auto format pre-commit hooks --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
updates: - [github.com/astral-sh/ruff-pre-commit: v0.14.14 → v0.15.0](astral-sh/ruff-pre-commit@v0.14.14...v0.15.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Skalski <[email protected]>
Bumps the uv group with 1 update in the / directory: [cryptography](https://github.com/pyca/cryptography). Updates `cryptography` from 44.0.3 to 46.0.5 - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](pyca/cryptography@44.0.3...46.0.5) --- updated-dependencies: - dependency-name: cryptography dependency-version: 46.0.5 dependency-type: indirect dependency-group: uv ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Piotr Skalski <[email protected]>
Bumps [types-tqdm](https://github.com/typeshed-internal/stub_uploader) from 4.67.0.20250809 to 4.67.3.20260205. - [Commits](https://github.com/typeshed-internal/stub_uploader/commits) --- updated-dependencies: - dependency-name: types-tqdm dependency-version: 4.67.3.20260205 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Piotr Skalski <[email protected]>
Bumps the uv group with 1 update in the / directory: [pillow](https://github.com/python-pillow/Pillow). Updates `pillow` from 11.3.0 to 12.1.1 - [Release notes](https://github.com/python-pillow/Pillow/releases) - [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst) - [Commits](python-pillow/Pillow@11.3.0...12.1.1) --- updated-dependencies: - dependency-name: pillow dependency-version: 12.1.1 dependency-type: indirect dependency-group: uv ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [mypy](https://github.com/python/mypy) from 1.15.0 to 1.19.1. - [Changelog](https://github.com/python/mypy/blob/master/CHANGELOG.md) - [Commits](python/mypy@v1.15.0...v1.19.1) --- updated-dependencies: - dependency-name: mypy dependency-version: 1.19.1 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.11.9 to 0.15.0. - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md) - [Commits](astral-sh/ruff@0.11.9...0.15.0) --- updated-dependencies: - dependency-name: ruff dependency-version: 0.15.0 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [types-aiofiles](https://github.com/typeshed-internal/stub_uploader) from 24.1.0.20250326 to 25.1.0.20251011. - [Commits](https://github.com/typeshed-internal/stub_uploader/commits) --- updated-dependencies: - dependency-name: types-aiofiles dependency-version: 25.1.0.20251011 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Piotr Skalski <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changelog
🚀 Added
TODO
🏆 Contributors
TODO