Add DEIMV2 Object Detection Model

[kprokofi]

Summary

resolves #5015

• Add DinoV3 and VIT tiny as a backbones for detection, primarily for DeimV2 model
• Add DEIMV2 model (OTXModel, Encoder, Decoder), e2e training, export
• Experiment with pre-processing, Copy-blend, EMA, learning rate and its schedule, model weights
• Add Unit tests, perf tests
• Provide final benchmark numbers (vs other DETR variants)

How to test

otx train --config src/otx/recipe/detection/deimv2_l.yaml --data_root tests/assests/car_tree_bug

Checklist

• The PR title and description are clear and descriptive
• I have manually tested the changes
• All changes are covered by automated tests
• All related issues are linked to this PR (if applicable)
• Documentation has been updated (if applicable)

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 83.84189% with 233 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
.../native/models/common/layers/transformer_layers.py	76.54%	76 Missing ⚠️
...ive/models/detection/necks/dfine_hybrid_encoder.py	72.41%	48 Missing ⚠️
...kend/native/models/detection/heads/deim_decoder.py	91.46%	28 Missing ⚠️
...kend/native/models/common/layers/position_embed.py	66.66%	26 Missing ⚠️
...x/backend/native/models/common/backbones/dinov3.py	91.87%	16 Missing ⚠️
...brary/src/otx/backend/native/models/utils/utils.py	14.28%	12 Missing ⚠️
...end/native/models/detection/backbones/dinov3sta.py	88.09%	10 Missing ⚠️
...c/otx/backend/native/models/modules/transformer.py	85.71%	5 Missing ⚠️
...backend/native/models/common/losses/gfocal_loss.py	42.85%	4 Missing ⚠️
library/src/otx/backend/native/models/base.py	25.00%	3 Missing ⚠️
... and 2 more

📢 Thoughts on this report? Let us know!

[kprokofi]

Model Manifests to be updated after a decision regarding the DETR models we want to expose

[Copilot]

Pull request overview

This PR adds the DEIMv2 object detection model to the OTX training extensions platform. DEIMv2 is an improved detection transformer that combines a DINOv3 backbone with spatial token attention (STA) and fine-grained distribution refinement (FDR) for enhanced object detection performance.

Key Changes:

• Added DEIMv2 model architecture with DINOv3/ViT-Tiny backbone and STA module
• Implemented transformer decoder with FDR for bounding box regression
• Added comprehensive unit tests and performance benchmarks
• Introduced data augmentation scheduling and multi-scale training support

Reviewed changes

Copilot reviewed 43 out of 43 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
library/src/otx/backend/native/models/detection/deimv2.py	DEIMv2 model class with factory pattern for model variants (x/l/m/s)
library/src/otx/backend/native/models/detection/backbones/dinov3sta.py	DINOv3 backbone with Spatial Token Attention for multi-scale features
library/src/otx/backend/native/models/detection/heads/deim_decoder.py	DEIM transformer decoder with FDR mechanism
library/src/otx/backend/native/models/detection/necks/dfine_hybrid_encoder.py	Hybrid encoder with FPN/PAN for feature fusion
library/src/otx/recipe/detection/deimv2_*.yaml	Training recipes for all DEIMv2 variants
library/tests/unit/backend/native/models/detection/test_deimv2.py	Comprehensive unit tests for DEIMv2 model
library/tests/perf_v2/tasks/detection.py	Performance test configuration for DEIMv2 variants

Comments suppressed due to low confidence (1)

library/src/otx/backend/native/models/detection/backbones/dinov3sta.py:1

• Duplicate code: lines 533-536 and 537-539 both check if self.eval_spatial_size and generate anchors. The second block overwrites the registered buffers from the first block. Remove the first block (lines 533-536) as it's redundant.

# Copyright (C) 2025 Intel Corporation

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[leoll2]

The policy is called no_aug, which means "no augmentation" if my assumption is correct. However, it uses RandomFlip which is an augmentation. Is this intended?

[kprokofi]

RandomFlip is very basic augmentation that gently enlarge training distribution. It is common to always include this augmentation as default. However, there is no experimental proof of it, just common thing to use.