[Feature] Support TTA

[zytx121]

mmdet >= 3.0.0rc6

python tools/test.py \
    configs/rotated_rtmdet/rotated_rtmdet_l-3x-dota.py \
    work_dirs/rotated_rtmdet_l-3x-dota/epoch_36.pth \
    --tta

[jamiechoi1995]

Hi, I adapt your code but got the following error:
my mmcv==2.0.0 and mmdet==3.0.0
I use rtmdet

Traceback (most recent call last):
  File "tools/test.py", line 145, in <module>
    main()
  File "tools/test.py", line 141, in main
    runner.test()
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1767, in test
    metrics = self.test_loop.run()  # type: ignore
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/mmengine/runner/loops.py", line 435, in run
    self.run_iter(idx, data_batch)
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/mmengine/runner/loops.py", line 454, in run_iter
    outputs = self.runner.model.test_step(data_batch)
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/mmengine/model/test_time_aug.py", line 140, in test_step
    predictions.append(self.module.test_step(data))
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 145, in test_step
    return self._run_forward(data, mode='predict')  # type: ignore
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 340, in _run_forward
    results = self(**data, mode=mode)
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "mmdetection-3.0.0/mmdet/models/detectors/base.py", line 94, in forward
    return self.predict(inputs, data_samples)
  File "mmdetection-3.0.0/mmdet/models/detectors/single_stage.py", line 109, in predict
    x = self.extract_feat(batch_inputs)
  File "mmdetection-3.0.0/mmdet/models/detectors/single_stage.py", line 148, in extract_feat
    x = self.neck(x)
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "mmdetection-3.0.0/mmdet/models/necks/cspnext_pafpn.py", line 153, in forward
    torch.cat([upsample_feat, feat_low], 1))
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 76 but got size 75 for tensor number 1 in the list.

Could you please take a look at this?

[liuyanyi]

Hi, I adapt your code but got the following error: my mmcv==2.0.0 and mmdet==3.0.0 I use ppyoloe

Traceback (most recent call last):
  File "tools/test.py", line 145, in <module>
    main()
  File "tools/test.py", line 141, in main
    runner.test()
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1767, in test
    metrics = self.test_loop.run()  # type: ignore
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/mmengine/runner/loops.py", line 435, in run
    self.run_iter(idx, data_batch)
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/mmengine/runner/loops.py", line 454, in run_iter
    outputs = self.runner.model.test_step(data_batch)
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/mmengine/model/test_time_aug.py", line 140, in test_step
    predictions.append(self.module.test_step(data))
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 145, in test_step
    return self._run_forward(data, mode='predict')  # type: ignore
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 340, in _run_forward
    results = self(**data, mode=mode)
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "mmdetection-3.0.0/mmdet/models/detectors/base.py", line 94, in forward
    return self.predict(inputs, data_samples)
  File "mmdetection-3.0.0/mmdet/models/detectors/single_stage.py", line 109, in predict
    x = self.extract_feat(batch_inputs)
  File "mmdetection-3.0.0/mmdet/models/detectors/single_stage.py", line 148, in extract_feat
    x = self.neck(x)
  File "/opt/conda/envs/mmdet3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "mmdetection-3.0.0/mmdet/models/necks/cspnext_pafpn.py", line 153, in forward
    torch.cat([upsample_feat, feat_low], 1))
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 76 but got size 75 for tensor number 1 in the list.

Could you please take a look at this?

RTMDet can't handle image size 1200, because it cant divide by 32, you change 1200 to 1280 in tta config

[jamiechoi1995]

RTMDet can't handle image size 1200, because it cant divide by 32, you change 1200 to 1280 in tta config

It works but another problem occurs when calculating mAP, I found that the gts in the following function is

({}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {})

mmrotate/mmrotate/evaluation/metrics/dota_metric.py

Lines 297 to 307 in 8b30525

	def compute_metrics(self, results: list) -> dict:
	"""Compute the metrics from processed results.
	Args:
	results (list): The processed results of each batch.
	Returns:
	dict: The computed metrics. The keys are the names of the metrics,
	and the values are corresponding results.
	"""
	logger: MMLogger = MMLogger.get_current_instance()
	gts, preds = zip(*results)

[chelseaztq]

RTMDet can't handle image size 1200, because it cant divide by 32, you change 1200 to 1280 in tta config

It works but another problem occurs when calculating mAP, I found that the gts in the following function is

({}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {})

mmrotate/mmrotate/evaluation/metrics/dota_metric.py

Lines 297 to 307 in 8b30525

	def compute_metrics(self, results: list) -> dict:
	"""Compute the metrics from processed results.
	Args:
	results (list): The processed results of each batch.
	Returns:
	dict: The computed metrics. The keys are the names of the metrics,
	and the values are corresponding results.
	"""
	logger: MMLogger = MMLogger.get_current_instance()
	gts, preds = zip(*results)

maybe you should change config like this
tta_model = dict(
type='RotatedTTAModel',
tta_cfg=dict(nms=dict(type='nms_rotated', iou_threshold=0.1), max_per_img=2000))

img_scales = [(1024, 1024), (800, 800), (1280, 1280) , (640,640),(1344,1344),(1534,1534)]
tta_pipeline = [
dict(type='mmdet.LoadImageFromFile', file_client_args=dict(backend='disk')),
dict(
type='mmdet.TestTimeAug',
transforms=[
[
dict(type='mmdet.Resize', scale=s, keep_ratio=True)
for s in img_scales
],
[dict(type='mmdet.LoadAnnotations', with_bbox=True, box_type='qbox')]
,
[dict(type='ConvertBoxType', box_type_mapping=dict(gt_bboxes='rbox'))]
,
# [
# # # RandomFlip must be placed before Pad, otherwise
# # # bounding box coordinates after flipping cannot be
# # # recovered correctly.
# # dict(type='mmdet.RandomFlip', prob=1.),
# # dict(type='mmdet.RandomFlip', prob=0.)
# ],
[
dict(
type='mmdet.Pad',
size=(1534, 1534),
pad_val=dict(img=(114, 114, 114))),
],
[
dict(
type='mmdet.PackDetInputs',
meta_keys=('img_id', 'img_path', 'ori_shape', 'img_shape',
'scale_factor'))
]
])
]

add [dict(type='mmdet.LoadAnnotations', with_bbox=True, box_type='qbox')]
,
[dict(type='ConvertBoxType', box_type_mapping=dict(gt_bboxes='rbox'))]
,
and change the val data address in your config

[QiaoLiuHit]

RTMDet can't handle image size 1200, because it cant divide by 32, you change 1200 to 1280 in tta config

It works but another problem occurs when calculating mAP, I found that the gts in the following function is

({}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {})

mmrotate/mmrotate/evaluation/metrics/dota_metric.py

Lines 297 to 307 in 8b30525

	def compute_metrics(self, results: list) -> dict:
	"""Compute the metrics from processed results.
	Args:
	results (list): The processed results of each batch.
	Returns:
	dict: The computed metrics. The keys are the names of the metrics,
	and the values are corresponding results.
	"""
	logger: MMLogger = MMLogger.get_current_instance()
	gts, preds = zip(*results)

@jamiechoi1995 did you solve this mAP calculation problem? I have the same problem.

[QiaoLiuHit]

RTMDet can't handle image size 1200, because it cant divide by 32, you change 1200 to 1280 in tta config

It works but another problem occurs when calculating mAP, I found that the gts in the following function is

({}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {})

mmrotate/mmrotate/evaluation/metrics/dota_metric.py

Lines 297 to 307 in 8b30525

	def compute_metrics(self, results: list) -> dict:
	"""Compute the metrics from processed results.
	Args:
	results (list): The processed results of each batch.
	Returns:
	dict: The computed metrics. The keys are the names of the metrics,
	and the values are corresponding results.
	"""
	logger: MMLogger = MMLogger.get_current_instance()
	gts, preds = zip(*results)

@jamiechoi1995 did you solve this mAP calculation problem? I have the same problem.

I have solve the problem. It should put the 'mmdet.LoadAnnotations' and 'ConvertBoxType' behind the 'mmdet.RandomFlip'.