Overwrite the weights in load_from with new backbone weights

[UnglvKitDe]

Prerequisite

• I have searched the existing and past issues but cannot get the expected help.
• I have read the FAQ documentation but cannot get the expected help.
• The bug has not been fixed in the latest version.

💬 Describe the reimplementation questions

Hello all,

I am currently trying to replace the backbone with a ResNeSt on one of the TOOD models.
Is there any way I can keep the weights of the original trained TOOD-Net with ResNet101 and just load in the weights of the new backbone (practically overwriting the weights of ResNet)?

tood_r101_fpn_dconv_c3-c5_mstrain_2x_custom.py:

_base_ = './tood_r101_fpn_mstrain_2x_coco.py'

model = dict(
    backbone=dict(
        dcn=dict(type='DCNv2', deformable_groups=1, fallback_on_stride=False),
        stage_with_dcn=(False, True, True, True),
        
        # RestNeSt 101
        stem_channels=128, 
        depth=101,
        init_cfg=dict(type='Pretrained',
                      checkpoint='open-mmlab://resnest101')),
    bbox_head=dict(num_dcn=2))

load_from = 'https://download.openmmlab.com/mmdetection/v2.0/tood/tood_r101_fpn_dconv_c3-c5_mstrain_2x_coco/tood_r101_fpn_dconv_c3-c5_mstrain_2x_coco_20211210_213728-4a824142.pth'

However, it seems that it is not the weights from the backbone that are initialized, but the ones from load_from. The output of the training with python train.py [config]:

2022-10-18 02:41:11,063 - mmdet - INFO - load checkpoint from http path: https://download.openmmlab.com/mmdetection/v2.0/tood/tood_r101_fpn_dconv_c3-c5_
mstrain_2x_coco/tood_r101_fpn_dconv_c3-c5_mstrain_2x_coco_20211210_213728-4a824142.pth
Downloading: "https://download.openmmlab.com/mmdetection/v2.0/tood/tood_r101_fpn_dconv_c3-c5_mstrain_2x_coco/tood_r101_fpn_dconv_c3-c5_mstrain_2x_coco_$
0211210_213728-4a824142.pth" to /home/ies/nfriederich/.cache/torch/hub/checkpoints/tood_r101_fpn_dconv_c3-c5_mstrain_2x_coco_20211210_213728-4a824142.p$
h
100.0%
2022-10-18 02:41:28,498 - root - INFO - ModulatedDeformConvPack backbone.layer2.0.conv2 is upgraded to version 2.
2022-10-18 02:41:28,500 - root - INFO - ModulatedDeformConvPack backbone.layer2.1.conv2 is upgraded to version 2.
2022-10-18 02:41:28,502 - root - INFO - ModulatedDeformConvPack backbone.layer2.2.conv2 is upgraded to version 2.
2022-10-18 02:41:28,504 - root - INFO - ModulatedDeformConvPack backbone.layer2.3.conv2 is upgraded to version 2.
2022-10-18 02:41:28,506 - root - INFO - ModulatedDeformConvPack backbone.layer3.0.conv2 is upgraded to version 2.
2022-10-18 02:41:28,510 - root - INFO - ModulatedDeformConvPack backbone.layer3.1.conv2 is upgraded to version 2.
2022-10-18 02:41:28,512 - root - INFO - ModulatedDeformConvPack backbone.layer3.2.conv2 is upgraded to version 2.
2022-10-18 02:41:28,515 - root - INFO - ModulatedDeformConvPack backbone.layer3.3.conv2 is upgraded to version 2.
2022-10-18 02:41:28,517 - root - INFO - ModulatedDeformConvPack backbone.layer3.4.conv2 is upgraded to version 2.
2022-10-18 02:41:28,520 - root - INFO - ModulatedDeformConvPack backbone.layer3.5.conv2 is upgraded to version 2.
2022-10-18 02:41:28,522 - root - INFO - ModulatedDeformConvPack backbone.layer3.6.conv2 is upgraded to version 2.
2022-10-18 02:41:28,525 - root - INFO - ModulatedDeformConvPack backbone.layer3.7.conv2 is upgraded to version 2.
2022-10-18 02:41:28,527 - root - INFO - ModulatedDeformConvPack backbone.layer3.8.conv2 is upgraded to version 2.
2022-10-18 02:41:28,530 - root - INFO - ModulatedDeformConvPack backbone.layer3.9.conv2 is upgraded to version 2.
2022-10-18 02:41:28,532 - root - INFO - ModulatedDeformConvPack backbone.layer3.10.conv2 is upgraded to version 2.
2022-10-18 02:41:28,535 - root - INFO - ModulatedDeformConvPack backbone.layer3.11.conv2 is upgraded to version 2.
2022-10-18 02:41:28,537 - root - INFO - ModulatedDeformConvPack backbone.layer3.12.conv2 is upgraded to version 2.
2022-10-18 02:41:28,540 - root - INFO - ModulatedDeformConvPack backbone.layer3.13.conv2 is upgraded to version 2.
2022-10-18 02:41:28,542 - root - INFO - ModulatedDeformConvPack backbone.layer3.14.conv2 is upgraded to version 2.
2022-10-18 02:41:28,544 - root - INFO - ModulatedDeformConvPack backbone.layer3.15.conv2 is upgraded to version 2.
2022-10-18 02:41:28,547 - root - INFO - ModulatedDeformConvPack backbone.layer3.16.conv2 is upgraded to version 2.
2022-10-18 02:41:28,549 - root - INFO - ModulatedDeformConvPack backbone.layer3.17.conv2 is upgraded to version 2.
2022-10-18 02:41:28,552 - root - INFO - ModulatedDeformConvPack backbone.layer3.18.conv2 is upgraded to version 2.
2022-10-18 02:41:28,554 - root - INFO - ModulatedDeformConvPack backbone.layer3.19.conv2 is upgraded to version 2.
2022-10-18 02:41:28,557 - root - INFO - ModulatedDeformConvPack backbone.layer3.20.conv2 is upgraded to version 2.
2022-10-18 02:41:28,559 - root - INFO - ModulatedDeformConvPack backbone.layer3.21.conv2 is upgraded to version 2.
2022-10-18 02:41:28,562 - root - INFO - ModulatedDeformConvPack backbone.layer3.22.conv2 is upgraded to version 2.
2022-10-18 02:41:28,565 - root - INFO - ModulatedDeformConvPack backbone.layer4.0.conv2 is upgraded to version 2.
2022-10-18 02:41:28,570 - root - INFO - ModulatedDeformConvPack backbone.layer4.1.conv2 is upgraded to version 2.
2022-10-18 02:41:28,575 - root - INFO - ModulatedDeformConvPack backbone.layer4.2.conv2 is upgraded to version 2.
2022-10-18 02:41:28,584 - root - INFO - ModulatedDeformConvPack bbox_head.inter_convs.0.conv is upgraded to version 2.
2022-10-18 02:41:28,585 - root - INFO - ModulatedDeformConvPack bbox_head.inter_convs.1.conv is upgraded to version 2.
2022-10-18 02:41:28,595 - mmdet - WARNING - The model and loaded state dict do not match exactly

size mismatch for backbone.conv1.weight: copying a param with shape torch.Size([64, 3, 7, 7]) from checkpoint, the shape in current model is torch.Size(
[128, 3, 7, 7]).
size mismatch for backbone.bn1.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.bn1.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone.bn1.running_mean: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]
).
size mismatch for backbone.bn1.running_var: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128])
.
size mismatch for backbone.layer1.0.conv1.weight: copying a param with shape torch.Size([64, 64, 1, 1]) from checkpoint, the shape in current model is t
orch.Size([64, 128, 1, 1]).
size mismatch for backbone.layer1.0.downsample.0.weight: copying a param with shape torch.Size([256, 64, 1, 1]) from checkpoint, the shape in current mo
del is torch.Size([256, 128, 1, 1]).
size mismatch for bbox_head.tood_cls.weight: copying a param with shape torch.Size([80, 256, 3, 3]) from checkpoint, the shape in current model is torch
.Size([2, 256, 3, 3]).

Thx a lot :)

Environment

Using Pytorch 1.13 and conda

Expected results

No response

Additional information

Config see above

[ZwwWayne]

You can set init_cfg in bbox_head. For example:

    bbox_head=dict(
        num_dcn=2,
        init_cfg=dict(
            type='Pretrained',
            checkpoint='https://download.openmmlab.com/mmdetection/v2.0/tood/tood_r101_fpn_dconv_c3-c5_mstrain_2x_coco/tood_r101_fpn_dconv_c3-c5_mstrain_2x_coco_20211210_213728-4a824142.pth'))

[UnglvKitDe]

Got it.Thx!