Can I use a pretrained classifier for object detection / instance segmentation

[umerhasan17]

I finetuned a pre-trained ResNet classifier and feel as if that will add some meaningful value to my object detection task which is very similar. I would like to use that pth file as the model being trained in detectron2 and compare the results. At the moment, I can only figure out how to use models already in the model zoo. I was wondering how I could use my custom models. I would also be happy to contribute this extra information to the documentation as I wasn't able to find a clear method.

I am currently using the example below, and I would just like to change the first line to use my model (likely means changing the 3rd line). Are there any limitations to consider when selecting which type of model it should be?

pretrained_model = "COCO-Detection/faster_rcnn_R_101_FPN_3x.yaml"
cfg = get_cfg()
cfg.merge_from_file(model_zoo.get_config_file(pretrained_model))
cfg.DATASETS.TRAIN = ("train_dataset",)
cfg.DATASETS.TEST = ()
cfg.DATALOADER.NUM_WORKERS = 2
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(pretrained_model)  # Let training initialize from model zoo
cfg.SOLVER.IMS_PER_BATCH = 2
cfg.SOLVER.BASE_LR = 0.00025  # pick a good LR
cfg.SOLVER.MAX_ITER = 500    # 300 iterations seems good enough for this toy dataset; you will need to train longer for a practical dataset
cfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 64   # faster, and good enough for this toy dataset (default: 512)
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 1  # only has one class (ballon). (see https://detectron2.readthedocs.io/tutorials/datasets.html#update-the-config-for-new-datasets)
# NOTE: this config means the number of classes, but a few popular unofficial tutorials incorrect uses num_classes+1 here.

os.makedirs(cfg.OUTPUT_DIR, exist_ok=True)
trainer = DefaultTrainer(cfg) 
trainer.resume_or_load(resume=False)
trainer.train()

[jdegeus]

Where is your trained model coming from? If you made your own config file with which you trained your model, you can do the following:

cfg.merge_from_file("path to config file")
cfg.MODEL.WEIGHTS = "path to .pth file"

[umerhasan17]

@jdegeus Thank you for your reply. I didn't use detectron2 but I did use pytorch. Do you think it might be possible to use this straight out of the box as I have the .pth file? If not, how is it possible to recreate this using a detectron2 config file?

This is the code:

model = models.resnet50(pretrained=True)

# Freeze all layers in model
for param in model.parameters():
    param.requires_grad = False
    
# Add the following layers 
model.fc = nn.Sequential(
    nn.Linear(2048, 512),
    nn.ReLU(),
    nn.Dropout(0.2),
    nn.Linear(512, len(trainloader.dataset.classes)),
    nn.LogSoftmax(dim=1))