Weights loading warnings

[Instinct323]

Hello! I've recently been trying to use your model, but I'm encountering some issues when loading the weights. I'd really appreciate it if you could take a quick look for me!

Here is my code:

import sys
import warnings

import matplotlib.pyplot as plt
import supervision as sv
import torch
from detectron2.config import get_cfg
from detectron2.data.detection_utils import read_image
from detectron2.engine.defaults import DefaultPredictor

sys.path.insert(0, "third_party/CenterNet2")
from centernet.config import add_centernet_config
from codet.config import add_codet_config

warnings.filterwarnings("ignore")


def setup_cfg(file):
    confidence_threshold = 0.5
    pred_all_class = False

    cfg = get_cfg()
    if not torch.cuda.is_available(): cfg.MODEL.DEVICE = "cpu"
    add_centernet_config(cfg)
    add_codet_config(cfg)
    cfg.merge_from_file(file)
    # Set score_threshold for builtin models
    cfg.MODEL.RETINANET.SCORE_THRESH_TEST = cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = (
        cfg.MODEL.PANOPTIC_FPN.COMBINE).INSTANCES_CONFIDENCE_THRESH = confidence_threshold
    cfg.MODEL.ROI_BOX_HEAD.ZEROSHOT_WEIGHT_PATH = 'rand'  # load later
    if not pred_all_class:
        cfg.MODEL.ROI_HEADS.ONE_CLASS_PER_PROPOSAL = True
    cfg.freeze()
    return cfg


def to_detections(result: dict):
    xyxy = result["instances"].get_fields()["pred_boxes"].tensor.cpu().data.numpy()
    return sv.Detections(xyxy=xyxy)


predictor = DefaultPredictor(setup_cfg("configs/CoDet_OVLVIS_SwinB_4x_ft4x.yaml"))
img = read_image("/media/tongzj/Data/Workbench/ModelsAPI/assets/desktop-c.png", format="BGR")

dets = to_detections(predictor(img))
anno = sv.BoxAnnotator(color_lookup=sv.ColorLookup.INDEX)
res = anno.annotate(img.copy(), dets)
plt.imshow(res)
plt.show()

The relevant configuration is as follows:

• MODEL.WEIGHTS: CoDet_OVLVIS_SwinB_4x_ft4x.pth, and I can confirm that the weight path is correct
• MODEL.CAT_FREQ_PATH: commented out, since I don't plan to train the model right now
  The warning message I received is:

Skip loading parameter 'roi_heads.box_predictor.0.cls_score.zs_weight' to the model due to incompatible shapes: (512, 5910) in the checkpoint but (512, 1204) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.0.fc1.weight' to the model due to incompatible shapes: (64, 448) in the checkpoint but (128, 896) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.0.fc1.bias' to the model due to incompatible shapes: (64,) in the checkpoint but (128,) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.0.fc2.weight' to the model due to incompatible shapes: (1, 64) in the checkpoint but (1, 128) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.0.weight_transform.0.weight' to the model due to incompatible shapes: (64, 448) in the checkpoint but (128, 896) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.0.weight_transform.0.bias' to the model due to incompatible shapes: (64,) in the checkpoint but (128,) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.0.weight_transform.2.weight' to the model due to incompatible shapes: (1, 64) in the checkpoint but (1, 128) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.1.cls_score.zs_weight' to the model due to incompatible shapes: (512, 5910) in the checkpoint but (512, 1204) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.1.fc1.weight' to the model due to incompatible shapes: (64, 448) in the checkpoint but (128, 896) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.1.fc1.bias' to the model due to incompatible shapes: (64,) in the checkpoint but (128,) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.1.fc2.weight' to the model due to incompatible shapes: (1, 64) in the checkpoint but (1, 128) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.1.weight_transform.0.weight' to the model due to incompatible shapes: (64, 448) in the checkpoint but (128, 896) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.1.weight_transform.0.bias' to the model due to incompatible shapes: (64,) in the checkpoint but (128,) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.1.weight_transform.2.weight' to the model due to incompatible shapes: (1, 64) in the checkpoint but (1, 128) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.2.cls_score.zs_weight' to the model due to incompatible shapes: (512, 5910) in the checkpoint but (512, 1204) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.2.fc1.weight' to the model due to incompatible shapes: (64, 448) in the checkpoint but (128, 896) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.2.fc1.bias' to the model due to incompatible shapes: (64,) in the checkpoint but (128,) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.2.fc2.weight' to the model due to incompatible shapes: (1, 64) in the checkpoint but (1, 128) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.2.weight_transform.0.weight' to the model due to incompatible shapes: (64, 448) in the checkpoint but (128, 896) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.2.weight_transform.0.bias' to the model due to incompatible shapes: (64,) in the checkpoint but (128,) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.2.weight_transform.2.weight' to the model due to incompatible shapes: (1, 64) in the checkpoint but (1, 128) in the model! You might want to double check if this is expected.
Some model parameters or buffers are not found in the checkpoint:
roi_heads.box_predictor.0.cls_score.{detection_weight, zs_weight}
roi_heads.box_predictor.0.fc1.{bias, weight}
roi_heads.box_predictor.0.fc2.weight
roi_heads.box_predictor.0.weight_transform.0.{bias, weight}
roi_heads.box_predictor.0.weight_transform.2.weight
roi_heads.box_predictor.1.cls_score.{detection_weight, zs_weight}
roi_heads.box_predictor.1.fc1.{bias, weight}
roi_heads.box_predictor.1.fc2.weight
roi_heads.box_predictor.1.weight_transform.0.{bias, weight}
roi_heads.box_predictor.1.weight_transform.2.weight
roi_heads.box_predictor.2.cls_score.{detection_weight, zs_weight}
roi_heads.box_predictor.2.fc1.{bias, weight}
roi_heads.box_predictor.2.fc2.weight
roi_heads.box_predictor.2.weight_transform.0.{bias, weight}
roi_heads.box_predictor.2.weight_transform.2.weight

It seems that some weights were not loaded correctly. However, this doesn't appear to affect the model's ability to produce correct results.

Can I safely ignore these warnings? Or what would be the recommended way to resolve this issue?