About pretrained model

[Geuan666]

I want to know the https://drive.google.com/file/d/1lYrAg5qzFbNghEW-3UB36v9WL-mo5eN9/view?pli=1 if the right pretrained model? Because I download it and use it as the checkpoint.pt ,however it lack for the "epoch" and "state_dict"?

[sStonemason]

The model we provide does not contain "epoch" and "state_dict" keys. You load the information about state_dict directly, as shown below:
sd = {k: v for k, v in checkpoint.items() if "bert.pooler" not in k} ✔
sd = {k: v for k, v in checkpoint["state_dict"].items() if "bert.pooler" not in k} ❌

[Geuan666]

I have a competition based on ODIR-5k. I`d like to finetune the model you provided. Is there someting, I need to care? And I find that Only Using Vit( from RET-CLIP) to do few-shot made a great progress! AUPR achieved 0.817! Look forward to your response.

[sStonemason]

Sorry for the late reply.

You can think of our base model as a general-purpose image feature extractor, where you can perform any operation you want on top of the features, such as the linear layer for classification tasks, or the segmentation head for segmentation tasks. In fact, you don't need to apply special tricks, and our model can achieve superior performance by just fine-tuning the classification head. If you can explain your task in more detail, I would be happy to help in more detail!