resume_form_checkpoint with different hyperparameters

[sirtris]

Hi,
I am training a classifier with a pretrained backbone. I froze the backbone and only trained a linear classifier for some epochs. Now I want to continue training with an unfrozen backbone and a smaller learning rate.

Can this be done by just just using the resume_from_checkpoint argument in the trainer or will the encoder still be frozen?
If it is not possible what would the simplest way to achieve my goal be?

Thanks

[rohitgr7]

Can this be done by just just using the resume_from_checkpoint argument in the trainer or will the encoder still be frozen?

no, since Trainer doesn't know what your backbone is and it doesn't freeze any module manually.
you can check out BackboneFinetuning for your usecase.

[sirtris]

k, THX.

So the checkpoint does not save the requires_grad parameter of a layer?

[rohitgr7]

it just saves what model.state_dict returns which comes from pytorch and AFAIK it doesn't. But you can verify

[sirtris]

Thanks for pointing out the BackboneFinetuning callback. It seems to do exactly what I was trying to do.