retrain pretrained model

[shandongchong]

hello Thank you very much for your open project. My custom data is unusual I want to train the new model and updata the weights of pretrained model. could u give me some suggestions. whether my idea is feasibility?

[ashwinvaidya17]

@ccsself Do you want to update the weights of only the backbone or do you want to load weights for the entire model? If you want the entire model then you can have a look at trainer.resume_from_checkpoint key in the config.yaml file. You can just set this parameter with the path to the trained model.

[xiaoyunwei]

If the config remains the same, will the new training resume from the existing model like "model.ckpt" or "model-v1.ckpt" by default?

[xiaoyunwei]

I tried the following setting:
resume_from_checkpoint: D:\GitHub\anomalib-main\results\cfa\mvtec\wood\run\weights\model-v1.ckpt

And I got the following error in "trainer.fit(model=model, datamodule=datamodule)".

Traceback (most recent call last): | 0/21 [00:00<?, ?it/s]
self._run(model, ckpt_path=self.ckpt_path)
File "C:\Users\MSI-NB.conda\envs\anomalib\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1112, in _run
results = self._run_stage()
File "C:\Users\MSI-NB.conda\envs\anomalib\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1191, in _run_stage
self._run_train()
File "C:\Users\MSI-NB.conda\envs\anomalib\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1214, in _run_train
self.fit_loop.run()
File "C:\Users\MSI-NB.conda\envs\anomalib\lib\site-packages\pytorch_lightning\loops\loop.py", line 194, in run
self.on_run_start(*args, **kwargs)
File "C:\Users\MSI-NB.conda\envs\anomalib\lib\site-packages\pytorch_lightning\loops\fit_loop.py", line 218, in on_run_start
self.trainer._call_lightning_module_hook("on_train_start")
File "C:\Users\MSI-NB.conda\envs\anomalib\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1356, in _call_lightning_module_hook
output = fn(*args, **kwargs)
File "C:\Users\MSI-NB.conda\envs\anomalib\lib\site-packages\anomalib\models\cfa\lightning_model.py", line 73, in on_train_start
self.model.initialize_centroid(data_loader=self.trainer.datamodule.train_dataloader()) # type: ignore
File "C:\Users\MSI-NB.conda\envs\anomalib\lib\site-packages\anomalib\models\cfa\torch_model.py", line 158, in initialize_centroid
self.memory_bank = ((self.memory_bank * i) + target_features.mean(dim=0, keepdim=True)) / (i + 1)
RuntimeError: The size of tensor a (3136) must match the size of tensor b (56) at non-singleton dimension 3
Training: 21it [00:02, ?it/s]

Any suggestion?

Best