when I configure callbacks in the model instead of in the Trainer function, will the model look for the best model or the current model when testing?

[Struggle-Forever]

A question: when I configure callbacks in the model instead of in the Train function, will the model look for the best model or the current model when testing?

For example: configure the configure_callbacks in my model
`

    def configure_callbacks(self):
        early_stop_callback = EarlyStopping(monitor="val_f1", min_delta=0.00, patience=self.args.patience,
                                            verbose=False, mode="max")
        checkpoint_callback = ModelCheckpoint(monitor='val_f1',
                                              dirpath="{}".format(self.save_log_path),
                                              filename='best_{}'.format(self.index_times),
                                              save_top_k=1,
                                              mode='max',
                                              save_last=False)
        return [checkpoint_callback, early_stop_callback]

    def on_train_start(self):
        self.print("Training is started!")

    def on_train_end(self):
        self.print("Training is done!")

`

Not configure the configure_callbacks in the Trainer
trainer = Trainer(devices="auto", accelerator="auto", logger=False, multiple_trainloader_mode='max_size_cycle', terminate_on_nan=True, logger=tb_logger, log_every_n_steps=1, flush_logs_every_n_steps=5, max_epochs=args.epochs)

When i perform the trainer like this:
`

    trainer.fit(model)

    result = trainer.test(model)

`

My question is: Does the model use the parameters at the last epoch when testing, or the best parameters loaded?

[akihironitta]

Does the model use the parameters at the last epoch when testing, or the best parameters loaded?

Hi @Struggle-Forever, yes, the model is in the last state, but not in the best state. I hope the following example in the documentation will explain it enough:

# run full training
trainer.fit(model)

# (1) load the best checkpoint automatically (lightning tracks this for you)
trainer.test(ckpt_path="best")

# (2) test using a specific checkpoint
trainer.test(ckpt_path="/path/to/my_checkpoint.ckpt")

# (3) test with an explicit model (will use this model and not load a checkpoint)
trainer.test(model)

https://pytorch-lightning.readthedocs.io/en/1.6.4/common/evaluation.html#test-after-fit

[Struggle-Forever]

thank you very much!

[Struggle-Forever]

thank you very much!