trainer.test() does not return anything

[thistlillo]

My model test methods are as follows:

def test_step(self, batch, batch_idx):
        print('test_step')
        X, y = batch
        y_est = self.forward(X)
        loss = self.criterion(y_est, y)
        acc = self.compute_acc(y_est, y)
        return {'test_batch_loss': loss, 'test_batch_acc': acc}

def test_epoch_end(self, outputs):
        avg_loss = torch.stack([x['test_batch_loss'] for x in outputs]).mean()
        avg_acc = torch.stack([x['test_batch_acc'] for x in outputs]).mean()
        print('test epoch ends', avg_loss)
        return {'loss': avg_loss, 'acc': avg_acc}

The model is trained then tested with:

    trainer.fit(model, datamodule=dm)
    print("testing")
    results = trainer.test(verbose=True)
    print(results)

The data module returns three dataloaders (different): training, validation and test.

But results is empty, when it should not. According to the documentation here, trainer.test():

Returns
List of dictionaries with metrics logged during the test phase, e.g., in model- or callback hooks like test_step(), test_epoch_end(), etc. The length of the list corresponds to the number of test dataloaders used.

Actual output from my code is:

testing                                                                                                                                         
/opt/conda/lib/python3.8/site-packages/pytorch_lightning/core/datamodule.py:423: LightningDeprecationWarning: DataModule.prepare_data has already been called, so it will not be called again. In v1.6 this behavior will change to always call DataModule.prepare_data.
  rank_zero_deprecation(
Testing: 0it [00:00, ?it/s]test_step
test_step
test_step
test_step
test_step
test_step
test_step
test_step
Testing: 100%|####################################################################################################| 8/8 [00:00<00:00, 29.85it/s]
test epoch ends tensor(0.6921)
--------------------------------------------------------------------------------
DATALOADER:0 TEST RESULTS
{}
--------------------------------------------------------------------------------
Testing: 100%|####################################################################################################| 8/8 [00:00<00:00, 23.87it/s]
results [{}]

As you can see, results from trainer.test() is empty.

What did I fail to understand? For sure, I do not understand the initial LightningDeprecationWarning.

[rohitgr7]

only logged metrics are returned
and test_epoch_end doesn't consider any returned outputs

def test_epoch_end(self, outputs):
    avg_loss = torch.stack([x['test_batch_loss'] for x in outputs]).mean()
    avg_acc = torch.stack([x['test_batch_acc'] for x in outputs]).mean()
    self.log_dict({'test_loss': avg_loss, 'test_acc': avg_acc})

if you don't want to log it inside the logger, then use self.log_dict(..., logger=False), this way it will return what you need and won't log it if you don't want it to be logged.