Logger isn't finalized upon test/predict completion

[mooey5775]

🐛 Bug

Some loggers (like TensorBoardLogger) require flushing/finalize to be called for logged metrics to be properly written to disk. However, finalize is not currently called after test/predict runs.

This manifests in metrics failing to be logged if .test or .predict is run on Trainer with a TensorBoardLogger.

I believe this could be fixed with an extra .finalize call (like what is already in https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/loops/fit_loop.py#L265-L266) in EvaluationLoop and PredictionLoop. I'm not entirely sure if adding this would cause any issues though, so I'm a bit hesitant to submit a PR myself.

To Reproduce

https://colab.research.google.com/drive/14j7qCoy33y4giqjA8xNfNMCK81F_OvFH?usp=sharing

Expected behavior

Metrics should be logged, regardless of whether .fit, .test, or .predict is called.

Environment

• CUDA:
  • GPU:
  • available: False
  • version: 10.2
• Packages:
  • numpy: 1.19.5
  • pyTorch_debug: False
  • pyTorch_version: 1.9.0+cu102
  • pytorch-lightning: 1.4.0dev
  • tqdm: 4.41.1
• System:
  • OS: Linux
  • architecture:x
    • 64bit
  • processor: x86_64
  • python: 3.7.10
  • version: # 1 SMP Sat Jun 5 09:50:34 PDT 2021

[mooey5775]

@ananthsub

[ananthsub]

@awaelchli this is what #8217 was attempting to solve