Should we use torch.cuda.memory instead of nvidia-smi for the GPU stats monitor?

[ananthsub]

This pytorch module offers a number of stats directly: https://pytorch.org/docs/master/cuda.html#memory-management
list_gpu_processes will report the total memory used, like nvidia-smi. This seems more convenient and flexible than requiring that nvidia-smi is installed and querying these stats via a separate process in this callback: https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/callbacks/gpu_stats_monitor.py

[awaelchli]

This looks great. Need to be aware that some of these ops are not available in torch < 1.8, at least I couldn't find list_gpu_processes in the docs.