MNIST demo is not utilizing the GPU

[delip]

I am running the demo code on a host with V100 (as seen via. nvidia-smi), but the GPU utilization is zero. I confirmed this by instrumenting line 99 onwards to see if the model was loaded on the GPU and if a CUDA device was available.

                if (batch_idx == 0) or ((batch_idx + 1) % hparams.log_interval == 0):
                    print(model.device)
                    print(torch.cuda.is_available())
                    print(
                        "Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}".format(
                            epoch,
                            batch_idx * len(data),
                            len(train_loader.dataset),
                            100.0 * batch_idx / len(train_loader),
                            loss.item(),
                        )

I notice that the GPU was visible to the process and yet the model was running on the CPU. I also tried to "force" it to use all available GPUs, by modifying the final line to the following:

    Lite(accelerator="gpu", devices=-1).run(hparams)

And still, the model is loaded and optimized on the CPU. I have two questions: 1) Why is the model not running on the GPU here?, and 2) How do I force the Lite Trainer to use the GPU?

Other information:

Python implementation: CPython
Python version       : 3.10.4
IPython version      : 8.4.0

Compiler    : GCC 9.4.0
OS          : Linux
Release     : 5.4.0-1086-azure
Machine     : x86_64
Processor   : x86_64
CPU cores   : 6
Architecture: 64bit

watermark        : 2.3.1
torch            : 1.12.1
torchvision      : 0.13.1
argparse         : 1.1
pytorch_lightning: 1.7.1

nvidia-smi:

$ nvidia-smi 
Thu Aug 18 19:43:44 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.65.01    Driver Version: 515.65.01    CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla V100-PCIE...  On   | 00000001:00:00.0 Off |                    0 |
| N/A   28C    P0    26W / 250W |      4MiB / 16384MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

[akihironitta]

@delip I've just run the example in my environment but don't see any issue with it.

env

* CUDA:
	- GPU:
		- NVIDIA GeForce RTX 3090
		- NVIDIA GeForce RTX 3090
		- NVIDIA GeForce RTX 3090
		- NVIDIA GeForce RTX 3090
		- NVIDIA GeForce RTX 3090
		- NVIDIA GeForce RTX 3090
		- NVIDIA GeForce RTX 3090
		- NVIDIA GeForce RTX 3090
	- available:         True
	- version:           11.6
* Lightning:
	- pytorch-lightning: 1.7.2
	- torch:             1.12.1+cu116
	- torchaudio:        0.12.1+cu116
	- torchmetrics:      0.9.3
	- torchvision:       0.13.1+cu116
* Packages:
	- absl-py:           1.2.0
	- aiohttp:           3.8.1
	- aiosignal:         1.2.0
	- async-timeout:     4.0.2
	- attrs:             22.1.0
	- cachetools:        5.2.0
	- certifi:           2022.6.15
	- charset-normalizer: 2.1.0
	- frozenlist:        1.3.1
	- fsspec:            2022.7.1
	- google-auth:       2.10.0
	- google-auth-oauthlib: 0.4.6
	- grpcio:            1.47.0
	- idna:              3.3
	- markdown:          3.4.1
	- markupsafe:        2.1.1
	- multidict:         6.0.2
	- numpy:             1.23.2
	- oauthlib:          3.2.0
	- packaging:         21.3
	- pillow:            9.2.0
	- pip:               22.1.2
	- protobuf:          3.19.4
	- pyasn1:            0.4.8
	- pyasn1-modules:    0.2.8
	- pydeprecate:       0.3.2
	- pyparsing:         3.0.9
	- pytorch-lightning: 1.7.2
	- pyyaml:            6.0
	- requests:          2.28.1
	- requests-oauthlib: 1.3.1
	- rsa:               4.9
	- setuptools:        61.2.0
	- six:               1.16.0
	- tensorboard:       2.10.0
	- tensorboard-data-server: 0.6.1
	- tensorboard-plugin-wit: 1.8.1
	- torch:             1.12.1+cu116
	- torchaudio:        0.12.1+cu116
	- torchmetrics:      0.9.3
	- torchvision:       0.13.1+cu116
	- tqdm:              4.64.0
	- typing-extensions: 4.3.0
	- urllib3:           1.26.11
	- werkzeug:          2.2.2
	- wheel:             0.37.1
	- yarl:              1.8.1
* System:
	- OS:                Linux
	- architecture:
		- 64bit
		- ELF
	- processor:         x86_64
	- python:            3.10.4
	- version:           #59-Ubuntu SMP Wed Oct 21 09:38:44 UTC 2020

so, I guess there may be an issue with your environment setup. Would you mind running the following script and share the output here? Creating a fresh environment and starting over may simply resolve your issue.

wget https://raw.githubusercontent.com/Lightning-AI/lightning/master/requirements/collect_env_details.py
python collect_env_details.py