GPU Peak Memory #30
Description
Are you sure the maximum GPU peak memory is really 10GB? I tried it in a Colab notebook with a 15GB T4 GPU and got an error.
!git clone -q https://github.com/NUS-HPC-AI-Lab/Enhance-A-Video.git
!pip install -q condacolab
import condacolab
condacolab.install()
%cd Enhance-A-Video
!conda create -n enhanceAvideo python=3.10
!conda activate enhanceAvideo
!pip install -r requirements.txt
%cd Enhance-A-Video
!python cogvideox.py
Loading pipeline components...: 100% 5/5 [00:16<00:00, 3.23s/it]
Traceback (most recent call last):
File "/content/Enhance-A-Video/cogvideox.py", line 9, in <module>
pipe.to("cuda")
File "/usr/local/lib/python3.11/site-packages/diffusers/pipelines/pipeline_utils.py", line 460, in to
module.to(device, dtype)
File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3110, in to
return super().to(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1343, in to
return self._apply(convert)
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 903, in _apply
module._apply(fn)
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 903, in _apply
module._apply(fn)
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 903, in _apply
module._apply(fn)
[Previous line repeated 4 more times]
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 930, in _apply
param_applied = fn(param)
^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1329, in convert
return t.to(
^^^^^
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 80.00 MiB. GPU 0 has a total capacity of 14.74 GiB of which 68.12 MiB is free. Process 25784 has 14.67 GiB memory in use. Of the allocated memory 14.11 GiB is allocated by PyTorch, and 474.11 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)