CUDA out of memory (trying to use 36Gb on Tesla T4 + GCP)

[far3x]

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 36.00 GiB. GPU 0 has a total capacity of 14.56 GiB of which 10.22 GiB is free. Including non-PyTorch memory, this process has 4.11 GiB memory in use. Process 17435 has 240.00 MiB memory in use. Of the allocated memory 3.72 GiB is allocated by PyTorch, and 264.21 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
2025-11-25T10:59:14+0000 [INFO] [:1] _ (scheme=http,method=POST,path=/generate,type=application/vnd.bentoml+pickle,length=334179) (status=500,type=application/json,length=74) 6320.340ms (trace=228f2d74cf88e1b1fd44a0361bde54f7,span=3c0274921753d612,sampled=0,service.name=CatVTONService)
Traceback (most recent call last):

any fix for this issue? thanks in advance

im running this on GCP, Tesla T4, and it's been few days since I can't fix this

[rahul0304-tech]

You're trying 36GB of memory while T4 only offers about 15GB and only about 10-12 that you can utilize so you need to figure out how much you can run on Tesla T4 or switch to premium cloud GPU that offers 36GB of memort