Cupy out of memory

[diegorv1994]

When I train with a GPU in prodigy this error appears: cupy.cuda.memory.OutOfMemoryError: Out of memory allocating 101,347,328 bytes (allocated so far: 1,218,353,664 bytes). How can I solve this problem?

[ljvmiranda921]

Hi @diegorv1994 , what kind of model are you training? Might need more information to your problem (e.g., base model, prodigy recipe, etc.) 🤔 As an immediate measure, perhaps try lowering the batch size so that not all documents are loaded at the same time?