Error trying to train a model in fast-dreambooth on paperspace :((

[choriflai]

Hi, I've paid the service in paperspace due to recent Google Colab restrictions. I loved this notebook since colab but now that I tried to run it on paperspace it throws me an error of "CUDA out of memory" trying to train the text encoder (When the error shows up it skips the text encoder step and start with Unet, so the results are none like the instance images".
This notebook on gradient its working for everyone else?

Have a great week!. (sorry if there's some errors with my english, it's not my language :c )

[TheLastBen]

at what resolution are you training ?

[choriflai]

You mean without uploading instance images?
Just dependencies, model and session, right?

[choriflai]

Well, I'm getting the same. I don't know if I'm doing something wrong. You have it running in paperspace lately?
It throws the error and keeps with the next step that's training the Unet. I can keep it running until it finishes and saves a ckpt file, but the results are not the instance image 'cause it has no text encoder I assume :c

[choriflai]

[TheLastBen]

you can't use dreambooth with an 8GB GPU

[choriflai]

Oh, I didn't notice that I picked a smaller one now, mb
The notebook it's working for you?
To see if I'm doing it wrong or its just an error in the notebook
Works for me in colab, but I'm having this trouble in paperspace
Thanks for responding btw I'm a newbie in all this

[TheLastBen]

if you're encountering errors, copy the log here

[choriflai]

I ran the latest notebook on /PPS and on a more powerful gpu (a4000), and seems to be working. I think that the first problem was that I was running an outdated notebook, and then the gpu with low memory incident
Now on the A4000 it's not showing any errors so that was it. Thanks for the help, just mb!

[cutecoelho]

Why you don't use the Google Colab to train models? Still working on the free plan. Only A1111 isn't working.

[TheLastBen]

it's inconvenient if you can't test the trained model with the webui

[choriflai]

Yeah, I can't test the results anymore, and because of Colab problem running automatic1111 I paid paperspace anyways, so in theory I have access to much powerful gpus (Training of 3000 steps on colab was like 2:30hrs on a T4, and now if I can catch an rtx A4000 it's like 15-20min)

[cutecoelho]

Ohh, get it. I never used paperspace, don't know how it works. Thought would be a easier way to upload the model and test it