Is multi-gpu supported?

[maxenko]

Has anyone got Kohya to train on multiple GPUs (on same system)?
Please share your config/setup.
Thank you.

[kalkal11]

Yes, you just have to select the relevant options when setting up accelerate config

[7sanal7san]

can you please share the config details or config file, i have been trying to get it to work for a month now, i tried almost everything, if i select multi gpu in accelerate config it gives an error

[kalkal11]

First up when you set up accelerate - instead of typing all for which GPU's to use, try typing the numbers.

for example on a 8x system, you would type:
0,1,2,3,4,5,6,7

(I had issues and this actually fixed this propblem)

Also depending on other dependencies, I have had issues with various versions relating to all GPU's functioning at the same time, for example at one point I could only use 0-6 without crashing.

As SD-Scripts does not handle multi-gpu support directly, accelerate handles this, it's all a compatibility thing between the dependencies.

Assuming you still can't get training working, please post your error and config you're running with.