Contributing Intel Arc support back

[jbaboval]

I started hacking on a branch to try things out on my Arc A770. It got a bit more involved than I planned, and it's working pretty well. It needs a rebase and a bit of a cleanup, which I'm willing to do, but probably won't bother unless there's interest.

Is this something you'd like contributed back? And if so we should have a chat about how you like things structured before I start tidying my mess up. :)

https://github.com/jbaboval/stable-diffusion-webui/tree/oneapi

[ClashSAN]

is training possible here or with diffusers library? How is your experience with this?

[jbaboval]

I should know better. Nothing works until you test it.

There doesn't appear to be a drop-in intel equivalent to the cuda GradScaler. So there will need to be some work.

[ClashSAN]

What about the inference modes?
I was just curious if you could run dreambooth/regular finetuning with the 16gb. I think I saw some AMD user (16gb) do this with just cross attention optimization (you could check the diffusers issue page)

To get auto's opinion you'd likely have to submit some intial PR draft. none of us except for you can test, so it'll be up to you whether or not you could help maintain this aspect of the repo. having a standalone is important too. (that's my idea anyway)

[jbaboval]

I've been taking a bit of a crash course in this stuff over the last week. I started this mostly because I wanted to play with my new GPU and didn't have any exposure to ML.

I think that there is enough memory to use FP32 and not have to deal with underflow and gradient scaling. I don't think it would be too hard to make the change. However.

I'm not comfortable enough with the outcomes to be sure of what success looks like. At my day job I can hash some stuff, and if it matches I didn't ruin things. Here, success is being in the statistical ballpark. Short of it not blowing up, I don't know enough yet to tell if I broke anything. And there aren't really any tests around this stuff to use as a crutch, and even if there were I don't have an nvidia system to run it on to test the existing use cases.

[jbaboval]

As for the inference modes, they appear to work. At least most of the time. Sometimes you run with the same seeds and get NaN for everything. And sometimes you run with the same seeds and your system reboots. AFAICT that's just immaturity in the intel drivers, and not because I've broken something in the webui codebase. It's really quite preferable to having no support for the Intel GPU at all.

[jbaboval]

I got another hard drive and installed Linux to see if the linux drivers were more stable than the Windows/WSL2 driver.

.....they aren't. They're broken differently.

For now I'm just going to keep a rolling re-base going. I might open a PR to isolate some of the device/accelerator support in a way that makes supporting new GPUs out-of-tree easier. Need to find a way to test CUDA first though.