Pytorch2 works fine on AMD, at least on a RX 6800

[cspace-echo]

Hey,

I saw a list of upcoming changes to this webUI ans one of them caught my eye:

"switch to torch 2.0.0 (except for AMD GPUs)"

What would be the reason for this?

Because I have Pytorch 2.0.0+rocm5.4.2 working fine on my RX 6800. Only trouble I had getting it working was that the initial install script replaced that version of pytorch with the nvidia/cuda-specific version in the venv. After uninstalling and installing the rocm version in it's place inside the venv it works at arounf 7 I/s on 512v512, only needing the --no-half-vae option.

If it's for broader compatibility then sure, but all AMDs only working on a specific version combination of rocm and pytorch is old news.

[Soulreaver90]

Hey,

I saw a list of upcoming changes to this webUI ans one of them caught my eye:

"switch to torch 2.0.0 (except for AMD GPUs)"

What would be the reason for this?

Because I have Pytorch 2.0.0+rocm5.4.2 working fine on my RX 6800. Only trouble I had getting it working was that the initial install script replaced that version of pytorch with the nvidia/cuda-specific version in the venv. After uninstalling and installing the rocm version in it's place inside the venv it works at arounf 7 I/s on 512v512, only needing the --no-half-vae option.

If it's for broader compatibility then sure, but all AMDs only working on a specific version combination of rocm and pytorch is old news.

Funny enough, that is exactly the same thing I had to go through. Installed, removed and reinstalled rocm torch and set —no-half-vae. I’ve been running 2.0 for weeks with no issues, I’ve run a ton of models and vaes with no issues. So far the people who’ve reported issues seem to have 5000 series cards. I have a 6700 xt for reference.

[willymanman]

Hey,

I saw a list of upcoming changes to this webUI ans one of them caught my eye:

"switch to torch 2.0.0 (except for AMD GPUs)"

What would be the reason for this?

Because I have Pytorch 2.0.0+rocm5.4.2 working fine on my RX 6800. Only trouble I had getting it working was that the initial install script replaced that version of pytorch with the nvidia/cuda-specific version in the venv. After uninstalling and installing the rocm version in it's place inside the venv it works at arounf 7 I/s on 512v512, only needing the --no-half-vae option.

If it's for broader compatibility then sure, but all AMDs only working on a specific version combination of rocm and pytorch is old news.

Do you use xformers with your pytorch 2.0 install? I've been trying to work on having them both work and I'm at a loss. Also how exactly did you get around the install? Every time I've tried it has refused to work, even if I do a clean install with pytorch 2.0 from the beginning.

[cspace-echo]

Haven't touched xformers as I'm not doing anything the requires them.

If you mean how I got the UI working then:

1. go to the UI's folder and activate the venv with source venv/bin/activate
2. list your pytorch versions there with pip list | grep 'torch'
3. If torch torchvision torchaudio don't end with the rocm version (in my case rocm5.4.2) then uninstall them with pip uninstall torch==<WrongVersionsHere>
4. Install them back by running the command from https://pytorch.org/get-started/locally/ that you get with Stable(2.0)+Linux+pip+python+ROCm selected
5. Then just leave the venv with deactivate and run webui.sh --no-half-vae.

That works-on-my-machine™, anecdotally another linux user I know got that working by simply deleting the install script generated venv and recreating it. And if this wasn't new information to you then at least I wrote it out for the next poor soul googling this and only finding information made a dozen versions ago or that it'll never work on AMD.

[tdm1000]

I got it sort of working on my 6700 xt, webui starts just fine and will generate but my issue is that I only get a black square in the generation. I've already tried --precision full and --no-half arguments together and individually. Any ideas?

[Soulreaver90]

You do NOT need those arguments at all, not with a 6700 XT. All you need is --no-half-vae. Did you verify the torch you have installed? Make sure its 2.0+rocm5.4.2. Also if you launch launch.py manually, run export HSA_OVERRIDE_GFX_VERSION=10.3.0 prior. Or you can add it as an argument in webui-user.sh or whatever it was called. The HSA override is important or else the rocm won't work correctly.

[tdm1000]

You do NOT need those arguments at all, not with a 6700 XT. All you need is --no-half-vae. Did you verify the torch you have installed? Make sure its 2.0+rocm5.4.2. Also if you launch launch.py manually, run export HSA_OVERRIDE_GFX_VERSION=10.3.0 prior. Or you can add it as an argument in webui-user.sh or whatever it was called. The HSA override is important or else the rocm won't work correctly.

Tried it out with --no-half-vae, I kept getting tensors produced with all NANs so I added --disable-nan-check and now it just generates an image of noise. I took a screenshot of my installed torch versions. I wasn't sure if torchvision also needed to be 2.0+rocm5.4.2 but its the version I got with the command from the pytorch site. EDIT: forgot to mention that I did export HSA_OVERRIDE_GFX_VERSION=10.3.0 before launching.

[Chilluminati91]

RX 6650XT needs export HSA_OVERRIDE_GFX_VERSION=10.3.0 but not --no-half-vae. Wonder why this is different for all those cards. There is no performance increase in comparison though.

[Soulreaver90]

Do you use vae with your models? If not, then you don’t need --no-half-vae, otherwise you’ll get OOM issues.

[waelisa]

GPU: AMD ATI Radeon RX 5600 OEM/5600 XT / 5700/5700 XT
didn't work with me, i try
pip install torch==2.0.1+rocm5.4.2 torchvision==0.15.2+rocm5.4.2 torchaudio==2.0.2+rocm5.4.2 --index-url https://download.pytorch.org/whl/rocm5.4.2
and run with --no-half-vae
GPU load high but not progress

[set-soft]

Same here for Navi 14 RX 5500 XT.
I tried 2.0.0, 2.0.1 and 2.1.0 nightly. All of them die with a memory access error.

[waelisa]

You can use stable-diffusion-webui on docker but i don't want, i wait for stable-diffusion-webui work on AMD or i cant buy new NVIDIA card.

[set-soft]

I use a docker image, it doesn't have any relation with the fact that PyTorch doesn't work on Navi14 and similar boards.

[Soulreaver90]

I use a docker image, it doesn't have any relation with the fact that PyTorch doesn't work on Navi14 and similar boards.

Exactly. I’ve seen countless threads about SD not working for 5k cards, all stemming from the fact that rocm doesn’t natively support that model type. As far as I can tell, ROCM does not support RDNA1/Navi1.x cards. Funny enough, it does support older polaris/GCN5 cards such as the 500 series, although requires extra work.

[set-soft]

The problem is with PyTorch 2.x, I tried 1.10, 1.12 and 1.13, all of them work.
And yes, RDNA1 is not officially supported, you must fake the GPU version and also force F32 (F16 not implemented)