Trying to get comfyUI to run on a AMD Radeon AI Pro R9700 with 32 gb vram

[ReinerBforartists]

Hey friends,
I have migrated from Nvidia to an AMD card. An AMD Radeon AI Pro R9700 with 32 gb vram. I am on Windows 11.
I have then downloaded the newest portable AMD version of ComfyUI, 0.8.1. But none of my workflows works anymore. Comfy simply quits. The content is never generated. And i don't get any error message. Just a Pess any Key to continue in the console.

I was able to reduce the problem to Python and Torch: Example error message:

0x00007FFA0F1AB520, G:\comfyamd\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\lib\torch_cpu.dll(0x00007FFA0F1A0000) + 0xB520 byte(s), ?count_nonzero@Tensor@at@@qeba?AV12@V?$optional@_J@std@@@z() + 0x30 byte(s)
0x00007FFA4832BA14, G:\comfyamd\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\lib\torch_python.dll(0x00007FFA48000000) + 0x32BA14 byte(s), ?release@?$THPPointer@U_frame@@@@QEAAPEAU_frame@@xz() + 0x1C2BA4 byte(s)
0x00007FFAA41879E4, G:\comfyamd\ComfyUI_windows_portable\python_embeded\python312.dll(0x00007FFAA4150000) + 0x379E4 byte(s), PyThread_acquire_lock_timed() + 0x5F0 byte(s)
0x00007FFAA41D2018, G:\comfyamd\ComfyUI_windows_portable\python_embeded\python312.dll(0x00007FFAA4150000) + 0x82018 byte(s), PyObject_Vectorcall() + 0x688 byte(s)

The bat that i have used to end at least in an error message was made by ChatGPT:

@echo off
REM -----------------------------------------------
REM 1. Umgebungsvariablen für AMD / DirectML stabil setzen
set TORCH_DML_DISABLE_GRAPH_FUSION=1
set TORCH_DML_ENABLE_DYNAMIC_TENSORS=0

REM -----------------------------------------------
REM 2. Python-Pfad auf Embedded Python
set PYTHON_EXE=.\python_embeded\python.exe

REM 3. ComfyUI Pfad
set COMFY_MAIN=ComfyUI\main.py

REM 4. ComfyUI starten mit sicheren Flags
%PYTHON_EXE% %COMFY_MAIN% --windows-standalone-build --force-fp32 --disable-smart-memory --dont-upcast-attention

pause

So here my question, how can i get ComfyUI to work with my AMD card? What is your advice?

Kind regards
Reiner

[LukeG89]

ComfyUI crashes during model loading? In this case it is generally a problem with pagefile size. Try to increase it or set it to "managed by the system"

[ReinerBforartists]

Thanks for the idea, but it crashes at starting to do the job. At the clip text encode node. Or at the prompt schedule node. The models loads fine.

[ReinerBforartists]

Hm, i think my problem is much more fundamental. Since i haven't installed Rocm yet. I thought everything needed comes with Comfy Portable already.

I can though just find advice to install Rocm for a venv and with cloning the repo. Which will not work with the portable. And this is even Linux. I am at Windows.

https://rocm.docs.amd.com/en/7.10.0-preview/rocm-for-ai/pytorch-comfyui.html

[madias123]

If you can wait, wait for at least ROCm 7.2. In 7.1.1. there are really nasty memory bugs and other stupid things. Even 6xx was more stable. I don't know why everyone is pushing ROCm support (under windows!) as "offical" - it's not even pre alpha.

[ReinerBforartists]

Ah this sounds not good :D

Waiting, well i have some images to generate. Would it make sense to install a dual boot environment and use comfy under ubuntu?

And what is the official steps to get comfy working together with my card under Windows? I would even go with an old Rocm version if this would make Comfy at least somehow functional again.

[ReinerBforartists]

Okay, the desktop version shows the same behaviour than the portable. It pauses and stops without any error message.

[ReinerBforartists]

Found windows 7.1.1 hip here: https://www.amd.com/en/resources/support-articles/release-notes/RN-AMDGPU-WINDOWS-PYTORCH-7-1-1.html?utm_source=chatgpt.com

Trying.

[ReinerBforartists]

... and you need to install / compile pytorch wheels. Thats me finally out. I don't want to ruin my system. Geez AMD. Didn't realize that they are technically this far behind.

Okay, now let's have a look if Linux is a solution.

[Exobit42]

What driver are you using?
I'm on Windows11 + AMD as well (9070 XT and AI Max+ 395) - both run fine with Comfy's default BAT when using the driver from this site:
https://www.amd.com/en/resources/support-articles/release-notes/RN-AMDGPU-WINDOWS-PYTORCH-7-1-1.html

Be sure to disable auto updating because once Adrenalin installs the standard driver Comfy won't work anymore.

Good luck!

[ReinerBforartists]

Oh, this driver works for you? I have still the default driver applied. i didn't want to toy around with it. But now I will give it a try tomorrow. Many thanks :)

[ReinerBforartists]

The special driver was indeed the solution. I can now generate images with Z Image, and i can generate videos with Wan 2.2. More is not needed, for now.

I uninstalled everything with Display Driver Uninstaller in safe mode. And have set up the new driver then.

The speed increase of the AMD card is a bit disappointing compared to the 4060 Ti. A Wan 2.2 workflow that took just over 10 minutes with the old card now takes 8.5 minutes. I had expected more. But at least I now have enough RAM to avoid constantly running into an out-of-memory (OOM) situation. And Z Image runs four times faster. Here i have indeed a big speed boost.

And so far it runs stable enough. So i think i will skip my plans with the dual boot system and trying Ubuntu. Maybe at a later point.

Many thanks for the help :)

Be sure to disable auto updating because once Adrenalin installs the standard driver Comfy won't work anymore.

I cannot find out how. Adrenaline does not have a checkbox from what i can see. How do i?

[Exobit42]

The driver installer asks about that setting on the last screen. Afterwards you can change it via:
Open AMD Adrenalin -> Cogwheel / settings -> System -> Manage Updates (? Updates verwalten) -> scroll down and uncheck Keep AMD Software up to date (? AMD Software automatisch auf dem neuesten Stand halten).

[ReinerBforartists]

Thank you. Now i found it. But Adrenalin doesn't let me untick exactly this point. Crazy. But no worries, i need to test out Linux anyways. So i don't mind if Adrenalin is installing the up to date driver anymore.

[ReinerBforartists]

Okay, there are some roadblocks in the way. Some custom nodes requires special treatment now. Like the
QwenVL node. Let's have a look if i can compile it myself.

[ReinerBforartists]

Nope :(

Windows is not currently supported.

[Exobit42]

I use LM Studio for prompt generation (and other AI tasks). Sailing the AMD-boat I try to keep my workflows as basic as possible.
Maybe give it a shot?

[ReinerBforartists]

That was in fact my substitute then :)

[ReinerBforartists]

And the next pitfall. Other sofware does not like to play well with the torch version of the driver.

Okay, this means i have to go Linux with ComfyUI. Was a nice dream with Windows and AMD.

[Exobit42]

Now that got me curious - what workflow are you using? I would live to give it a shot with my 9070 XT which should be the same as your R9700 expect for RAM (add random amount of jealousy here - I decided to go for AI Max for my homeserver instead).

[ReinerBforartists]

This is not a workflow, this is Blender not starting anymore with the Torch version of the AMD driver :)

[ReinerBforartists]

And here comes the uncommon workaround: let Adrenalin install the latest official driver. Then install the Pytorch driver again, and untick Keep Adrenalin Software Up to Date. Now i have both working, ComfyUI, and Blender.

[ReinerBforartists]

... and now my game has quit working. Time for DDU and starting over.

It remains sad. Comfy UI + Windows + Rocm = no go.

[jamilnielsen]

have you considered Docker? its great for reducing variables.
docker+docker compose+portainer+mostly copy/paste text and it might work?
oh and rocm, on host system of course.
i dont have that specific graphic card, i am hoping to buy it in a few months though, before i went shopping for amd cards i tried my AMD Radeon RX 6700 XT 12gb, also had trouble actually running workflows, until i switched to docker, much easier to control dependencies and specific versions without tainting the host system or fearing for auto update.
yanwk/comfyui-boot.

let me know if it works for ye.

[jamilnielsen]

two steps forward, two steps back.
uh, try :rocm instead of :rocm7? "The difference between rocm and rocm7 images is that rocm7 is built on rocm/pytorch image provided by AMD, which will update earlier than rocm when new ROCm version is released." newer is not always better.
also i cant see your "- HSA_OVERRIDE_GFX_VERSION=12.0.0" but i figure you removed the # and set it to correct version, uh...try a few?
otherwise not sure what could be wrong currently.
side note: since you are defining "/home/x/storage:/root" then you probably dont need to define the rest of those volumes as they're sub folders of root.

PS: a quick look around reveals that apparently there was a problem with ROCm 7 YanWenKun/ComfyUI-Docker#157
might not be related but there's some reading material if you are interested.
and here's a issue tracker more directly for AMD.
ROCm/ROCm#5245

[ReinerBforartists]

Yeah, that's linux. Always a challenge to get even the simplest things to work. That's the fun part :)

Many thanks for the ideas. Reading.

GFX version is set in the yml.

environment:
  - CLI_ARGS=
  # - CLI_ARGS="--disable-smart-memory --use-pytorch-cross-attention"
  - HSA_OVERRIDE_GFX_VERSION=12.0.0

What did not work was to uncomment the CLI_ARGS. It threw me an error then.

How do i start the docker with disable smart memory?

I currently start the docker with sudo docker start comfyui-rocm7. runing it with docker run with all these commands did not work. But the yml way works good it seems. Minus that it gets stuck in the loop then at the ksampler with interruptions.

Trying the normal rocm at the moment, downloading ...

[jamilnielsen]

no clue about disable smart memory, ones i got things to work in test i stopped there.

[ReinerBforartists]

I wish i would reach that point :D

But i found it, It can also be done in the yml file. Uncomment the line # - CLI_ARGS="--disable-smart-memory --use-pytorch-cross-attention", and comment the line with the empty CLI arguments. I think at the first try i forgot to do that, and had two versions of the CLI_ARGS active.

[ReinerBforartists]

I finally got it to work. My mistake was the wrong GFX version. I had chosen 12.0.0, but need 12.0.1

• HSA_OVERRIDE_GFX_VERSION=12.0.1

The docker now works. And it was well worth the hassle. My Wan Video test workflow finished in 300 seconds. At windows with the rocm driver i ended in 570 seconds. That's nearly the double speed.

Now let's have a look for tomorrow and the official Adrenalin rocm driver. I expect the docker at Linux still to be faster. But then i can at least doodle around with images without to reboot.

[ReinerBforartists]

Honestly this turns into a nightmare. I have now installed rocm and comfy on ubuntu. I don't see any errors. rocminfo shows everything correct. The device is there. Console looks okay. Comfy starts. But when it starts to calculate then Python crashes. Tested with z image and a wan image workflow.

The console then shows a memory access error. What wonders me here is, i have rocm installed, i use the rocm comfyui version, but it still talks about Cuda?

CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cuda:0, dtype: torch.float16
Speicherzugriffsfehler (Speicherabzug geschrieben)

In case somebody has a clue here, i am open to any suggestion.

[madias123]

...you really should wait a few days, believe me I´ve spend hours for nothing:
https://www.reddit.com/r/ROCm/comments/1qd3aly/amd_to_launch_adrenalin_edition_2611_drivers_with/

[ReinerBforartists]

New driver is out. It works. But i regularly run into a rocm crash. Not funny when you do lots of videos or images.

:0:C:\develop\TheRock\rocm-systems\projects\clr\rocclr\device\device.cpp:379 : 8936464523 us: Memobj map does not have ptr: 0x80c00000

Currently experimenting with these run_amd_gpu.bat settings

REM stability under Windows + AMD ROCm by better garbage collection
set PYTORCH_HIP_ALLOC_CONF=garbage_collection_threshold:0.8,max_split_size_mb:256

REM disable-pinned-memory and reserve-vram 0.05 reduces pointer problems with rocm. 0.05 vram reserved means 5%
.\python_embeded\python.exe -s ComfyUI\main.py --windows-standalone-build --disable-pinned-memory --reserve-vram 0.05
pause

It seems to be a bit more stable with that. Future will tell.

[jamilnielsen]

is the new driver faster? any exciting performance boost?

[ReinerBforartists]

Mh with the old pytorch driver my test video took ten minutes, now it is down to between 7 minutes and still 10 minutes, in case it even finishes. As told, i run into the memjob map does not have ptr error.

The performance boost happens at Linux here. 5 minutes :)

[jamilnielsen]

reminds me how loud is it? the graphic card, i have heard mixed things.

[ReinerBforartists]

Let me put it this way: It is very vocal ! :D

The noise level is comparable to a nvidia 3060 XT. It is disturbing when you play games and reach the value where it starts to make full noise. At approximately 80 percent capacity it starts to take off. For making videos with comfyui it is even a good thing. You hear clearly when it arrives at the ksamplers :)

And while at speed, currently Z Image needs between 4 and 10 seconds in the first run, and sometimes 30 up to 240 seconds in the second run and everthing afterwards. Very nasty. Curious enough, i did not have this problem with the old pytorch only driver from before two weeks.

As it is now, ComfyUI is simply useless at Windows with the current driver. Let's see if they can fix this problem with a future version. But ChatGPT did not make me high hopes here. It's a design problem. And i cannot go back to the older driver since then my other software quits working.

Short, straight answer:

On AMD GPUs under Windows, the first ComfyUI run is fast because VRAM is clean.
After that, VRAM gets fragmented or partially spilled to system RAM due to AMD drivers + PyTorch’s allocator.
The second run then crawls from seconds to minutes.

Fixes that actually help:

Use a stricter allocator:
PYTORCH_HIP_ALLOC_CONF=garbage_collection_threshold:0.6,max_split_size_mb:128

Disable Sage Attention (--disable-sage-attention)

Don’t reuse latents / Z-images

Unload & reload the model between runs

Avoid the newest Adrenalin AI Bundle; PRO or older drivers are more stable

There is no real VRAM defragmentation on Windows + AMD. Reboot works because it fully clears VRAM.