ComfyUI crashes due to invalid or unsupported data type Float8_e4m3fn.

[Benix-laj]

So I've just installed ComfyUI and ComfyUI manager, and downloaded resources for the Flux fp8 model. When I try to queue a prompt the miniconda terminal crashes due to invalid or unsupported data type Float8_e4m3fn so the process stops, and I get no image. I've got an AMD card (6800 XT) . I don't really understand any of the things I'm doing, I'm just trying to get the AI model running :/. It gave the same error without the manager too, so the installation of the manager shouldn't be a problem.

I installed ComfyUI based on this tutorial (miniconda, git, directml):
https://www.youtube.com/watch?v=8rB7RqKvU5U

Here is the Flux workflow I use:

Here's the list of links of the resources I downloaded for flux:
VAE: https://huggingface.co/black-forest-labs/FLUX.1-schnell/blob/main/ae.safetensors
Text encoder: https://huggingface.co/comfyanonymous/flux_text_encoders/blob/main/t5xxl_fp8_e4m3fn.safetensors
Clip: https://huggingface.co/comfyanonymous/flux_text_encoders/blob/main/clip_l.safetensors
Flux: https://huggingface.co/Comfy-Org/flux1-schnell/blob/main/flux1-schnell-fp8.safetensors

C:\Users\nyirc>call conda activate comfyui
[START] Security scan
[DONE] Security scan

ComfyUI-Manager: installing dependencies done.

** ComfyUI startup time: 2024-10-23 14:22:15.895454
** Platform: Windows
** Python version: 3.10.12 | packaged by Anaconda, Inc. | (main, Jul 5 2023, 19:01:18) [MSC v.1916 64 bit (AMD64)]
** Python executable: D:\Programs\Miniconda\envs\comfyui\python.exe
** ComfyUI Path: C:\Users\nyirc\ComfyUI
** Log path: C:\Users\nyirc\ComfyUI\comfyui.log

Prestartup times for custom nodes:
0.8 seconds: C:\Users\nyirc\ComfyUI\custom_nodes\ComfyUI-Manager

Using directml with device:
Total VRAM 1024 MB, total RAM 32544 MB
pytorch version: 2.4.1+cpu
Set vram state to: NORMAL_VRAM
Device: privateuseone
Using sub quadratic optimization for cross attention, if you have memory or speed issues try using: --use-split-cross-attention
[Prompt Server] web root: C:\Users\nyirc\ComfyUI\web
D:\Programs\Miniconda\envs\comfyui\lib\site-packages\kornia\feature\lightglue.py:44: FutureWarning: torch.cuda.amp.custom_fwd(args...) is deprecated. Please use torch.amp.custom_fwd(args..., device_type='cuda') instead.
@torch.cuda.amp.custom_fwd(cast_inputs=torch.float32)

Loading: ComfyUI-Manager (V2.51.8)

ComfyUI Revision: 2781 [915fdb5] | Released on '2024-10-22'

Import times for custom nodes:
0.0 seconds: C:\Users\nyirc\ComfyUI\custom_nodes\websocket_image_save.py
0.2 seconds: C:\Users\nyirc\ComfyUI\custom_nodes\ComfyUI-Manager

Starting server

To see the GUI go to: http://127.0.0.0:0000
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json
FETCH DATA from: C:\Users\nyirc\ComfyUI\custom_nodes\ComfyUI-Manager\extension-node-map.json [DONE]
got prompt
Using split attention in VAE
Using split attention in VAE
model weight dtype torch.float8_e4m3fn, manual cast: torch.float32
model_type FLOW
Requested to load FluxClipModel_
Loading 1 new model
loaded completely 0.0 9319.23095703125 True
clip missing: ['text_projection.weight']
Requested to load Flux
Loading 1 new model
[F1023 14:27:43.000000000 dml_util.cc:118] Invalid or unsupported data type Float8_e4m3fn.

[a-mecham]

Was there ever a fix for this.
I am having the same trouble using a RX 7600XT, using ComfyUI in Miniconda environment with Python 3.12.7. and dirtectml.

My Error when using Flux1 Dev is [F1202 12:21:25.000000000 dml_util.cc:118] Invalid or unsupported data type Float8_e4m3fn.

[iamone0201]

I am facing the same issue, and found the root cause of it in a YouTube comment:

The torch-directml package supports up to PyTorch 2.3.1, and unfortunately, PyTorch 2.3.1 is outdated and does not support Float8_e4m3fn.

If you install latest torch, it is not supported by AMD GPUs and comfyUI won't start.
So we will have to wait till torch-directml package gets updated to support new PyTorch version.

[John10v10]

AI developers be like what tf is an AMD!?
Me: I don't know... maybe the type of GPU in every modern computer these days?

[Leadros]

I can run most flux models in Ubuntu with Python 3.10. There are issues with 3.12 and even 3.11, but 3.10 works best. In Windows, I get the same error. I recently upgraded Hipsdk to 6.2 and have more issues than ever. I was able to run flux fine before this. I have a 8700g CPU and 7800xt

[Leadros]

I am going to downgrade to 5.7 again and see if that works. Being limited to a Linux os sucks. I will let you know if it works

[Leadros]

I tried to downgrade and no es bueno. I will use flux in Linux and not wsl. In Windows, I will use fooocus since it works fine and doesn't crash. Hopefully, AMD will get off their ass and get some support other than a version of an outdated pytorch. Still slow as dirt but is this more a comfyui issue or AMD?

[cuongptnk]

I have same issue with RX 6800 XT

[Slickytail]

It's because you're loading flux in float8. Load a bf16/fp16 flux checkpoint instead.

[Leadros]

I am using flux checkpoints from fp4-fp16. It works very slowly, and after 3-4 gens you get the error so dunno. I have not messed with it in a few weeks

[rjceledon]

It's because you're loading flux in float8. Load a bf16/fp16 flux checkpoint instead.

Can you elaborate a bit more? I'm a total newbie and don't get where I should be changing this, I want to give it a try

[dijidl]

+1 same issue RX6800M

[Leadros]

Has anyone found a work around? Hopefully they found a workaround or something for the RDNA 4 cards. I tried hip SDK 6.1.2 and no change. I have not tried the newest version, or tried in weeks, supposedly 6+ drivers have tensorflow support. I am getting a 5090

[BinaryCortex]

Unfortunately, the only workaround I have found is to run in CPU only mode. Not ideal, but it works.

[Garruk929]

RX 7900 XTX OC, same problem here

[stan4cb]

amd side stuff progresses so slow, I couldn't belive these problems still presist

[Leadros]

They are amazing for gaming and good for video gaming. On a side note, they at least said they will be pushing rocm for windows finally!

[allanmaze]

I have the same here "[dml_util.cc:118] Invalid or unsupported data type Float8_e4m3fn" trying to work with Flux.
AMD Radeon RX 6800 XT, windows, using Anaconda's environment with python 3.10.12.
I tried different versions of HIP SDK and Cudas, different torch versions, fork ComfyUI-Zluda, but the only version that even partially works is Anaconda one. AMD in AI stuff it's feels like nothing but pain now.

Guys, write if you find a solution to this problem (don’t suggest getting rid of AMD and buying Nvidia, everything has its time).

[Leadros]

AMD is not there yet. The support is growing, and it's good for LLMs and SD, and not much else. I bought a 5090 and no regrets. AMD announced that it will be pushing the SDK for Windows. It was too much of a headache for now. 5000 series has its own issues, but I can do anything in comfyui

[PhantomSeventyEight]

Hi Hier same rx9070xt when some fix is please write here that we all can fix it . thx a lot

[rjceledon]

Hello, same issue here with AMD Ryzen 7 5700G with Vega 8 graphics

[patientx]

try fp8_e5m2 versions of the models or try zluda or therock (pytorch rocm for selective gpu's atm for windows)

[JimmyAlan]

I am having the same basic message too:

FETCH ComfyRegistry Data: 5/95
FETCH ComfyRegistry Data: 10/95
FETCH ComfyRegistry Data: 15/95
got prompt
Using split attention in VAE
Using split attention in VAE
VAE load device: privateuseone:0, offload device: cpu, dtype: torch.float16
Using scaled fp8: fp8 matrix mult: False, scale input: False
got prompt
CLIP/text encoder model load device: privateuseone:0, offload device: cpu, current: cpu, dtype: torch.float16
FETCH ComfyRegistry Data: 20/95
got prompt
got prompt
Requested to load WanTEModel
0 models unloaded.
FETCH ComfyRegistry Data: 25/95
[F827 18:00:59.000000000 dml_util.cc:118] Invalid or unsupported data type Float8_e4m3fn.

(env) (base) C:\pinokio\api\comfy.git\app>