ZImage output is just noise after hack to make zimage work in fp16

[mamei16]

Custom Node Testing

• I have tried disabling custom nodes and the issue persists (see how to disable custom nodes if you need help)

Expected Behavior

The output should somewhat resemble what it was before when I ran the model and "manual cast: FP32" was printed in the logs.

Actual Behavior

The output is just noise.

Steps to Reproduce

I can reproduce it by simply running the following workflow:

Debug Logs

Checkpoint files will always be loaded safely.
Total VRAM 12272 MB, total RAM 31494 MB
pytorch version: 2.4.1+rocm6.1
AMD arch: gfx1030
ROCm version: (6, 1)
Set vram state to: NORMAL_VRAM
Device: cuda:0 AMD Radeon RX 6800M : native
Enabled pinned memory 29919.0
Using sub quadratic optimization for attention, if you have memory or speed issues try using: --use-split-cross-attention
Python version: 3.11.13 (main, Jun  5 2025, 13:12:00) [GCC 11.2.0]
ComfyUI version: 0.3.76
ComfyUI frontend version: 1.33.10
[Prompt Server] web root: /home/user/envs/stable_diffusion/lib/python3.11/site-packages/comfyui_frontend_package/static
Total VRAM 12272 MB, total RAM 31494 MB
pytorch version: 2.4.1+rocm6.1
AMD arch: gfx1030
ROCm version: (6, 1)
Set vram state to: NORMAL_VRAM
Device: cuda:0 AMD Radeon RX 6800M : native
Enabled pinned memory 29919.0
Skipping loading of custom nodes
Context impl SQLiteImpl.
Will assume non-transactional DDL.
No target revision found.
Starting server

To see the GUI go to: http://127.0.0.1:8188
got prompt
Using split attention in VAE
Using split attention in VAE
VAE load device: cuda:0, offload device: cpu, dtype: torch.float32
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
Requested to load ZImageTEModel_
loaded completely; 11028.80 MB usable, 7672.25 MB loaded, full load: True
model weight dtype torch.float16, manual cast: None
model_type FLOW
unet missing: ['norm_final.weight']
Requested to load Lumina2
loaded partially; 4795.20 MB usable, 4720.20 MB loaded, 7019.35 MB offloaded, 75.00 MB buffer reserved, lowvram patches: 0
100%|██████████████████████████████████████████████████████████| 8/8 [01:21<00:00, 10.14s/it]
Requested to load AutoencodingEngine
0 models unloaded.
loaded completely; 4773.80 MB usable, 319.75 MB loaded, full load: True
Prompt executed in 114.16 seconds

Other

I am fairly certain that the issue stems from commit #11057.

[mamei16]

Maybe related issue: #11078

[Ph0rk0z]

That's a very old pytorch. I have the same card as #11078 and it works for me.

[mamei16]

That's a very old pytorch. I have the same card as #11078 and it works for me.

I'll check tomorrow if the outdated Pytorch version is to blame.

[dermesut]

i get the same output in comdyui (with z-image).

this is the setup:
Python version: 3.11.8 (tags/v3.11.8:db85d51, Feb 6 2024, 22:03:32) [MSC v.1937 64 bit (AMD64)]
ComfyUI version: 0.3.76
ComfyUI frontend version: 1.33.10

Total VRAM 11264 MB, total RAM 130956 MB
pytorch version: 2.4.0+cu121
xformers version: 0.0.27.post2

Device: cuda:0 NVIDIA GeForce RTX 2080 Ti : native

in forge neo, z-image does work, using the same "nodes" (model, vae, text encoder) as in comfyui. the setup in neo is:
pytorch version: 2.9.1+cu128

so, i'm guessing the (old) pytorch is a problem, at least for me.

[patientx]

use pytorch 2.7.x for amd , I have rx 6800 and can confirm the fp16 hack works and gives 2x speed up

[mamei16]

Using torch 2.9.1 does fix the noise issue!

One drawback is that I can now no longer run VAE decode for Wan 2.1, Z-Image or Flux.1 without running out of memory and falling back to tiled decoding. Also, I now have to pass --fp32-vae for Z-Image, because VAE decoding otherwise takes a prohibitively long time.
Edit: I was able to fix the VAE decode issue by setting the environment variable MIOPEN_FIND_MODE=2 (see #8289 (comment))

Still, this issue is fixed after updating torch, so thank you @Ph0rk0z for pointing it out.

[Ph0rk0z]

I use tae-f1 for z-image. Used both fp32 and --fp16-vae with not much difference but I'm still on torch 2.8. Waiting till like 2.10 to upgrade.

[dermesut]

i get the same output in comdyui (with z-image).

this is the setup: Python version: 3.11.8 (tags/v3.11.8:db85d51, Feb 6 2024, 22:03:32) [MSC v.1937 64 bit (AMD64)] ComfyUI version: 0.3.76 ComfyUI frontend version: 1.33.10

Total VRAM 11264 MB, total RAM 130956 MB pytorch version: 2.4.0+cu121 xformers version: 0.0.27.post2

Device: cuda:0 NVIDIA GeForce RTX 2080 Ti : native

in forge neo, z-image does work, using the same "nodes" (model, vae, text encoder) as in comfyui. the setup in neo is: pytorch version: 2.9.1+cu128

so, i'm guessing the (old) pytorch is a problem, at least for me.

i upgraded pytorch to pytorch version: 2.9.1+cu128 and xformers to xformers version: 0.0.33.post1.
the problem is gone now, for me.