Flux.1 int4 in openvino pipeline using GPU but blur image, but CPU Work fine

[ak-sangal]

I am using Genai Openvino pipeline using Flux1 int4 by getting inspiration from notebooks/flux.1-image-generation
but i got burry image while using gpu but when i use CPU it works well but takes lot of time
intel i71355U
iGPU irisx with 16 GB ram
steps 256*256
steps 4
device GPU
i attached script and image plzz fix it

flux_gpu.py

[Wovchena]

CPU can't infer f16 so it switches to f32. Do you get blurry image if you switch INFERENCE_PRECISION_HINT from f16 to f32 for iGPU?

[ak-sangal]

But it takes lots of time more than 1 mint some time 3 mint

[ak-sangal]

is there any way to reduce image rendering time it takes more than 5 mint when using F32

[newbhumika]

t looks like this may be a precision + quantization interaction issue rather than a pure performance issue.

Since CPU falls back to f32 and produces correct output, but GPU with f16 produces blur, it may indicate that the int4 weights + f16 accumulation are introducing significant error.

A couple of questions:

Does the blur persist if the model is run in int8 instead of int4 on GPU?

Does forcing f32 on GPU remove the blur entirely?

Is the iGPU path using native fp16 or emulated?

If needed, I can try to reproduce and isolate whether this is specific to Flux.1 int4 or a more general precision issue on Iris Xe.

[ak-sangal]

Does the blur persist if the model is run in int8 instead of int4 on GPU?
i have not tried yet int8.

Does forcing f32 on GPU remove the blur entirely?
NO ,vary slow takes 8 mints

Is the iGPU path using native fp16 or emulated?
native f16

i am using other stable diffusion models such as sdxl turbo, dreamshaper 8 in openvino format wors perfectly

[Wovchena]

But it takes lots of time more than 1 mint some time 3 mint

So, do you get blurry image if you switch INFERENCE_PRECISION_HINT from f16 to f32 for iGPU?

[ak-sangal]

When I use f32 for igpu blurry image doesn't happen but it takes more than 5 mins
Now I am trying to use int8 flux 1.model
I will share the results later after using flux 1 int8 model

[Wovchena]

You can try #3349 to save compute time while inferring f32. You can also try exporting f16 model and inferring f16.

[ak-sangal]

i used flux.1 int 8 model with image size 512*512 with 2 steps using gpu i got below result but sometimes laptop freeze for a second

[newbhumika]

Looking at the behavior so far, this appears to be a precision–quantization interaction rather than a general pipeline issue.

We observe:

INT4 + FP16 (GPU) → blurred output

INT4 + FP32 (GPU) → correct output (but slow)

INT8 (GPU) → improved quality

Other SD models → working correctly on same hardware

Since FP32 restores correctness even with INT4, this strongly suggests the degradation is happening during FP16 accumulation rather than in the quantized weights themselves.

Flux.1 may be more numerically sensitive to reduced accumulation precision compared to SDXL-based models, especially if per-channel scales or dynamic ranges are large.

A few points that may help narrow this down:

Does the INT4 GPU kernel accumulate in FP16 or internally promote to FP32?

Is accumulation precision strictly controlled by INFERENCE_PRECISION_HINT, or partially overridden in kernel implementation?

Are INT4 dequantization scales computed in FP16 on GPU?

This looks like a kernel-level precision sensitivity on Iris Xe rather than a model export problem.

If useful, I can try to reproduce and compare INT4 FP16 vs FP32 outputs quantitatively to confirm accumulation precision as the root cause.

[ak-sangal]

Thanks for the detailed breakdown. I am running this on an Intel Core i7-1355U (13th Gen) with Iris Xe Graphics (16GB RAM) on Windows 11 Pro.
I suspect you are right about the kernel-level sensitivity. When running Flux.1 in INT4, the output is significantly degraded compared to SDXL-based models on this same hardware.
It would be extremely helpful if you could compare the INT4 FP16 vs. FP32 outputs quantitatively. If the Iris Xe is indeed overriding the INFERENCE_PRECISION_HINT or accumulating strictly in FP16, it would explain why Flux.1 (which has a much larger dynamic range) is breaking while smaller models stay stable.

[newbhumika]

To properly isolate the blur source, I think we should control for both quantization level and accumulation precision under identical conditions (same seed, resolution, and steps).

A useful comparison would be:

INT4 + FP16 accumulation

INT4 + FP32 accumulation

INT8 + FP16

If blur appears only in INT4 + FP16 but not in INT4 + FP32, that would strongly suggest accumulation precision is amplifying quantization noise.

If both INT4 variants show similar degradation, then the quantization itself is likely the dominant factor.

Since diffusion models are iterative, even small rounding errors can propagate across steps, so accumulation precision could be playing a significant role here.