"get sd version from file failed:" To work with img2img and gaianet/FLUX.1-Fill-dev-GGUF

[ServeurpersoCom]

I love stable-diffusion.cpp; I try a flux.1-fill model https://huggingface.co/gaianet/FLUX.1-Fill-dev-GGUF (Quantized with stable-diffusion.cpp master-c3eeb669.)

#105
#511
"Adds support for sd1.x inpaint, sd2.x inpaint, sdxl inpaint, and Flux Fill models, as well as "inpainting" (masked img2img) with normal diffusion models."

const char* model_version_to_str[] = {
    "SD 1.x",
    "SD 1.x Inpaint",
    "SD 2.x",
    "SD 2.x Inpaint",
    "SDXL",
    "SDXL Inpaint",
    "SVD",
    "SD3.x",
    "Flux",
    "Flux Fill"};
...

->
        version = model_loader.get_sd_version();
        if (version == VERSION_COUNT) {
            LOG_ERROR("get sd version from file failed: '%s'", model_path.c_str());
            return false;
        }

If I override version with VERSION_FLUX_FILL (I tried with and without --vae from flux) I get same :

ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 4060 Ti, compute capability 8.9, VMM: yes
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.conv_in.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.conv_in.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.conv_out.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.conv_out.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.attn_1.k.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.attn_1.k.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.attn_1.norm.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.attn_1.norm.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.attn_1.proj_out.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.attn_1.proj_out.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.attn_1.q.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.attn_1.q.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.attn_1.v.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.attn_1.v.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.block_1.conv1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.block_1.conv1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.block_1.conv2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.block_1.conv2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.block_1.norm1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.block_1.norm1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.block_1.norm2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.block_1.norm2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.block_2.conv1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.block_2.conv1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.block_2.conv2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.block_2.conv2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.block_2.norm1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.block_2.norm1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.block_2.norm2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.mid.block_2.norm2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.norm_out.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.norm_out.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.0.conv1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.0.conv1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.0.conv2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.0.conv2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.0.nin_shortcut.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.0.nin_shortcut.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.0.norm1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.0.norm1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.0.norm2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.0.norm2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.1.conv1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.1.conv1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.1.conv2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.1.conv2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.1.norm1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.1.norm1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.1.norm2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.1.norm2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.2.conv1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.2.conv1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.2.conv2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.2.conv2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.2.norm1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.2.norm1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.2.norm2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.0.block.2.norm2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.0.conv1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.0.conv1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.0.conv2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.0.conv2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.0.nin_shortcut.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.0.nin_shortcut.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.0.norm1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.0.norm1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.0.norm2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.0.norm2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.1.conv1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.1.conv1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.1.conv2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.1.conv2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.1.norm1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.1.norm1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.1.norm2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.1.norm2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.2.conv1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.2.conv1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.2.conv2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.2.conv2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.2.norm1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.2.norm1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.2.norm2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.block.2.norm2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.upsample.conv.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.1.upsample.conv.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.2.block.0.conv1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.2.block.0.conv1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.2.block.0.conv2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.2.block.0.conv2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.2.block.0.norm1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.2.block.0.norm1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.2.block.0.norm2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.2.block.0.norm2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.2.block.1.conv1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.2.block.1.conv1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.2.block.1.conv2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.2.block.1.conv2.weight' not in model file

So the real issue seems to be that even with --vae, the external ae.safetensors (provided by the original Flux model authors) is either:

• Not being loaded properly
• Or not merged/used during the model initialization phase (even though all first_stage_model.decoder.* tensors are present in the VAE file)

Maybe the VAE external loading logic is disabled or bypassed for non-standard model versions like VERSION_FLUX_FILL? Or perhaps the loading logic only works for certain architectures like SDXL?

Would love a pointer to where in model.cpp / model_loader.cpp this logic is handled, happy to test/debug/provide trace if needed!

Happy to test any WIP/fix branch. My server is running 24/7 and already compiles custom builds.

gguf-dump.txt

[stduhpf]

If I override version with VERSION_FLUX_FILL

What do you mean override? You shouldn't have to override anything. If you got the "get sd version from file failed:" error, then it would probably because you tried to load the diffusion model with -mor --model instead of --diffusion-model. Most Flux models (including the one you linked to) need to be loaded with --diffusion-model.

Flux fill is working on my end, even with the quants and vae from gaianet/FLUX.1-Fill-dev-GGUF.

Here's an example command that worked for me:
.\sd.exe -M img2img --diffusion-model .\models\Flux\dev\flux1-fill-dev-Q3_K.gguf --t5xxl .\models\clip\t5\t5xxl_q4_k.gguf --vae .\models\vae\ae.safetensors -p "holding a painting of a dog" --color --strength 1 -i "cat.png" --mask "cat-mask.png" --steps 24 --cfg-scale 1 --guidance 30 --clip_l .models\clip\clip_l\clip_l.q8_0.gguf --sampling-method euler

Images

cat.png	cat-mask.png	output.png

[ServeurpersoCom]

Oh!!!!! the habit of llama.cpp and its -m
The fact that I managed to create lots of images and outpainting with:

"-m stable-diffusion.cpp/models/sd3.5_large_turbo-q8_0.gguf"
and
"-m stable-diffusion.cpp/models/stable-diffusion-inpainting/512-inpainting-ema.safetensors"

Misled me! I try --diffusion-model right now thanks a lot !

https://github.com/user-attachments/assets/326c17aa-fe49-48af-a8d4-4b1c578c4c20
Lot of fun with infinite-zoom script (only stable-diffusion.cpp and imagemagik convert !!!!)

And I discovered I make the same mistake with --cfg-scale and --guidance 😂

[ServeurpersoCom]

First test: outpaint the entire area around the image and paste the original image at the correct scale to show the difference:

Now i need some tuning and outpaint only 4 bands It works. I can close it :)

[Justin-12138]

@stduhpf @ServeurpersoCom hey,I got the same issue using the command as follow:

sd --diffusion-model  /mnt/cache/flux/flux1-schnell-Q8_0/flux1-schnell-q8_0.gguf \
--vae /mnt/cache/flux/flux1-schnell-Q8_0ls/ae.sft \
--clip_l /mnt/cache/flux/flux1-schnell-Q8_0/models/clip_l.safetensors \
--t5xxl /mnt/cache/flux/flux1-schnell-Q8_0/models/t5xxl_fp16.safetensors \
-p "a lovely cat holding a sign says 'flux.cpp'" \
--cfg-scale 1.0 --sampling-method euler -v --steps 1

[DEBUG] model.cpp:1727 - loading tensors from /mnt/cache/flux/flux1-schnell-Q8_0/flux_text_encoders/clip_l.safetensors
  |========>                                         | 196/1192 - 0.00it/s[DEBUG] model.cpp:1727 - loading tensors from /mnt/cache/flux/flux1-schnell-Q8_0/flux_text_encoders/t5xxl_fp16.safetensors
  |=================>                                | 413/1192 - 0.00it/s[INFO ] model.cpp:1897 - unknown tensor 'text_encoders.t5xxl.transformer.encoder.embed_tokens.weight | f16 | 2 [4096, 32128, 1, 1, 1]' in model file
  |=================>                                | 416/1192 - 6.25it/s[DEBUG] model.cpp:1727 - loading tensors from /mnt/cache/flux/flux1-schnell-Q8_0/flux1-schnell-Q8_0.gguf
  |==================================================| 1192/1192 - 200.00it/s

[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.3.block.1.norm2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.3.block.2.conv1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.3.block.2.conv1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.3.block.2.conv2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.3.block.2.conv2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.3.block.2.norm1.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.3.block.2.norm1.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.3.block.2.norm2.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.3.block.2.norm2.weight' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.3.upsample.conv.bias' not in model file
[ERROR] model.cpp:1938 - tensor 'first_stage_model.decoder.up.3.upsample.conv.weight' not in model file
[ERROR] stable-diffusion.cpp:441 - load tensors from model loader failed
new_sd_ctx_t failed
is that caused by "413/1192 - 0.00it/s[INFO ] model.cpp:1897 - unknown tensor" ?

[Justin-12138]

my falut,it works well with the fowllowing command

sd --diffusion-model  /mnt/cache/flux/flux1-schnell-Q8_0/flux1-schnell-q8_0.gguf \
--vae /mnt/cache/flux/flux1-schnell-Q8_0/models/ae.safetensors \
--clip_l /mnt/cache/flux/flux1-schnell-Q8_0/models/clip_l.safetensors \
--t5xxl /mnt/cache/flux/flux1-schnell-Q8_0/models/t5xxl_fp16.safetensors \
-p "a lovely cat holding a sign says 'flux.cpp'" \
--cfg-scale 1.0 --sampling-method euler -v --steps 1