add support for Qwen Image Pruning

[wbruna]

For #851 . Allow the model loading logic to tolerate missing layers, which is enough to run the 12B Pruning variant:

https://huggingface.co/OPPOer/Qwen-Image-Pruning

Tested with the Q4_K_M quant from https://huggingface.co/wsbagnsv1/Qwen-Image-Pruning-GGUF :

[wbruna]

Quality seems a little worse than the Lightning model, with ~30% less peak VRAM usage, and similar speed gains.

[wbruna]

@leejet , looks like a123e25 from the qwen_image_edit branch is enough to support the '13b' pruned model. Thanks!

The '12b' variant still doesn't work, though, maybe because it has non-contiguous layers. I guess they're keeping the non-pruned layers with the same number as they have on the original model.

[LostRuins]

@wbruna I think it might be a problem with the GGUF quant, not the model. Look at the GGUF https://huggingface.co/wsbagnsv1/Qwen-Image-Pruning-GGUF/tree/main?show_file_info=Qwen-Image-Pruning-12b-Q4_0.gguf Transformer block 18,

versus the original https://huggingface.co/OPPOer/Qwen-Image-Pruning/tree/main/Qwen-Image-12B/transformer?show_file_info=Qwen-Image-12B%2Ftransformer%2Fdiffusion_pytorch_model-00002-of-00003.safetensors

I've reported it to the GGUF quant guy but he doesn't seem to get what i mean https://huggingface.co/wsbagnsv1/Qwen-Image-Pruning-GGUF/discussions/1

[wbruna]

@leejet , looks like a123e25 from the qwen_image_edit branch is enough to support the '13b' pruned model. Thanks!

Not working anymore after d21d1aa (didn't check which revision). I'll retest and update this PR.

[LostRuins]

What do you mean "Not working anymore", is it still generating an image? It seems to work for me

[wbruna]

What do you mean "Not working anymore", is it still generating an image? It seems to work for me

This PR still works; I was referring to my previous comment, which mentioned that a123e25 (from the qwen_image branch itself) made this PR unnecessary.

[leejet]

The support for dynamic number of Qwen image transformer blocks is available in the qwen_image_edit branch.

[wbruna]

The support for dynamic number of Qwen image transformer blocks is available in the qwen_image_edit branch.

I tested it with the same model I use to test this PR (the 13b, they renamed it afterwards):

[DEBUG] model.cpp:2088 - loading tensors from /opt/sdif/models/SD/Qwen-Image-Pruning-Q4_K_M.gguf
  |===================================>              | 1293/1825 - 45.33it/s
[DEBUG] model.cpp:2088 - loading tensors from /opt/llm/Qwen2.5-VL-7B-Instruct-IQ4_XS.gguf
  |============================================>     | 1631/1825 - 43.55it/s
[DEBUG] model.cpp:2088 - loading tensors from /opt/sdif/models/VAE/Qwen_Image-VAE.safetensors
  |============================================>     | 1634/1825 - 43.63it/s[INFO ] model.cpp:2358 - unknown tensor 'first_stage_model.conv1.bias | bf16 | 1 [32, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:2358 - unknown tensor 'first_stage_model.conv1.weight | bf16 | 4 [1, 1, 1, 1024, 1]' in model file
  |==================================================| 1825/1825 - 48.21it/s
[INFO ] model.cpp:2326 - loading tensors completed, taking 37.90s (process: 0.05s, read: 24.94s, memcpy: 0.00s, convert: 0.47s, copy_to_backend: 12.08s)
[ERROR] model.cpp:2399 - tensor 'model.diffusion_model.transformer_blocks.40.attn.add_k_proj.bias' not in model file
[ERROR] model.cpp:2399 - tensor 'model.diffusion_model.transformer_blocks.40.attn.add_k_proj.weight' not in model file
[ERROR] model.cpp:2399 - tensor 'model.diffusion_model.transformer_blocks.40.attn.add_q_proj.bias' not in model file
[ERROR] model.cpp:2399 - tensor 'model.diffusion_model.transformer_blocks.40.attn.add_q_proj.weight' not in model file
[ERROR] model.cpp:2399 - tensor 'model.diffusion_model.transformer_blocks.40.attn.add_v_proj.bias' not in model file
(...)

[leejet]

Please use the latest code from this PR #877.

[wbruna]

Oh, I see: I forgot that rev was on the qwen_image_edit branch 🤦

@LostRuins , btw: if you're already syncing with that one, feel free to drop my changes, they shouldn't be necessary.

[leejet]

I noticed that this PR was automatically closed after the qwen_image branch was deleted.
If the related changes are still relevant, please feel free to reopen it for further review.

[wbruna]

I noticed that this PR was automatically closed after the qwen_image branch was deleted. If the related changes are still relevant, please feel free to reopen it for further review.

Thanks. I was keeping this open just to further investigate about the 12b variant, but I could just open another one instead.