🧑‍🍳 New Cookbook Recipe: Add a recipe showcasing the usage of Pruna with Diffusers

[ParagEkbote]

pruna is a library which contains a comprehensive suite of compression algorithms including caching, quantization, pruning, distillation and compilation techniques to make your models more efficient to use. I would be interested in contributing a recipe using pruna and the diffusers library.

Based on a discussion with @davidberenstein1957, the recipe would feature a diffusers model quantized using the smash config, then the smashed model could be used for synthetic dataset creation, or we could evaluate the model against its unquantized version as well. Could you please let me know your thoughts and suggestions if the scope of the recipe is sufficient or any changes to make it more applied as well.

cc: @stevhliu

[stevhliu]

I think a recipe for synthetic dataset creation would be nice. We already have some Diffusers docs on Pruna here, so I would try to avoid overlapping with it too much

[davidberenstein1957]

Awesome, @ParagEkbote. As mentioned, I think we could show a distillation / synthetic data pipeline where we use a VLM to evaluate prompts-image pairs and re-use rewritten prompts for re-generations over several iterations. I think we could use a subset of https://huggingface.co/datasets/data-is-better-together/open-image-preferences-v1 to do that.

I would be happy to guide you or draft a first version myself.

As mentioned, something like FLUX-kontext would be super interesting for this too as it allow for more fine-grained correction and alteration but is likely too heavy and costly for a simple PoC.

[davidberenstein1957]

@sayakpaul, have you ever seen anything like using VLMs-as-a-judge to evaluate generated images? I found some research papers from the folks of Prometheus hinting towards it, but I haven't seen them being used in practice yet.

[sayakpaul]

Yes very much: https://gist.github.com/sayakpaul/3ddc7a56854c36791143bd73705fe760