Does this also work on non-video VAE?

[markg85]

Hi,

Just out of curiosity as the repo and paper talk only about video purposes.
Does this also work on non-video VAE like Qwen-Image, Flux, etc...

[ZouYa99]

Hi,
I think that the two points mentioned in the article should also be applicable to 2D VAEs:
(1) Applying depthwise separable convolution to reduce parameter redundancy in the VAE decoder.
(2) Utilizing a decoder-only design along with feature alignment distillation to improve training efficiency.
However, further experiments are needed to validate this. If you have any experimental progress to share, I would be very interested to discuss it.

[markg85]

Ha 🙂 I'm afraid i lack the knowledge and hardware to do any experimentation (beyond just plugging it in as VAE and see what happens). But i'd very much encourage you to give it a try!

[ZouYa99]

I don't have plans in this regard recently, sorry for that. :( Maybe you could check out the DC-AE model, which has also released a turbo version (https://huggingface.co/mit-han-lab/dc-ae-lite-f32c32-sana-1.1-diffusers). It also adopts a fixed encoder and a lightweight decoder to accelerate inference (huggingface/diffusers#12169).