Integrate latest AnimateDiff-Lightning model

[hensuninnovation]

Hello!
Thank you very much for open source these wonderful sign language processing toolkit and models! Truly remarkable! I come across this new AnimateDiff model from ByteDance, which looks fast and promising. I am wondering how hard to integrate into this project.

https://huggingface.co/ByteDance/AnimateDiff-Lightning

https://arxiv.org/pdf/2403.12706.pdf

[AmitMY]

It should be as simple as changing the adapter ID, scheduler config, and maybe adjusting some of the diffusion parameters.

If you have access to a GPU, you could play with it immediately here:
https://github.com/sign-language-processing/pose-to-video/blob/main/pose_to_video/processors/animatediff/__init__.py

I don't expect it to take too much work. Feel free to contribute a solution! :)