tutorials/video/wan/wan2-2-s2v

[giscus[bot]]

tutorials/video/wan/wan2-2-s2v

This is a native workflow example for Wan2.2-S2V audio-driven video generation in ComfyUI.

https://docs.comfy.org/tutorials/video/wan/wan2-2-s2v

[dodoclan]

Works FINE, thanks, for amazing work!!!

[mojokb]

Good Work! Thanks aLOT!

[Lolik97]

Does not work on the stable version, you need to roll back the old one.

[comfyui-wiki]

Which version are you using? The latest stable one should be 0.3.55.

[photomanz]

Very nice job. Works great! Just a comment that the Clip Text Encode negative prompt is in Chinese. ;)

[jerryjin2018]

Actually, I have leveraged the latest version of ComfyUI to generate video according to this example within this blog, but I found that generated video with a little bit issue, such as the accuracy of hands.

[jelehmann]

ComfyUI 0.3.56, all custom nodes updated (nightly), models, vae and encoders at the right locations and visible to the ui. mp3 and image loaded. But:

got prompt
Failed to validate prompt for output 113:
* WanSoundImageToVideoExtend 79:76:
  - Return type mismatch between linked nodes: length, received_type(FLOAT) mismatch input_type(INT)
* KSampler 79:77:
  - Return type mismatch between linked nodes: cfg, received_type(INT) mismatch input_type(FLOAT)
* WanSoundImageToVideoExtend 85:76:
  - Return type mismatch between linked nodes: length, received_type(FLOAT) mismatch input_type(INT)
* KSampler 85:77:
  - Return type mismatch between linked nodes: cfg, received_type(INT) mismatch input_type(FLOAT)
Output will be ignored
invalid prompt: {'type': 'prompt_outputs_failed_validation', 'message': 'Prompt outputs failed validation', 'details': '', 'extra_info': {}}

...any idea what might be the problem?

[NiklasH208]

Got the same problem. I checked the datatype compatability between the nodes but everything looks right to me. (default "Audio-Driven Video Gen" template workflow)

[dimmm7]

Strange, fom me it works...but does not really work: mouth is hardly moving background stays static... ok i was using a hard image (with fp8 model) with two people in the foreground and a few others in the background.. i did change also the prompt after a few tries trying to enhance expressions and mouth movement. But no luck.
The hand on the image test was not in sync and during the first two seconds the mouth was not moving at all. Thanks for the effort any way. maybe i should try a single person on a limbo background, and after all this model is still experimental. The regular Wan 2.2 i2v is still fantastic, and super fast. Thank you for hard work. I 'll definitely experiment more S2V.