How can I set up model parallelism during inference?

[waitfor-night]

Feature request / 功能建议

怎么设置可以在推理的时候模型并行呢？
How can I set up model parallelism during inference?

Motivation / 动机

80G，视频总帧数320
args:
mode: inference
i2v_encode_video: True
latent_channels: 16
load: SCAIL-Preview/model
batch_size: 1
input_type: txt
input_file: input.txt
sampling_image_size: [512, 896]
vae_compress_size: [4, 8, 8] # t h w
bf16: True
sampling_fps: 32
force_inference: True
sequence_parallel_size: 1
image2video: True
use_i2v_clip: True
use_pose: True
representation: 'smpl_downsample'
save_tag: official
output_dir: SCAIL/examples
爆显存了。

Your contribution / 您的贡献

。

[teal024]

应该是总帧数太大了；Preview版本不像Wan Animate那样支持基于上一段的推理，在目前的仓库里仅推荐使用81帧。正式版会分段推理，把每一段帧数控制在合理范围内，可以减少显存占用。
目前仓库仅支持使用mpi和序列并行来进行多gpu推理，我们刚才在config里更新了如何使用序列并行进行推理的注释。需要把sequence_parallel_size: 1改为您的gpu数量(如2, 4)。

It should be that the total number of frames is too large; the Preview version does not support inference based on the previous segment like Wan Animate, and it is recommended to use only 81 frames in the current repository. The official version will perform segment-wise inference, controlling the number of frames in each segment within a reasonable range, which can reduce memory usage.
Currently, the repository only supports using mpi and sequence parallelism for multi-GPU inference. We just updated the comments in the config on how to use sequence parallelism for inference. You need to change sequence_parallel_size: 1 to the number of your GPUs(eg.2, 4).