variables

[pooyafayyazs]

I had a couple of questions regarding the following variables and their usage in the code:

• i2v_condition
• mask_lat_size
• latent_condition

Could you please clarify what each of these represents and how they differ?

In particular, I noticed this line:

latent_i2v_condition = torch.cat([mask_lat_size, latent_condition], dim=1)

And later:

latent_model_input = torch.cat([latents, i2v_condition], dim=1).to(transformer_dtype)

Could you also explain the role of latent_model_input in this context and how it relates to the other tensors?

Thanks again!

[ewrfcas]

Good question! This part is mainly copied from RealisDance-DiT (https://github.com/damo-cv/RealisDance/blob/main/src/pipelines/rd_dit_pipeline.py).
In my understanding, i2v_condition is useless in RealisDance-DiT. Because RealisDance-DiT adopts image conditions from ref_condition, while i2v_condition can be regarded as a blank input (all zero inputs before the VAE), which is retained to fit the original input formulation of Wan2.1.