Skip to content

Is it possible to do a V2V job while giving the first frame of the new video? #392

@warscape

Description

@warscape

I mean the new video will copy the motion or pose of the video input, while its frames are generated based on an image input.
It looks like we can use the samples and the image_cond_latents inputs to do this, but my tries failed.
I've checked the source code and it seems these two inputs can't work together.

   if samples is not None:
        if len(samples["samples"].shape) == 5:
            B, T, C, H, W = samples["samples"].shape
            latents = samples["samples"]
        if len(samples["samples"].shape) == 4:
            B, C, H, W = samples["samples"].shape
            latents = None
    if image_cond_latents is not None:
        B, T, C, H, W = image_cond_latents["samples"].shape

Is there a way to do this at all?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions