-
Couldn't load subscription status.
- Fork 6.5k
Mochi docs #9934
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mochi docs #9934
Changes from 3 commits
dd65ae5
ee28596
9ba4aaf
fc7d50b
9519ffc
4c64987
25aea7d
dd40011
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -13,7 +13,7 @@ | |
| # limitations under the License. | ||
| --> | ||
|
|
||
| # Mochi | ||
| # Mochi 1 Preview | ||
|
|
||
| [Mochi 1 Preview](https://huggingface.co/genmo/mochi-1-preview) from Genmo. | ||
|
|
||
|
|
@@ -25,6 +25,50 @@ Make sure to check out the Schedulers [guide](../../using-diffusers/schedulers.m | |
|
|
||
| </Tip> | ||
|
|
||
| ## Generating videos with Mochi-1 Preview | ||
|
|
||
| The following example will download the full precision `mochi-1-preview` weights and produce the highest quality results but will require at least 42GB VRAM to run. | ||
|
|
||
| ```python | ||
| import torch | ||
| from diffusers import MochiPipeline | ||
| from diffusers.utils import export_to_video | ||
|
|
||
| pipe = MochiPipeline.from_pretrained("genmo/mochi-1-preview") | ||
|
|
||
| # Enable memory savings | ||
| pipe.enable_model_cpu_offload() | ||
| pipe.enable_vae_tiling() | ||
|
|
||
| prompt = "Close-up of a chameleon's eye, with its scaly skin changing color. Ultra high resolution 4k." | ||
|
|
||
| with torch.autocast("cuda", torch.bfloat16, cache_enabled=False): | ||
| frames = pipe(prompt, num_frames=84).frames[0] | ||
|
||
|
|
||
| export_to_video(frames, "mochi.mp4", fps=30) | ||
| ``` | ||
|
|
||
| ## Using a lower precision variant to save memory | ||
|
|
||
| The following example will use the `bfloat16` variant of the model and requires 22GB VRAM to run. There is a slight drop in the quality of the generated video as a result. | ||
sayakpaul marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ```python | ||
| import torch | ||
| from diffusers import MochiPipeline | ||
| from diffusers.utils import export_to_video | ||
|
|
||
| pipe = MochiPipeline.from_pretrained("genmo/mochi-1-preview", variant="bf16", torch_dtype=torch.bfloat16) | ||
|
|
||
| # Enable memory savings | ||
| pipe.enable_model_cpu_offload() | ||
| pipe.enable_vae_tiling() | ||
|
|
||
| prompt = "Close-up of a chameleon's eye, with its scaly skin changing color. Ultra high resolution 4k." | ||
| frames = pipe(prompt, num_frames=84).frames[0] | ||
|
||
|
|
||
| export_to_video(frames, "mochi.mp4", fps=30) | ||
| ``` | ||
|
|
||
| ## MochiPipeline | ||
|
|
||
| [[autodoc]] MochiPipeline | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(One line above this) Only FlowMatchEulerDiscreteScheduler has invert_sigmas, so anything else wouldn't work as of now as I understand it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cc: @hlky here too. @DN6 do we wanna remove this bit? I think we need to remove it from all flow pipelines.