Skip to content

Hunyuan VAE tiling fixes and transformer docs #10295

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Dec 19, 2024
Merged

Hunyuan VAE tiling fixes and transformer docs #10295

merged 3 commits into from
Dec 19, 2024

Conversation

@a-r-r-o-w
Copy link
Contributor

While training loras, found some things that slipped by when we merged HunyuanVideo

  • Enables gradient checkpointing in transformer
  • Fixes the encode implementation in VAE
  • Lowers the default frame tile size. A tile size of 64 requires ~24 GB memory, while a tile size of 16 requires only ~8 GB at 768x512 resolution
  • Adds docstring for transformer

@a-r-r-o-w a-r-r-o-w requested a review from yiyixuxu December 19, 2024 01:57
@HuggingFaceDocBuilderDev
Copy link

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

yiyixuxu
yiyixuxu approved these changes Dec 19, 2024
View reviewed changes
Copy link
Collaborator

@yiyixuxu yiyixuxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@a-r-r-o-w
Copy link
Contributor Author

The test now also matches the VAE in model test with VAE in transformer test for consistency

@yiyixuxu
Copy link
Collaborator

feel free to merge!

@a-r-r-o-w a-r-r-o-w merged commit f781b8c into main Dec 19, 2024
14 of 15 checks passed
@a-r-r-o-w a-r-r-o-w deleted the hunyuan-fixes branch December 19, 2024 04:58
Foundsheep pushed a commit to Foundsheep/diffusers that referenced this pull request Dec 23, 2024
@a-r-r-o-w @Foundsheep
Hunyuan VAE tiling fixes and transformer docs (huggingface#10295)
f8c1e4a 
* update

* udpate

* fix test
sayakpaul pushed a commit that referenced this pull request Dec 23, 2024
@a-r-r-o-w @sayakpaul
Hunyuan VAE tiling fixes and transformer docs (#10295)
6d7896e 
* update

* udpate

* fix test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiyixuxu yiyixuxu yiyixuxu approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@a-r-r-o-w @HuggingFaceDocBuilderDev @yiyixuxu