Skip to content

Commit 9747c0b

Browse files
authored
Merge branch 'main' into lumina-naming-consistency
2 parents 1e035c0 + 9c7e205 commit 9747c0b

File tree

202 files changed

+6464
-1759
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

202 files changed

+6464
-1759
lines changed
Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
name: "\U0001F31F Remote VAE"
2+
description: Feedback for remote VAE pilot
3+
labels: [ "Remote VAE" ]
4+
5+
body:
6+
- type: textarea
7+
id: positive
8+
validations:
9+
required: true
10+
attributes:
11+
label: Did you like the remote VAE solution?
12+
description: |
13+
If you liked it, we would appreciate it if you could elaborate what you liked.
14+
15+
- type: textarea
16+
id: feedback
17+
validations:
18+
required: true
19+
attributes:
20+
label: What can be improved about the current solution?
21+
description: |
22+
Let us know the things you would like to see improved. Note that we will work optimizing the solution once the pilot is over and we have usage.
23+
24+
- type: textarea
25+
id: others
26+
validations:
27+
required: true
28+
attributes:
29+
label: What other VAEs you would like to see if the pilot goes well?
30+
description: |
31+
Provide a list of the VAEs you would like to see in the future if the pilot goes well.
32+
33+
- type: textarea
34+
id: additional-info
35+
attributes:
36+
label: Notify the members of the team
37+
description: |
38+
Tag the following folks when submitting this feedback: @hlky @sayakpaul

.github/workflows/pr_style_bot.yml

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
name: PR Style Bot
2+
3+
on:
4+
issue_comment:
5+
types: [created]
6+
7+
permissions:
8+
contents: write
9+
pull-requests: write
10+
11+
jobs:
12+
run-style-bot:
13+
if: >
14+
contains(github.event.comment.body, '@bot /style') &&
15+
github.event.issue.pull_request != null
16+
runs-on: ubuntu-latest
17+
18+
steps:
19+
- name: Extract PR details
20+
id: pr_info
21+
uses: actions/github-script@v6
22+
with:
23+
script: |
24+
const prNumber = context.payload.issue.number;
25+
const { data: pr } = await github.rest.pulls.get({
26+
owner: context.repo.owner,
27+
repo: context.repo.repo,
28+
pull_number: prNumber
29+
});
30+
31+
// We capture both the branch ref and the "full_name" of the head repo
32+
// so that we can check out the correct repository & branch (including forks).
33+
core.setOutput("prNumber", prNumber);
34+
core.setOutput("headRef", pr.head.ref);
35+
core.setOutput("headRepoFullName", pr.head.repo.full_name);
36+
37+
- name: Check out PR branch
38+
uses: actions/checkout@v3
39+
env:
40+
HEADREPOFULLNAME: ${{ steps.pr_info.outputs.headRepoFullName }}
41+
HEADREF: ${{ steps.pr_info.outputs.headRef }}
42+
with:
43+
# Instead of checking out the base repo, use the contributor's repo name
44+
repository: ${{ env.HEADREPOFULLNAME }}
45+
ref: ${{ env.HEADREF }}
46+
# You may need fetch-depth: 0 for being able to push
47+
fetch-depth: 0
48+
token: ${{ secrets.GITHUB_TOKEN }}
49+
50+
- name: Debug
51+
env:
52+
HEADREPOFULLNAME: ${{ steps.pr_info.outputs.headRepoFullName }}
53+
HEADREF: ${{ steps.pr_info.outputs.headRef }}
54+
PRNUMBER: ${{ steps.pr_info.outputs.prNumber }}
55+
run: |
56+
echo "PR number: ${{ env.PRNUMBER }}"
57+
echo "Head Ref: ${{ env.HEADREF }}"
58+
echo "Head Repo Full Name: ${{ env.HEADREPOFULLNAME }}"
59+
60+
- name: Set up Python
61+
uses: actions/setup-python@v4
62+
63+
- name: Install dependencies
64+
run: |
65+
pip install .[quality]
66+
67+
- name: Download Makefile from main branch
68+
run: |
69+
curl -o main_Makefile https://raw.githubusercontent.com/huggingface/diffusers/main/Makefile
70+
71+
- name: Compare Makefiles
72+
run: |
73+
if ! diff -q main_Makefile Makefile; then
74+
echo "Error: The Makefile has changed. Please ensure it matches the main branch."
75+
exit 1
76+
fi
77+
echo "No changes in Makefile. Proceeding..."
78+
rm -rf main_Makefile
79+
80+
- name: Run make style and make quality
81+
run: |
82+
make style && make quality
83+
84+
- name: Commit and push changes
85+
id: commit_and_push
86+
env:
87+
HEADREPOFULLNAME: ${{ steps.pr_info.outputs.headRepoFullName }}
88+
HEADREF: ${{ steps.pr_info.outputs.headRef }}
89+
PRNUMBER: ${{ steps.pr_info.outputs.prNumber }}
90+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
91+
run: |
92+
echo "HEADREPOFULLNAME: ${{ env.HEADREPOFULLNAME }}, HEADREF: ${{ env.HEADREF }}"
93+
# Configure git with the Actions bot user
94+
git config user.name "github-actions[bot]"
95+
git config user.email "github-actions[bot]@users.noreply.github.com"
96+
97+
# Make sure your 'origin' remote is set to the contributor's fork
98+
git remote set-url origin "https://x-access-token:${GITHUB_TOKEN}@github.com/${{ env.HEADREPOFULLNAME }}.git"
99+
100+
# If there are changes after running style/quality, commit them
101+
if [ -n "$(git status --porcelain)" ]; then
102+
git add .
103+
git commit -m "Apply style fixes"
104+
# Push to the original contributor's forked branch
105+
git push origin HEAD:${{ env.HEADREF }}
106+
echo "changes_pushed=true" >> $GITHUB_OUTPUT
107+
else
108+
echo "No changes to commit."
109+
echo "changes_pushed=false" >> $GITHUB_OUTPUT
110+
fi
111+
112+
- name: Comment on PR with workflow run link
113+
if: steps.commit_and_push.outputs.changes_pushed == 'true'
114+
uses: actions/github-script@v6
115+
with:
116+
script: |
117+
const prNumber = parseInt(process.env.prNumber, 10);
118+
const runUrl = `${process.env.GITHUB_SERVER_URL}/${process.env.GITHUB_REPOSITORY}/actions/runs/${process.env.GITHUB_RUN_ID}`
119+
120+
await github.rest.issues.createComment({
121+
owner: context.repo.owner,
122+
repo: context.repo.repo,
123+
issue_number: prNumber,
124+
body: `Style fixes have been applied. [View the workflow run here](${runUrl}).`
125+
});
126+
env:
127+
prNumber: ${{ steps.pr_info.outputs.prNumber }}

.github/workflows/pr_tests.yml

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@ name: Fast tests for PRs
22

33
on:
44
pull_request:
5-
branches:
6-
- main
5+
branches: [main]
6+
types: [synchronize]
77
paths:
88
- "src/diffusers/**.py"
99
- "benchmarks/**.py"
@@ -64,6 +64,7 @@ jobs:
6464
run: |
6565
python utils/check_copies.py
6666
python utils/check_dummies.py
67+
python utils/check_support_list.py
6768
make deps_table_check_updated
6869
- name: Check if failure
6970
if: ${{ failure() }}
@@ -120,7 +121,8 @@ jobs:
120121
run: |
121122
python -m venv /opt/venv && export PATH="/opt/venv/bin:$PATH"
122123
python -m uv pip install -e [quality,test]
123-
python -m uv pip install accelerate
124+
pip uninstall transformers -y && python -m uv pip install -U transformers@git+https://github.com/huggingface/transformers.git --no-deps
125+
pip uninstall accelerate -y && python -m uv pip install -U accelerate@git+https://github.com/huggingface/accelerate.git --no-deps
124126
125127
- name: Environment
126128
run: |

.github/workflows/push_tests.yml

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,13 @@
11
name: Fast GPU Tests on main
22

33
on:
4+
pull_request:
5+
branches: main
6+
paths:
7+
- "src/diffusers/models/modeling_utils.py"
8+
- "src/diffusers/models/model_loading_utils.py"
9+
- "src/diffusers/pipelines/pipeline_utils.py"
10+
- "src/diffusers/pipeline_loading_utils.py"
411
workflow_dispatch:
512
push:
613
branches:
@@ -160,6 +167,7 @@ jobs:
160167
path: reports
161168

162169
flax_tpu_tests:
170+
if: ${{ github.event_name != 'pull_request' }}
163171
name: Flax TPU Tests
164172
runs-on:
165173
group: gcp-ct5lp-hightpu-8t
@@ -208,6 +216,7 @@ jobs:
208216
path: reports
209217

210218
onnx_cuda_tests:
219+
if: ${{ github.event_name != 'pull_request' }}
211220
name: ONNX CUDA Tests
212221
runs-on:
213222
group: aws-g4dn-2xlarge
@@ -256,6 +265,7 @@ jobs:
256265
path: reports
257266

258267
run_torch_compile_tests:
268+
if: ${{ github.event_name != 'pull_request' }}
259269
name: PyTorch Compile CUDA tests
260270

261271
runs-on:
@@ -299,6 +309,7 @@ jobs:
299309
path: reports
300310

301311
run_xformers_tests:
312+
if: ${{ github.event_name != 'pull_request' }}
302313
name: PyTorch xformers CUDA tests
303314

304315
runs-on:
@@ -349,7 +360,6 @@ jobs:
349360
container:
350361
image: diffusers/diffusers-pytorch-cuda
351362
options: --gpus 0 --shm-size "16gb" --ipc host
352-
353363
steps:
354364
- name: Checkout diffusers
355365
uses: actions/checkout@v3
@@ -359,7 +369,6 @@ jobs:
359369
- name: NVIDIA-SMI
360370
run: |
361371
nvidia-smi
362-
363372
- name: Install dependencies
364373
run: |
365374
python -m venv /opt/venv && export PATH="/opt/venv/bin:$PATH"

.github/workflows/run_tests_from_a_pr.yml

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,8 @@ on:
77
default: 'diffusers/diffusers-pytorch-cuda'
88
description: 'Name of the Docker image'
99
required: true
10-
branch:
11-
description: 'PR Branch to test on'
10+
pr_number:
11+
description: 'PR number to test on'
1212
required: true
1313
test:
1414
description: 'Tests to run (e.g.: `tests/models`).'
@@ -43,8 +43,8 @@ jobs:
4343
exit 1
4444
fi
4545
46-
if [[ ! "$PY_TEST" =~ ^tests/(models|pipelines) ]]; then
47-
echo "Error: The input string must contain either 'models' or 'pipelines' after 'tests/'."
46+
if [[ ! "$PY_TEST" =~ ^tests/(models|pipelines|lora) ]]; then
47+
echo "Error: The input string must contain either 'models', 'pipelines', or 'lora' after 'tests/'."
4848
exit 1
4949
fi
5050
@@ -53,13 +53,13 @@ jobs:
5353
exit 1
5454
fi
5555
echo "$PY_TEST"
56+
57+
shell: bash -e {0}
5658

5759
- name: Checkout PR branch
5860
uses: actions/checkout@v4
5961
with:
60-
ref: ${{ github.event.inputs.branch }}
61-
repository: ${{ github.event.pull_request.head.repo.full_name }}
62-
62+
ref: refs/pull/${{ inputs.pr_number }}/head
6363

6464
- name: Install pytest
6565
run: |

docs/source/en/api/activations.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,3 +25,16 @@ Customized activation functions for supporting various models in 🤗 Diffusers.
2525
## ApproximateGELU
2626

2727
[[autodoc]] models.activations.ApproximateGELU
28+
29+
30+
## SwiGLU
31+
32+
[[autodoc]] models.activations.SwiGLU
33+
34+
## FP32SiLU
35+
36+
[[autodoc]] models.activations.FP32SiLU
37+
38+
## LinearActivation
39+
40+
[[autodoc]] models.activations.LinearActivation

docs/source/en/api/attnprocessor.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,3 +147,20 @@ An attention processor is a class for applying different types of attention mech
147147
## XLAFlashAttnProcessor2_0
148148

149149
[[autodoc]] models.attention_processor.XLAFlashAttnProcessor2_0
150+
151+
## XFormersJointAttnProcessor
152+
153+
[[autodoc]] models.attention_processor.XFormersJointAttnProcessor
154+
155+
## IPAdapterXFormersAttnProcessor
156+
157+
[[autodoc]] models.attention_processor.IPAdapterXFormersAttnProcessor
158+
159+
## FluxIPAdapterJointAttnProcessor2_0
160+
161+
[[autodoc]] models.attention_processor.FluxIPAdapterJointAttnProcessor2_0
162+
163+
164+
## XLAFluxFlashAttnProcessor2_0
165+
166+
[[autodoc]] models.attention_processor.XLAFluxFlashAttnProcessor2_0

docs/source/en/api/loaders/lora.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ LoRA is a fast and lightweight training method that inserts and trains a signifi
2323
- [`LTXVideoLoraLoaderMixin`] provides similar functions for [LTX-Video](https://huggingface.co/docs/diffusers/main/en/api/pipelines/ltx_video).
2424
- [`SanaLoraLoaderMixin`] provides similar functions for [Sana](https://huggingface.co/docs/diffusers/main/en/api/pipelines/sana).
2525
- [`HunyuanVideoLoraLoaderMixin`] provides similar functions for [HunyuanVideo](https://huggingface.co/docs/diffusers/main/en/api/pipelines/hunyuan_video).
26+
- [`Lumina2LoraLoaderMixin`] provides similar functions for [Lumina2](https://huggingface.co/docs/diffusers/main/en/api/pipelines/lumina2).
2627
- [`AmusedLoraLoaderMixin`] is for the [`AmusedPipeline`].
2728
- [`LoraBaseMixin`] provides a base class with several utility methods to fuse, unfuse, unload, LoRAs and more.
2829

@@ -68,6 +69,10 @@ To learn more about how to load LoRA weights, see the [LoRA](../../using-diffuse
6869

6970
[[autodoc]] loaders.lora_pipeline.HunyuanVideoLoraLoaderMixin
7071

72+
## Lumina2LoraLoaderMixin
73+
74+
[[autodoc]] loaders.lora_pipeline.Lumina2LoraLoaderMixin
75+
7176
## AmusedLoraLoaderMixin
7277

7378
[[autodoc]] loaders.lora_pipeline.AmusedLoraLoaderMixin

docs/source/en/api/normalization.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,3 +29,43 @@ Customized normalization layers for supporting various models in 🤗 Diffusers.
2929
## AdaGroupNorm
3030

3131
[[autodoc]] models.normalization.AdaGroupNorm
32+
33+
## AdaLayerNormContinuous
34+
35+
[[autodoc]] models.normalization.AdaLayerNormContinuous
36+
37+
## RMSNorm
38+
39+
[[autodoc]] models.normalization.RMSNorm
40+
41+
## GlobalResponseNorm
42+
43+
[[autodoc]] models.normalization.GlobalResponseNorm
44+
45+
46+
## LuminaLayerNormContinuous
47+
[[autodoc]] models.normalization.LuminaLayerNormContinuous
48+
49+
## SD35AdaLayerNormZeroX
50+
[[autodoc]] models.normalization.SD35AdaLayerNormZeroX
51+
52+
## AdaLayerNormZeroSingle
53+
[[autodoc]] models.normalization.AdaLayerNormZeroSingle
54+
55+
## LuminaRMSNormZero
56+
[[autodoc]] models.normalization.LuminaRMSNormZero
57+
58+
## LpNorm
59+
[[autodoc]] models.normalization.LpNorm
60+
61+
## CogView3PlusAdaLayerNormZeroTextImage
62+
[[autodoc]] models.normalization.CogView3PlusAdaLayerNormZeroTextImage
63+
64+
## CogVideoXLayerNormZero
65+
[[autodoc]] models.normalization.CogVideoXLayerNormZero
66+
67+
## MochiRMSNormZero
68+
[[autodoc]] models.transformers.transformer_mochi.MochiRMSNormZero
69+
70+
## MochiRMSNorm
71+
[[autodoc]] models.normalization.MochiRMSNorm

0 commit comments

Comments
 (0)