Skip to content

Commit 71843a0

Browse files
sayakpaulsayakpaul
authored
Merge branch 'main' into local-model-info
2 parents d117474 + 38740dd commit 71843a0

File tree

127 files changed

+3844
-4238
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

127 files changed

+3844
-4238
lines changed

‎.github/workflows/pr_modular_tests.yml‎

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
name: Fast PR tests for Modular
2+
3+
on:
4+
pull_request:
5+
branches: [main]
6+
paths:
7+
- "src/diffusers/modular_pipelines/**.py"
8+
- "src/diffusers/models/modeling_utils.py"
9+
- "src/diffusers/models/model_loading_utils.py"
10+
- "src/diffusers/pipelines/pipeline_utils.py"
11+
- "src/diffusers/pipeline_loading_utils.py"
12+
- "src/diffusers/loaders/lora_base.py"
13+
- "src/diffusers/loaders/lora_pipeline.py"
14+
- "src/diffusers/loaders/peft.py"
15+
- "tests/modular_pipelines/**.py"
16+
- ".github/**.yml"
17+
- "utils/**.py"
18+
- "setup.py"
19+
push:
20+
branches:
21+
- ci-*
22+
23+
concurrency:
24+
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
25+
cancel-in-progress: true
26+
27+
env:
28+
DIFFUSERS_IS_CI: yes
29+
HF_HUB_ENABLE_HF_TRANSFER: 1
30+
OMP_NUM_THREADS: 4
31+
MKL_NUM_THREADS: 4
32+
PYTEST_TIMEOUT: 60
33+
34+
jobs:
35+
check_code_quality:
36+
runs-on: ubuntu-22.04
37+
steps:
38+
- uses: actions/checkout@v3
39+
- name: Set up Python
40+
uses: actions/setup-python@v4
41+
with:
42+
python-version: "3.10"
43+
- name: Install dependencies
44+
run: |
45+
python -m pip install --upgrade pip
46+
pip install .[quality]
47+
- name: Check quality
48+
run: make quality
49+
- name: Check if failure
50+
if: ${{ failure() }}
51+
run: |
52+
echo "Quality check failed. Please ensure the right dependency versions are installed with 'pip install -e .[quality]' and run 'make style && make quality'" >> $GITHUB_STEP_SUMMARY
53+
54+
check_repository_consistency:
55+
needs: check_code_quality
56+
runs-on: ubuntu-22.04
57+
steps:
58+
- uses: actions/checkout@v3
59+
- name: Set up Python
60+
uses: actions/setup-python@v4
61+
with:
62+
python-version: "3.10"
63+
- name: Install dependencies
64+
run: |
65+
python -m pip install --upgrade pip
66+
pip install .[quality]
67+
- name: Check repo consistency
68+
run: |
69+
python utils/check_copies.py
70+
python utils/check_dummies.py
71+
python utils/check_support_list.py
72+
make deps_table_check_updated
73+
- name: Check if failure
74+
if: ${{ failure() }}
75+
run: |
76+
echo "Repo consistency check failed. Please ensure the right dependency versions are installed with 'pip install -e .[quality]' and run 'make fix-copies'" >> $GITHUB_STEP_SUMMARY
77+
78+
run_fast_tests:
79+
needs: [check_code_quality, check_repository_consistency]
80+
strategy:
81+
fail-fast: false
82+
matrix:
83+
config:
84+
- name: Fast PyTorch Modular Pipeline CPU tests
85+
framework: pytorch_pipelines
86+
runner: aws-highmemory-32-plus
87+
image: diffusers/diffusers-pytorch-cpu
88+
report: torch_cpu_modular_pipelines
89+
90+
name: ${{ matrix.config.name }}
91+
92+
runs-on:
93+
group: ${{ matrix.config.runner }}
94+
95+
container:
96+
image: ${{ matrix.config.image }}
97+
options: --shm-size "16gb" --ipc host -v /mnt/hf_cache:/mnt/cache/
98+
99+
defaults:
100+
run:
101+
shell: bash
102+
103+
steps:
104+
- name: Checkout diffusers
105+
uses: actions/checkout@v3
106+
with:
107+
fetch-depth: 2
108+
109+
- name: Install dependencies
110+
run: |
111+
python -m venv /opt/venv && export PATH="/opt/venv/bin:$PATH"
112+
python -m uv pip install -e [quality,test]
113+
pip uninstall transformers -y && python -m uv pip install -U transformers@git+https://github.com/huggingface/transformers.git --no-deps
114+
pip uninstall accelerate -y && python -m uv pip install -U accelerate@git+https://github.com/huggingface/accelerate.git --no-deps
115+
116+
- name: Environment
117+
run: |
118+
python -m venv /opt/venv && export PATH="/opt/venv/bin:$PATH"
119+
python utils/print_env.py
120+
121+
- name: Run fast PyTorch Pipeline CPU tests
122+
if: ${{ matrix.config.framework == 'pytorch_pipelines' }}
123+
run: |
124+
python -m venv /opt/venv && export PATH="/opt/venv/bin:$PATH"
125+
python -m pytest -n 8 --max-worker-restart=0 --dist=loadfile \
126+
-s -v -k "not Flax and not Onnx" \
127+
--make-reports=tests_${{ matrix.config.report }} \
128+
tests/modular_pipelines
129+
130+
- name: Failure short reports
131+
if: ${{ failure() }}
132+
run: cat reports/tests_${{ matrix.config.report }}_failures_short.txt
133+
134+
- name: Test suite reports artifacts
135+
if: ${{ always() }}
136+
uses: actions/upload-artifact@v4
137+
with:
138+
name: pr_${{ matrix.config.framework }}_${{ matrix.config.report }}_test_reports
139+
path: reports
140+
141+

‎docs/source/en/_toctree.yml‎

Lines changed: 25 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -112,22 +112,24 @@
112112
sections:
113113
- local: modular_diffusers/overview
114114
title: Overview
115-
- local: modular_diffusers/modular_pipeline
116-
title: Modular Pipeline
117-
- local: modular_diffusers/components_manager
118-
title: Components Manager
115+
- local: modular_diffusers/quickstart
116+
title: Quickstart
119117
- local: modular_diffusers/modular_diffusers_states
120-
title: Modular Diffusers States
118+
title: States
121119
- local: modular_diffusers/pipeline_block
122-
title: Pipeline Block
120+
title: ModularPipelineBlocks
123121
- local: modular_diffusers/sequential_pipeline_blocks
124-
title: Sequential Pipeline Blocks
122+
title: SequentialPipelineBlocks
125123
- local: modular_diffusers/loop_sequential_pipeline_blocks
126-
title: Loop Sequential Pipeline Blocks
124+
title: LoopSequentialPipelineBlocks
127125
- local: modular_diffusers/auto_pipeline_blocks
128-
title: Auto Pipeline Blocks
129-
- local: modular_diffusers/end_to_end_guide
130-
title: End-to-End Example
126+
title: AutoPipelineBlocks
127+
- local: modular_diffusers/modular_pipeline
128+
title: ModularPipeline
129+
- local: modular_diffusers/components_manager
130+
title: ComponentsManager
131+
- local: modular_diffusers/guiders
132+
title: Guiders
131133

132134
- title: Training
133135
isExpanded: false
@@ -282,6 +284,18 @@
282284
title: Outputs
283285
- local: api/quantization
284286
title: Quantization
287+
- title: Modular
288+
sections:
289+
- local: api/modular_diffusers/pipeline
290+
title: Pipeline
291+
- local: api/modular_diffusers/pipeline_blocks
292+
title: Blocks
293+
- local: api/modular_diffusers/pipeline_states
294+
title: States
295+
- local: api/modular_diffusers/pipeline_components
296+
title: Components and configs
297+
- local: api/modular_diffusers/guiders
298+
title: Guiders
285299
- title: Loaders
286300
sections:
287301
- local: api/loaders/ip_adapter

‎docs/source/en/api/modular_diffusers/guiders.md‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Guiders
2+
3+
Guiders are components in Modular Diffusers that control how the diffusion process is guided during generation. They implement various guidance techniques to improve generation quality and control.
4+
5+
## BaseGuidance
6+
7+
[[autodoc]] diffusers.guiders.guider_utils.BaseGuidance
8+
9+
## ClassifierFreeGuidance
10+
11+
[[autodoc]] diffusers.guiders.classifier_free_guidance.ClassifierFreeGuidance
12+
13+
## ClassifierFreeZeroStarGuidance
14+
15+
[[autodoc]] diffusers.guiders.classifier_free_zero_star_guidance.ClassifierFreeZeroStarGuidance
16+
17+
## SkipLayerGuidance
18+
19+
[[autodoc]] diffusers.guiders.skip_layer_guidance.SkipLayerGuidance
20+
21+
## SmoothedEnergyGuidance
22+
23+
[[autodoc]] diffusers.guiders.smoothed_energy_guidance.SmoothedEnergyGuidance
24+
25+
## PerturbedAttentionGuidance
26+
27+
[[autodoc]] diffusers.guiders.perturbed_attention_guidance.PerturbedAttentionGuidance
28+
29+
## AdaptiveProjectedGuidance
30+
31+
[[autodoc]] diffusers.guiders.adaptive_projected_guidance.AdaptiveProjectedGuidance
32+
33+
## AutoGuidance
34+
35+
[[autodoc]] diffusers.guiders.auto_guidance.AutoGuidance
36+
37+
## TangentialClassifierFreeGuidance
38+
39+
[[autodoc]] diffusers.guiders.tangential_classifier_free_guidance.TangentialClassifierFreeGuidance

‎docs/source/en/api/modular_diffusers/pipeline.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Pipeline
2+
3+
## ModularPipeline
4+
5+
[[autodoc]] diffusers.modular_pipelines.modular_pipeline.ModularPipeline

‎docs/source/en/api/modular_diffusers/pipeline_blocks.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Pipeline blocks
2+
3+
## ModularPipelineBlocks
4+
5+
[[autodoc]] diffusers.modular_pipelines.modular_pipeline.ModularPipelineBlocks
6+
7+
## SequentialPipelineBlocks
8+
9+
[[autodoc]] diffusers.modular_pipelines.modular_pipeline.SequentialPipelineBlocks
10+
11+
## LoopSequentialPipelineBlocks
12+
13+
[[autodoc]] diffusers.modular_pipelines.modular_pipeline.LoopSequentialPipelineBlocks
14+
15+
## AutoPipelineBlocks
16+
17+
[[autodoc]] diffusers.modular_pipelines.modular_pipeline.AutoPipelineBlocks

‎docs/source/en/api/modular_diffusers/pipeline_components.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Components and configs
2+
3+
## ComponentSpec
4+
5+
[[autodoc]] diffusers.modular_pipelines.modular_pipeline.ComponentSpec
6+
7+
## ConfigSpec
8+
9+
[[autodoc]] diffusers.modular_pipelines.modular_pipeline.ConfigSpec
10+
11+
## ComponentsManager
12+
13+
[[autodoc]] diffusers.modular_pipelines.components_manager.ComponentsManager
14+
15+
## InsertableDict
16+
17+
[[autodoc]] diffusers.modular_pipelines.modular_pipeline_utils.InsertableDict

‎docs/source/en/api/modular_diffusers/pipeline_states.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Pipeline states
2+
3+
## PipelineState
4+
5+
[[autodoc]] diffusers.modular_pipelines.modular_pipeline.PipelineState
6+
7+
## BlockState
8+
9+
[[autodoc]] diffusers.modular_pipelines.modular_pipeline.BlockState

‎docs/source/en/api/pipelines/flux.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,8 @@ Original model checkpoints for Flux can be found [here](https://huggingface.co/b
2525

2626
Flux can be quite expensive to run on consumer hardware devices. However, you can perform a suite of optimizations to run it faster and in a more memory-friendly manner. Check out [this section](https://huggingface.co/blog/sd3#memory-optimizations-for-sd3) for more details. Additionally, Flux can benefit from quantization for memory efficiency with a trade-off in inference latency. Refer to [this blog post](https://huggingface.co/blog/quanto-diffusers) to learn more. For an exhaustive list of resources, check out [this gist](https://gist.github.com/sayakpaul/b664605caf0aa3bf8585ab109dd5ac9c).
2727

28+
[Caching](../../optimization/cache) may also speed up inference by storing and reusing intermediate outputs.
29+
2830
</Tip>
2931

3032
Flux comes in the following variants:

‎docs/source/en/api/pipelines/hidream.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818

1919
<Tip>
2020

21-
Make sure to check out the Schedulers [guide](../../using-diffusers/schedulers) to learn how to explore the tradeoff between scheduler speed and quality, and see the [reuse components across pipelines](../../using-diffusers/loading#reuse-a-pipeline) section to learn how to efficiently load the same components into multiple pipelines.
21+
[Caching](../../optimization/cache) may also speed up inference by storing and reusing intermediate outputs.
2222

2323
</Tip>
2424

‎docs/source/en/api/pipelines/ltx_video.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,7 @@ export_to_video(video, "output.mp4", fps=24)
8888
</hfoption>
8989
<hfoption id="inference speed">
9090

91-
[Compilation](../../optimization/fp16#torchcompile) is slow the first time but subsequent calls to the pipeline are faster.
91+
[Compilation](../../optimization/fp16#torchcompile) is slow the first time but subsequent calls to the pipeline are faster. [Caching](../../optimization/cache) may also speed up inference by storing and reusing intermediate outputs.
9292

9393
```py
9494
import torch

0 commit comments

Comments
 (0)