You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/source/en/modular_diffusers/getting_started.md
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -692,9 +692,9 @@ ComponentSpec(
692
692
693
693
### Customizing Guidance Techniques
694
694
695
-
Guiders are guidance techniques that can be applied during the denoising process to improve generation quality, control, and adherence to prompts. They work by modifying the noise predictions or model behavior to steer the generation process in desired directions. In diffusers, guiders are implemented as subclasses of `BaseGuidance` and can be easily integrated into modular pipelines, providing a flexible way to enhance generation quality without modifying the underlying diffusion models.
695
+
Guiders are implementations of different [classifier-free guidance](https://huggingface.co/papers/2207.12598) techniques that can be applied during the denoising process to improve generation quality, control, and adherence to prompts. They work by steering the model predictions towards desired directions and away from undesired directions. In diffusers, guiders are implemented as subclasses of `BaseGuidance`. They can easily be integrated into modular pipelines and provide a flexible way to enhance generation quality without modifying the underlying diffusion models.
696
696
697
-
**ClassifierFreeGuidance (CFG)** is the first and most common guidance technique, used in all our standard pipelines. But we offer many more guidance techniques beyond CFG, including **PerturbedAttentionGuidance (PAG)**, **SkipLayerGuidance (SLG)**, **SmoothedEnergyGuidance (SEG)**, and others that can provide even better results for specific use cases.
697
+
**ClassifierFreeGuidance (CFG)** is the first and most common guidance technique, used in all our standard pipelines. We also offer many other guidance techniques from the latest research in this area - **PerturbedAttentionGuidance (PAG)**, **SkipLayerGuidance (SLG)**, **SmoothedEnergyGuidance (SEG)**, and others that can provide better results for specific use cases.
698
698
699
699
This section demonstrates how to use guiders using the component updating methods we just learned. Since `BaseGuidance` components are stateless (similar to schedulers), they are typically created with default configurations during pipeline initialization using `default_creation_method='from_config'`. This means they don't require loading specs from the repository - you won't see guider listed in `modular_model_index.json` files.
700
700
@@ -756,7 +756,7 @@ ClassifierFreeGuidance {
756
756
757
757
#### Switch to a Different Guider Type
758
758
759
-
Since guiders are `from_config` components (ConfigMixin objects), you can pass guider objects directly to switch between different guidance techniques:
759
+
Switching between guidance techniques is as simple as passing a guider object of that technique:
760
760
761
761
```py
762
762
from diffusers import LayerSkipConfig, PerturbedAttentionGuidance
@@ -776,7 +776,7 @@ ModularPipeline.update_components: adding guider with new type: PerturbedAttenti
776
776
<Tip>
777
777
778
778
💡 **Component Loading Methods**:
779
-
- For `from_config` components (like guiders, schedulers): You can pass the object directly OR pass a ComponentSpec directly (which calls `create()` under the hood)
779
+
- For `from_config` components (like guiders, schedulers): You can pass an object of required type OR pass a ComponentSpec directly (which calls `create()` under the hood)
780
780
- For `from_pretrained` components (like models): You must use ComponentSpec to ensure proper tagging and loading
781
781
782
782
</Tip>
@@ -915,7 +915,7 @@ Of course, you can also directly modify the `modular_model_index.json` to add a
915
915
-**SmoothedEnergyGuidance (SEG)**: Helps with energy distribution smoothing
916
916
-**AdaptiveProjectedGuidance (APG)**: Adaptive guidance that projects predictions for better quality
917
917
918
-
Experiment with different techniques and parameters to find what works best for your specific use case!
918
+
Experiment with different techniques and parameters to find what works best for your specific use case! Additionally, you can write your own guider implementations, for example, CFG Zero* combined with Skip Layer Guidance, and they should be compatible out-of-the-box with modular diffusers!
0 commit comments