Skip to content

Commit b680f16

Browse files
Enhance Pruna documentation with new examples and clarifications
- Added an image to illustrate the optimization process. - Updated the explanation for sharing and loading optimized models on the Hugging Face Hub. - Clarified the evaluation process for optimized models using the EvaluationAgent. - Improved descriptions for defining metrics and evaluating standalone diffusers models.
1 parent f6aeaad commit b680f16

File tree

1 file changed

+21
-25
lines changed

1 file changed

+21
-25
lines changed

docs/source/en/optimization/pruna.md

Lines changed: 21 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -81,27 +81,32 @@ smashed_pipe = smash(pipe, smash_config)
8181

8282
# run the model
8383
smashed_pipe("a knitted purple prune").images[0]
84+
```
8485

86+
<div class="flex justify-center">
87+
<img src="https://huggingface.co/datasets/PrunaAI/documentation-images/resolve/main/diffusers/flux_smashed_comparison.png">
88+
</div>
89+
90+
After optimization, we can share and load the optimized model using the Hugging Face Hub.
91+
92+
```python
8593
# save the model
8694
smashed_pipe.save_to_hub("<username>/FLUX.1-dev-smashed")
8795

8896
# load the model
8997
smashed_pipe = PrunaModel.from_hub("<username>/FLUX.1-dev-smashed")
9098
```
9199

92-
93-
<div class="flex justify-center">
94-
<img src="https://huggingface.co/datasets/PrunaAI/documentation-images/resolve/main/diffusers/flux_smashed_comparison.png">
95-
</div>
96-
97-
98100
## Evaluate and benchmark Diffusers models
99101

100102
Pruna provides the [EvaluationAgent](https://docs.pruna.ai/en/stable/docs_pruna/user_manual/evaluate.html) to evaluate the quality of your optimized models.
101103

102-
Define the metrics, such as total time and throughput, and the dataset to evaluate on. Then pass them to `Task` to create a task and pass it to the `EvaluationAgent`.
104+
We can metrics we care about, such as total time and throughput, and the dataset to evaluate on. We can define a model and pass it to the `EvaluationAgent`.
105+
106+
<hfoptions id="eval">
107+
<hfoption id="optimized model">
103108

104-
Call `evaluate` on the pipeline to execute the task passed to the `EvaluationAgent`.
109+
We can load and evaluate an optimized model by using the `EvaluationAgent` and pass it to the `Task`.
105110

106111
```python
107112
import torch
@@ -122,11 +127,6 @@ device = "cuda" if torch.cuda.is_available() else "mps" if torch.backends.mps.is
122127

123128
# load the model
124129
# Try PrunaAI/Segmind-Vega-smashed or PrunaAI/FLUX.1-dev-smashed with a small GPU memory
125-
pipe = FluxPipeline.from_pretrained(
126-
"black-forest-labs/FLUX.1-dev",
127-
torch_dtype=torch.bfloat16
128-
).to("cpu")
129-
wrapped_pipe = PrunaModel(model=pipe)
130130
smashed_pipe = PrunaModel.from_hub("PrunaAI/FLUX.1-dev-smashed")
131131

132132
# Define the metrics
@@ -144,26 +144,17 @@ datamodule.limit_datasets(10)
144144
task = Task(metrics, datamodule=datamodule, device=device)
145145
eval_agent = EvaluationAgent(task)
146146

147-
# Evaluate base model and offload it to CPU
148-
wrapped_pipe.move_to_device(device)
149-
base_model_results = eval_agent.evaluate(wrapped_pipe)
150-
wrapped_pipe.move_to_device("cpu")
151-
152147
# Evaluate smashed model and offload it to CPU
153148
smashed_pipe.move_to_device(device)
154-
smashed_model_results = eval_agent.evaluate(smashed_pipe)
149+
smashed_pipe_results = eval_agent.evaluate(smashed_pipe)
155150
smashed_pipe.move_to_device("cpu")
156151
```
157152

158-
> [!TIP]
159-
> For more details about benchmarking Flux, check out the [Announcing FLUX-Juiced: The Fastest Image Generation Endpoint (2.6 times faster)!](https://huggingface.co/blog/PrunaAI/flux-fastest-image-generation-endpoint) blog post and the [InferBench](https://huggingface.co/spaces/PrunaAI/InferBench) Space.
160-
161-
### Evaluate and benchmark standalone diffusers models
153+
</hfoption>
154+
<hfoption id="standalone model">
162155

163156
Instead of comparing the optimized model to the base model, you can also evaluate the standalone `diffusers` model. This is useful if you want to evaluate the performance of the model without the optimization. We can do so by using the `PrunaModel` wrapper and run the `EvaluationAgent` on it.
164157

165-
Let's take a look at an example on how to evaluate and benchmark a standalone `diffusers` model.
166-
167158
```python
168159
import torch
169160
from diffusers import FluxPipeline
@@ -179,8 +170,13 @@ pipe = FluxPipeline.from_pretrained(
179170
wrapped_pipe = PrunaModel(model=pipe)
180171
```
181172

173+
</hfoption>
174+
</hfoptions>
175+
182176
Now that you have seen how to optimize and evaluate your models, you can start using Pruna to optimize your own models. Luckily, we have many examples to help you get started.
183177

178+
> [!TIP]
179+
> For more details about benchmarking Flux, check out the [Announcing FLUX-Juiced: The Fastest Image Generation Endpoint (2.6 times faster)!](https://huggingface.co/blog/PrunaAI/flux-fastest-image-generation-endpoint) blog post and the [InferBench](https://huggingface.co/spaces/PrunaAI/InferBench) Space.
184180
185181
## Reference
186182

0 commit comments

Comments
 (0)