Skip to content

Commit 40de88a

Browse files
authored
[docs] AutoModel (huggingface#12644)
* automodel * fix
1 parent 6a2309b commit 40de88a

File tree

3 files changed

+49
-9
lines changed

3 files changed

+49
-9
lines changed

docs/source/en/_toctree.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@
2222
title: Reproducibility
2323
- local: using-diffusers/schedulers
2424
title: Schedulers
25+
- local: using-diffusers/automodel
26+
title: AutoModel
2527
- local: using-diffusers/other-formats
2628
title: Model formats
2729
- local: using-diffusers/push_to_hub

docs/source/en/api/models/auto_model.md

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -12,15 +12,7 @@ specific language governing permissions and limitations under the License.
1212

1313
# AutoModel
1414

15-
The `AutoModel` is designed to make it easy to load a checkpoint without needing to know the specific model class. `AutoModel` automatically retrieves the correct model class from the checkpoint `config.json` file.
16-
17-
```python
18-
from diffusers import AutoModel, AutoPipelineForText2Image
19-
20-
unet = AutoModel.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", subfolder="unet")
21-
pipe = AutoPipelineForText2Image.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", unet=unet)
22-
```
23-
15+
[`AutoModel`] automatically retrieves the correct model class from the checkpoint `config.json` file.
2416

2517
## AutoModel
2618

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
<!--Copyright 2025 The HuggingFace Team. All rights reserved.
2+
3+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
4+
the License. You may obtain a copy of the License at
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
9+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
10+
specific language governing permissions and limitations under the License.
11+
-->
12+
13+
# AutoModel
14+
15+
The [`AutoModel`] class automatically detects and loads the correct model class (UNet, transformer, VAE) from a `config.json` file. You don't need to know the specific model class name ahead of time. It supports data types and device placement, and works across model types and libraries.
16+
17+
The example below loads a transformer from Diffusers and a text encoder from Transformers. Use the `subfolder` parameter to specify where to load the `config.json` file from.
18+
19+
```py
20+
import torch
21+
from diffusers import AutoModel, DiffusionPipeline
22+
23+
transformer = AutoModel.from_pretrained(
24+
"Qwen/Qwen-Image", subfolder="transformer", torch_dtype=torch.bfloat16, device_map="cuda"
25+
)
26+
27+
text_encoder = AutoModel.from_pretrained(
28+
"Qwen/Qwen-Image", subfolder="text_encoder", torch_dtype=torch.bfloat16, device_map="cuda"
29+
)
30+
```
31+
32+
[`AutoModel`] also loads models from the [Hub](https://huggingface.co/models) that aren't included in Diffusers. Set `trust_remote_code=True` in [`AutoModel.from_pretrained`] to load custom models.
33+
34+
```py
35+
import torch
36+
from diffusers import AutoModel
37+
38+
transformer = AutoModel.from_pretrained(
39+
"custom/custom-transformer-model", trust_remote_code=True, torch_dtype=torch.bfloat16, device_map="cuda"
40+
)
41+
```
42+
43+
If the custom model inherits from the [`ModelMixin`] class, it gets access to the same features as Diffusers model classes, like [regional compilation](../optimization/fp16#regional-compilation) and [group offloading](../optimization/memory#group-offloading).
44+
45+
> [!NOTE]
46+
> Learn more about implementing custom models in the [Community components](../using-diffusers/custom_pipeline_overview#community-components) guide.

0 commit comments

Comments
 (0)