|
| 1 | +<!--Copyright 2025 The HuggingFace Team. All rights reserved. |
| 2 | +
|
| 3 | +Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with |
| 4 | +the License. You may obtain a copy of the License at |
| 5 | +
|
| 6 | +http://www.apache.org/licenses/LICENSE-2.0 |
| 7 | +
|
| 8 | +Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on |
| 9 | +an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the |
| 10 | +specific language governing permissions and limitations under the License. |
| 11 | +--> |
| 12 | + |
| 13 | +# AutoModel |
| 14 | + |
| 15 | +The [`AutoModel`] class automatically detects and loads the correct model class (UNet, transformer, VAE) from a `config.json` file. You don't need to know the specific model class name ahead of time. It supports data types and device placement, and works across model types and libraries. |
| 16 | + |
| 17 | +The example below loads a transformer from Diffusers and a text encoder from Transformers. Use the `subfolder` parameter to specify where to load the `config.json` file from. |
| 18 | + |
| 19 | +```py |
| 20 | +import torch |
| 21 | +from diffusers import AutoModel, DiffusionPipeline |
| 22 | + |
| 23 | +transformer = AutoModel.from_pretrained( |
| 24 | + "Qwen/Qwen-Image", subfolder="transformer", torch_dtype=torch.bfloat16, device_map="cuda" |
| 25 | +) |
| 26 | + |
| 27 | +text_encoder = AutoModel.from_pretrained( |
| 28 | + "Qwen/Qwen-Image", subfolder="text_encoder", torch_dtype=torch.bfloat16, device_map="cuda" |
| 29 | +) |
| 30 | +``` |
| 31 | + |
| 32 | +[`AutoModel`] also loads models from the [Hub](https://huggingface.co/models) that aren't included in Diffusers. Set `trust_remote_code=True` in [`AutoModel.from_pretrained`] to load custom models. |
| 33 | + |
| 34 | +```py |
| 35 | +import torch |
| 36 | +from diffusers import AutoModel |
| 37 | + |
| 38 | +transformer = AutoModel.from_pretrained( |
| 39 | + "custom/custom-transformer-model", trust_remote_code=True, torch_dtype=torch.bfloat16, device_map="cuda" |
| 40 | +) |
| 41 | +``` |
| 42 | + |
| 43 | +If the custom model inherits from the [`ModelMixin`] class, it gets access to the same features as Diffusers model classes, like [regional compilation](../optimization/fp16#regional-compilation) and [group offloading](../optimization/memory#group-offloading). |
| 44 | + |
| 45 | +> [!NOTE] |
| 46 | +> Learn more about implementing custom models in the [Community components](../using-diffusers/custom_pipeline_overview#community-components) guide. |
0 commit comments