ControlNetInpaintPipeline: Errors for loading Controlnet Diffusers folder #37
Description
Thanks for your work, After I converted the Controlnet.pth to diffusers, an error still occurred while loading. Did my conversion fail?How can I solve this problem？The error log is as follows：
The config attributes {'controlnet_hint_channels': 3} were passed to UNet2DConditionModel, but are not expected and will be ignored. Please verify your config.json configuration file.
Traceback (most recent call last):
File "/home/sd/Harddisk/zxp/ControlNet-for-Diffusers/Test.py", line 8, in
pipe_control = StableDiffusionControlNetInpaintPipeline.from_pretrained("./control_sd15_seg",torch_dtype=torch.float16).to('cuda')
File "/home/sd/miniconda3/envs/Controlnet/lib/python3.9/site-packages/diffusers/pipelines/pipeline_utils.py", line 865, in from_pretrained
loaded_sub_model = load_method(os.path.join(cached_folder, name), **loading_kwargs)
File "/home/sd/miniconda3/envs/Controlnet/lib/python3.9/site-packages/diffusers/models/modeling_utils.py", line 567, in from_pretrained
raise ValueError(
ValueError: Cannot load <class 'diffusers.models.unet_2d_condition.UNet2DConditionModel'> from ./control_sd15_seg/controlnet because the following keys are missing:
up_blocks.1.resnets.2.conv_shortcut.weight, ... ....
up_blocks.2.resnets.2.norm2.bias, up_blocks.3.resnets.0.norm2.weight, up_blocks.2.resnets.0.conv1.bias, up_blocks.1.resnets.1.norm1.bias.
Please make sure to pass low_cpu_mem_usage=False and device_map=None if you want to randomely initialize those weights or else make sure your checkpoint file is correct.