Issue using ControlNet #1678

GigiGanzGross · 2023-03-03T19:09:42Z

GigiGanzGross
Mar 3, 2023

Hi,

I'm having issues with ControlNet since the beginning. I had watched several youtube videos about the correct settings and had reinstalled all components several times, but always received the following error I don't know how to fix. Also see a screenshot attached. Please help, I am super hyped to try this tool :)

Error
`Loading preprocessor: canny
0% 0/53 [00:00<?, ?it/s]
Error completing request
Arguments: ('task(0c03rvifwzz84qy)', 0, 'Anime, detail, flower, garden', '', [], <PIL.Image.Image image mode=RGBA size=512x512 at 0x7F80132EDE80>, None, None, None, None, None, None, 70, 0, 4, 0, 1, False, False, 1, 1, 7, 1.5, 0.75, 42.0, -1.0, 0, 0, 0, False, 1024, 768, 0, 0, 32, 0, '', '', '', [], 0, True, True, 'canny', 'control_canny-fp16 [e3fe7712]', 1, {'image': array([[[239, 245, 242],
[239, 245, 249],
[238, 248, 249],
...,
[145, 149, 204],
[167, 164, 198],
[184, 177, 190]],

   [[241, 246, 246],
    [239, 245, 249],
    [238, 242, 248],
    ...,
    [144, 151, 210],
    [163, 160, 198],
    [191, 181, 193]],

   [[237, 243, 244],
    [237, 244, 248],
    [236, 241, 247],
    ...,
    [144, 154, 209],
    [166, 165, 196],
    [194, 180, 186]],

   ...,

   [[225, 238, 247],
    [225, 236, 245],
    [224, 237, 244],
    ...,
    [ 54,  80, 216],
    [ 52,  80, 219],
    [ 48,  77, 221]],

   [[222, 238, 248],
    [223, 238, 245],
    [226, 237, 247],
    ...,

    [ 53,  78, 216],
    [ 52,  80, 219],
    [ 49,  76, 218]],

   [[222, 237, 242],
    [229, 243, 249],
    [227, 238, 245],
    ...,
    [ 52,  79, 213],
    [ 55,  84, 210],
    [ 58,  79, 199]]], dtype=uint8), 'mask': array([[[  0,   0,   0, 255],
    [  0,   0,   0, 255],
    [  0,   0,   0, 255],
    ...,
    [  0,   0,   0, 255],
    [  0,   0,   0, 255],
    [  0,   0,   0, 255]],

   [[  0,   0,   0, 255],
    [  0,   0,   0, 255],
    [  0,   0,   0, 255],
    ...,
    [  0,   0,   0, 255],
    [  0,   0,   0, 255],
    [  0,   0,   0, 255]],

   [[  0,   0,   0, 255],
    [  0,   0,   0, 255],
    [  0,   0,   0, 255],
    ...,
    [  0,   0,   0, 255],
    [  0,   0,   0, 255],
    [  0,   0,   0, 255]],

   ...,

   [[  0,   0,   0, 255],
    [  0,   0,   0, 255],
    [  0,   0,   0, 255],
    ...,
    [  0,   0,   0, 255],
    [  0,   0,   0, 255],
    [  0,   0,   0, 255]],

   [[  0,   0,   0, 255],
    [  0,   0,   0, 255],
    [  0,   0,   0, 255],
    ...,
    [  0,   0,   0, 255],
    [  0,   0,   0, 255],
    [  0,   0,   0, 255]],

   [[  0,   0,   0, 255],
    [  0,   0,   0, 255],
    [  0,   0,   0, 255],
    ...,
    [  0,   0,   0, 255],
    [  0,   0,   0, 255],
    [  0,   0,   0, 255]]], dtype=uint8)}, False, 'Just Resize', False, True, 512, 100, 200, 0, 1, False, '<ul>\n<li><code>CFG Scale</code> should be 2 or lower.</li>\n</ul>\n', True, True, '', '', True, 50, True, 1, 0, False, 4, 1, 'None', '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, '', '<p style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and height sliders to set tile size</p>', 64, 0, 2, 1, '', 0, '', 0, '', True, False, False, False, 0, None, 50) {}

Traceback (most recent call last):
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/call_queue.py", line 56, in f
res = list(func(*args, **kwargs))
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/img2img.py", line 171, in img2img
processed = process_images(p)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/processing.py", line 486, in process_images
res = process_images_inner(p)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/processing.py", line 632, in process_images_inner
samples_ddim = p.sample(conditioning=c, unconditional_conditioning=uc, seeds=seeds, subseeds=subseeds, subseed_strength=p.subseed_strength, prompts=prompts)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/processing.py", line 1048, in sample
samples = self.sampler.sample_img2img(self, self.init_latent, x, conditioning, unconditional_conditioning, image_conditioning=self.image_conditioning)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 322, in sample_img2img
samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 225, in launch_sampling
return func()
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 322, in
samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
File "/usr/local/lib/python3.8/dist-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/content/gdrive/MyDrive/sd/stablediffusion/src/k-diffusion/k_diffusion/sampling.py", line 145, in sample_euler_ancestral
denoised = model(x, sigmas[i] * s_in, **extra_args)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 117, in forward
x_out = self.inner_model(x_in, sigma_in, cond={"c_crossattn": [cond_in], "c_concat": [image_cond_in]})
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/content/gdrive/MyDrive/sd/stablediffusion/src/k-diffusion/k_diffusion/external.py", line 167, in forward
return self.get_v(input * c_in, self.sigma_to_t(sigma), **kwargs) * c_out + input * c_skip
File "/content/gdrive/MyDrive/sd/stablediffusion/src/k-diffusion/k_diffusion/external.py", line 177, in get_v
return self.inner_model.apply_model(x, t, cond)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_hijack_utils.py", line 17, in
setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_hijack_utils.py", line 28, in call
return self.__orig_func(*args, **kwargs)
File "/content/gdrive/MyDrive/sd/stablediffusion/ldm/models/diffusion/ddpm.py", line 858, in apply_model
x_recon = self.model(x_noisy, t, **cond)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/content/gdrive/MyDrive/sd/stablediffusion/ldm/models/diffusion/ddpm.py", line 1329, in forward
out = self.diffusion_model(x, t, context=cc)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/hook.py", line 193, in forward2
return forward(*args, **kwargs)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/hook.py", line 136, in forward
control = param.control_model(x=x_in, hint=param.hint_cond, timesteps=timesteps, context=context)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/cldm.py", line 115, in forward
return self.control_model(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/cldm.py", line 383, in forward
h = module(h, emb, context)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/content/gdrive/MyDrive/sd/stablediffusion/ldm/modules/diffusionmodules/openaimodel.py", line 84, in forward
x = layer(x, context)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/content/gdrive/MyDrive/sd/stablediffusion/ldm/modules/attention.py", line 334, in forward
x = block(x, context=context[i])
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/content/gdrive/MyDrive/sd/stablediffusion/ldm/modules/attention.py", line 269, in forward
return checkpoint(self._forward, (x, context), self.parameters(), self.checkpoint)
File "/content/gdrive/MyDrive/sd/stablediffusion/ldm/modules/diffusionmodules/util.py", line 114, in checkpoint
return CheckpointFunction.apply(func, len(inputs), *args)
File "/content/gdrive/MyDrive/sd/stablediffusion/ldm/modules/diffusionmodules/util.py", line 129, in forward
output_tensors = ctx.run_function(*ctx.input_tensors)
File "/content/gdrive/MyDrive/sd/stablediffusion/ldm/modules/attention.py", line 273, in _forward
x = self.attn2(self.norm2(x), context=context) + x
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/sd_hijack_optimizations.py", line 332, in xformers_attention_forward
k_in = self.to_k(context_k)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions-builtin/Lora/lora.py", line 178, in lora_Linear_forward
return lora_forward(self, input, torch.nn.Linear_forward_before_lora(self, input))
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (154x1024 and 768x320)`

GigiGanzGross · 2023-03-04T09:14:49Z

GigiGanzGross
Mar 4, 2023
Author

Sorry, it seems I placed the issue in the wrong region. I opened an issue.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Issue using ControlNet #1678

Uh oh!

{{title}}

Uh oh!

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Issue using ControlNet #1678

Uh oh!

GigiGanzGross Mar 3, 2023

Replies: 1 comment

Uh oh!

GigiGanzGross Mar 4, 2023 Author

GigiGanzGross
Mar 3, 2023

GigiGanzGross
Mar 4, 2023
Author