Compatibility with clarity_V3-inpainting.safetensors

[Gerrix90]

Does anyone know if I can use OpenVINO with clarity_V3-inpainting.safetensors?
I've tried many times with different settings, and I always get a different error. Maybe this is not the right place to ask. If you know a better place, let me know.

[ilya-lavrenov]

Hi @Gerrix90

I've tried many times with different settings, and I always get a different error

What have you tried to do?

clarity_V3-inpainting.safetensors

Is a file with model weights. Where is the model itself?
You need to convert model to OpenVINO IR format and then run it via OpenVINO API

[Gerrix90]

I've installed https://github.com/openvinotoolkit/stable-diffusion-webui.

It allows me to accelerate photo generation by using my GPU instead of my CPU with the help of the 'Accelerate with OpenVino' script. For some models, it works like a charm. I didn't do any conversion for those "working" models myself. However, when I try the same with clarity_V3-inpainting.safetensors, I can't achieve the same success.

I read that I need to convert it into OpenVINO IR, but it was working before without any conversion on my part, and I thought I could do the same without having to convert it.

If I need to convert it manually, please tell me where to place the converted model. I've provided screenshots of the UI with the settings and the error that I'm getting for those settings. If I don't use the 'Accelerate with OpenVino' script, everything works fine but is significantly slower.

If you see that I'm misunderstanding something about all of this, please point me in the right direction where I can learn more. I would really appreciate it.

Thank you in advance!

To create a public link, set `share=True` in `launch()`.
Creating model from config: C:\Projects\sd_env\stable-diffusion-webui\configs\v1-inpainting-inference.yaml
Startup time: 12.4s (import torch: 4.9s, import gradio: 1.7s, setup paths: 1.3s, initialize shared: 0.2s, other imports: 1.1s, setup codeformer: 0.2s, load scripts: 1.6s, create ui: 0.9s, gradio launch: 0.5s).
Applying attention optimization: InvokeAI... done.
Model loaded in 10.2s (load weights from disk: 1.2s, create model: 0.5s, apply weights to model: 8.3s, calculate empty prompt: 0.2s).
{'Mask blur': 4}
Loading weights [d9be14ba27] from C:\Projects\sd_env\stable-diffusion-webui\models\Stable-diffusion\clarity_V3-inpainting.safetensors
OpenVINO Script:  created model from config : C:\Projects\sd_env\stable-diffusion-webui\configs\v1-inpainting-inference.yaml
*** Error completing request
*** Arguments: ('task(gi1amigbir1lujy)', 2, 'wearing a professional suit with a tie, female, (1girl), high heels, black pants', '(monochrome:1.3), (deformed, distorted, disfigured:1.3), (hair), jeans, tattoo, wet, water, clothing, shadow, 3d render, cartoon, ((blurry)), duplicate, ((duplicate body parts)), (disfigured), (poorly drawn), ((missing limbs)), logo, signature, text, words, low res, boring, artifacts, bad art, gross, ugly, poor quality, low quality, poorly drawn, bad anatomy, wrong anatomy\u200b, nsfw, nude', [], <PIL.Image.Image image mode=RGBA size=2124x2940 at 0x1DD41EA29D0>, None, {'image': <PIL.Image.Image image mode=RGBA size=2124x2940 at 0x1DD43041550>, 'mask': <PIL.Image.Image image mode=RGB size=2124x2940 at 0x1DD430F2FD0>}, None, None, None, None, 20, 'Euler a', 4, 0, 1, 1, 1, 7, 1.5, 0.7, 0, 704, 512, 1, 1, 1, 32, 0, '', '', '', [], False, [], '', <gradio.routes.Request object at 0x000001DD413DC790>, 3, False, '', 0.8, -1, False, -1, 0, 0, 0, UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', inpaint_crop_input_image=True, hr_option='Both', save_detected_map=True, advanced_weighting=None), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', inpaint_crop_input_image=True, hr_option='Both', save_detected_map=True, advanced_weighting=None), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', inpaint_crop_input_image=True, hr_option='Both', save_detected_map=True, advanced_weighting=None), '* `CFG Scale` should be 2 or lower.', True, True, '', '', True, 50, True, 1, 0, False, 4, 0.5, 'Linear', 'None', 'None', 'None', 'GPU', True, 'Euler a', True, True, 'clarity_V3-inpainting.safetensors', 0.8, '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, '', '<p style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and height sliders to set tile size</p>', 64, 0, 2, 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False, None, None, False, None, None, False, None, None, False, 50) {}
    Traceback (most recent call last):
      File "C:\Projects\sd_env\stable-diffusion-webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
                   ^^^^^^^^^^^^^^^^^^^^^
      File "C:\Projects\sd_env\stable-diffusion-webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^
      File "C:\Projects\sd_env\stable-diffusion-webui\modules\img2img.py", line 206, in img2img
        processed = modules.scripts.scripts_img2img.run(p, *args)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "C:\Projects\sd_env\stable-diffusion-webui\modules\scripts.py", line 601, in run
        processed = script.run(p, *script_args)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "C:\Projects\sd_env\stable-diffusion-webui\scripts\openvino_accelerate.py", line 1235, in run
        processed = process_images_openvino(p, model_config, vae_ckpt, p.sampler_name, enable_caching, openvino_device, mode, is_xl_ckpt, refiner_ckpt, refiner_frac)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "C:\Projects\sd_env\stable-diffusion-webui\scripts\openvino_accelerate.py", line 895, in process_images_openvino
        shared.sd_diffusers_model = get_diffusers_sd_model(model_config, vae_ckpt, sampler_name, enable_caching, openvino_device, mode, is_xl_ckpt, refiner_ckpt, refiner_frac)
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "C:\Projects\sd_env\stable-diffusion-webui\scripts\openvino_accelerate.py", line 556, in get_diffusers_sd_model
        sd_model = StableDiffusionXLPipeline.from_single_file(checkpoint_path, original_config_file=checkpoint_config, use_safetensors=True, add_watermark=False, variant="fp32", dtype=torch.float32)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "C:\Projects\sd_env\Lib\site-packages\diffusers\loaders.py", line 2822, in from_single_file
        pipe = download_from_original_stable_diffusion_ckpt(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "C:\Projects\sd_env\Lib\site-packages\diffusers\pipelines\stable_diffusion\convert_from_ckpt.py", line 1670, in download_from_original_stable_diffusion_ckpt
        pipe = pipeline_class(
               ^^^^^^^^^^^^^^^
    TypeError: StableDiffusionXLPipeline.__init__() got an unexpected keyword argument 'safety_checker'

---