What am I doing wrong with my Inpainting Workflow??

[poisenbery]

I'm an Automatic1111 user but was attracted to ComfyUI because of it's node based approach. I have about a decade of blender node experience, so I figured that this would be a perfect match for me.

I'm finding that I have no idea how to make this work with the inpainting workflow I am used to using in Automatic1111.

Normally, I create the base image, upscale, and then inpaint "only masked" by using the webUI to draw over the area, and setting around .3 denoise to add more details.

I can't seem to figure out how to accomplish this in comfyUI.

My thought process for the workflow was to generate the image, use ClipSeg to define the mask, and pass that through the "VAE Encode for Inpainting" with the mask, and then pass that through another sampler node with a low denoise. All it does is replace the masked area with grey nothing. I really have no idea what I am doing wrong or why this is happening.

I know that this is obviously a user error, but I am not smart enough to understand what I'm overlooking. Automatic1111 seems to have made the process so easy, that I have become stupid and cannot make it work in ComfyUI

Can someone tell me what I'm overlooking, and why I can't make this work?

[comfyanonymous]

don't use "conditioning set mask", it's not for inpainting, it's for applying a prompt to a specific area of the image

"VAE Encode for inpainting" should be used with denoise of 100%, it's for true inpainting and is best used with inpaint models but will work with all models.

If you want to do img2img but on a masked part of the image use latent->inpaint->"Set Latent Noise Mask" instead. Then you can set a lower denoise and it will work.

[poisenbery]

Thanks!

[gee666]

thank you! it helped me too!

[ghostsquad]

Can you define what "true inpainting" means? I'm not sure if I understand.

[belladoreai]

@ghostsquad I think they use "true inpainting" to mean inpainting where any information in the masked area is completely destroyed and replaced with new information, as opposed to approaches where the original image is used as a starting point and iterated upon (even if iterated upon with 1.0 denoise, it will still affect the result).

[poisenbery]

@comfyanonymous I actually had one more question: The first node runs at 12 iterations/second
But after I upscale the image and pass it downstream with the mask, the 2nd sampler node runs at 8 seconds/iteration.

Does this mean that it does does img2img "Whole Image" but only changes the masked area?

Is there a way to crop the mask and the image so that it operates like the "Only Masked" method in Automatic1111, where it only looks at the masked area and then pastes it over the original image? I was looking at the crop nodes, but it looks like I have to manually specify the cropped regions.

The face is extremely derpy, which is why I'm pretty sure it's operating similar to the "Whole Image" method in Automatic1111

[comfyanonymous]

yeah right now it samples the whole image. an easy way to sample only the masked area for local img2img is not implemented yet.

[poisenbery]

Is there a difficult way?

[sanguivore-easyco]

https://github.com/ltdrdata/ComfyUI-Impact-Pack#how-to-use-ddetailer-feature - it's possible to pipe masks through into the Detailer nodes that are in this custom node pack, which I believe work that way.

[poisenbery]

I'm trying to create a mask analyzer node that can detect the mask position and dimensions.

How do I enable logging in the console so I can troubleshoot?

[ltdrdata]

just use
print(f"...").

And review code of MASK to SEGS node in impact pack.
I am doing that already inside there.

https://github.com/ltdrdata/ComfyUI-Impact-Pack/blob/737fb06786cc3b2539b70b4415fb3007804fd387/impact_core.py#L557

[poisenbery]

just use print(f"...").

And review code of MASK to SEGS node in impact pack. I am doing that already inside there.

https://github.com/ltdrdata/ComfyUI-Impact-Pack/blob/737fb06786cc3b2539b70b4415fb3007804fd387/impact_core.py#L557

Sorry, I actually just started using this today and there were a lot of nodes that I haven't seen yet.
So can I use one of your nodes to accomplish what I'm trying to do?

[ltdrdata]

you can use freely. :)

[dtxn]

yeah right now it samples the whole image. an easy way to sample only the masked area for local img2img is not implemented yet.

Hi. Maybe there is approximate date when it will be implemented ? I'm thinking to try ComfyUI, but first of all I try to search for functions I use most. I use a lot inpainting with "masked only" with large size images and also I use function in Auto1111 txt2img "batch size". Is there equivalent to "batch size" (not "batch count", as "batch size" 2x faster) in ComfyUI ?

[ltdrdata]

1. Inpainting for the cropped area corresponding to "masked only" is already available in various custom nodes. For example, in the Impact Pack, there is a feature that cuts out a specific masked area based on the crop_factor and inpaints it in the form of a "detailer."

2. The equivalent of "batch size" can be configured in different ways depending on the task. For T2I, you can set the batch_size through the Empty Latent Image, while for I2I, you can use the Repeat Latent Batch to expand the same latent to a batch size specified by amount.

[ansonkao]

Related issue in #1186