'Starting / Ending Control Step' option in ControlNet. #605

TheMoye · 2023-05-02T18:17:53Z

TheMoye
May 2, 2023

Hello everyone,

Is there a way to find certain ControlNet behaviors that are accessible through Automatic1111 options in ComfyUI?
I'm thinking of the 'Starting Control Step', 'Ending Control Step', and the three 'Control Mode (Guess Mode)' options: 'Balanced', 'My prompt is more important', and 'ControlNet is more important'.
The results obtained seem to be better than those obtained simply by changing the weight.

Thank you very much ! ✌️

Answered by comfyanonymous

May 3, 2023

Right now controlling which steps a controlnet gets applied to along with which steps a prompt gets applied to, etc... is done with KSampler Advanced.

You can see an example of how to use KSampler Advanced here:
https://comfyanonymous.github.io/ComfyUI_examples/noisy_latent_composition/

View full answer

comfyanonymous · 2023-05-03T06:11:03Z

comfyanonymous
May 3, 2023
Maintainer

Right now controlling which steps a controlnet gets applied to along with which steps a prompt gets applied to, etc... is done with KSampler Advanced.

You can see an example of how to use KSampler Advanced here:
https://comfyanonymous.github.io/ComfyUI_examples/noisy_latent_composition/

4 replies

TheMoye May 3, 2023
Author

Hi @comfyanonymous , thank you for this answer.

I followed the example you gave me, but I'm not sure I'm doing it right.
I lose a lot of details compared to Automatic1111 😯

Here is a comparison of my results between Comfy and A1111:

For both images I use:

The same model
The same VAE
The same ControlNet depth with the same image
The same sampling parameters
The same seed (from the GPU also for Comfy)
The same LoRA and Lycorys
The same prompts (without weights, I understood that A1111 and Comfy did not treat them the same way)

In Automatic1111, the ControlNet is applied with the following parameters:

Control Weight = 1.0
Starting Control Step = 0.0
Ending Control Step = 0.75
Control Mode = My prompt is more important

In Comfy, the ControlNet is applied as follows:

Strength = 1.0
Connected to a KSamplerAdvanced (start at step 0 / stop at step 17)
Connected to a new KSamplerAdvanced (start at step 17 / stop at step 25) that no longer uses the ControlNet in its inputs

The Latent that comes out of it is already very close to the final image.

Maybe this is not the right way to mimic the 0.0 to 0.75 of Automatic1111?
Maybe it just lacks the Control Mode? Is there a way to emulate it?

To continue in my investigations, I noticed that there were differences in rendering between A1111 and Comfy, even on basic prompts with the same GPU seed.
An example:

The first image comes from Automatic1111, the second from Comfy.
Even though they are extremely close, there are subtle differences.
Whereas if I use the same seed twice in A1111 and in Comfy, I would have the same two images.

I thought that the generation of noise based on the seed was slightly different between the two UIs. To check, I exported as text files the two noises obtained from the same seed for a 64x64 image.
They seem to be identical 🤷

      seeds=[3216071920]
      
      tensor([[[[-2.0973e-02, -9.5477e-01, -1.2929e+00,  1.0567e-01, -1.1612e+00, -3.2241e-01,  3.6962e-01,  1.3641e+00],
      [ 8.2025e-01,  1.0569e-01, -1.4120e+00,  1.3820e-01, -7.5404e-02, -8.0880e-02,  1.4795e+00,  1.3630e+00],
      [ 5.8766e-02, -8.7644e-01, -2.4227e-01,  1.4345e+00,  5.8446e-01, -8.8624e-01, -1.5455e-01,  2.0149e-01],
      [-6.0064e-01, -3.2008e-02,  1.4102e+00, -3.8238e-01, -1.0307e+00, -7.2646e-01,  3.7912e-01, -4.4667e-01],
      [ 7.0947e-01, -1.8725e-01, -2.6721e-01,  5.2522e-01,  7.8085e-01, -6.9582e-02,  1.7253e-01, -1.0892e-01],
      [-7.7526e-01,  7.4732e-02, -6.6871e-01, -1.3161e+00, -7.0381e-01, -1.5031e-01,  5.7754e-01, -1.5003e+00],
      [ 7.5601e-01, -1.4983e-01,  4.8849e-01, -8.0075e-01,  1.6008e+00, 1.1788e+00,  1.0147e+00, -7.8732e-01],
      [ 7.0080e-01,  1.2617e+00, -5.7344e-01,  5.3507e-01,  2.3166e-01, 3.0196e-01,  1.5931e+00,  5.3913e-01]],

     [[ 1.3080e+00, -5.7686e-02,  3.0104e+00, -4.3914e-01, -3.5036e-01, 1.8226e+00, -1.5981e+00, -1.5272e+00],
      [ 2.4509e-01,  9.4291e-03,  1.0970e-01, -6.1462e-01, -1.2813e+00, -1.0110e+00,  2.7343e-01,  1.3736e+00],
      [-3.7923e-01,  9.7695e-03, -1.2901e+00,  1.1176e+00,  1.0331e+00, -1.1651e+00,  6.0505e-01,  1.2480e+00],
      [ 1.2434e+00, -2.1891e-01, -2.2144e-01,  2.7791e-01, -8.2933e-01, -1.9455e-01,  9.9274e-01,  5.7786e-01],
      [-8.0363e-01, -2.3639e-03,  2.2417e+00, -4.8913e-01,  8.0453e-01, 1.5394e-01,  2.8283e-01, -6.6359e-01],
      [-4.3254e-01,  2.6585e-01, -4.7375e-01,  1.9378e+00, -1.4525e+00, 1.0557e+00, -1.8277e+00,  3.1335e-01],
      [ 5.4438e-01,  3.6345e-01,  5.7134e-01, -7.9616e-01, -1.7190e+00, 3.6922e-01,  5.0843e-01, -1.2864e-01],
      [-1.5426e-01, -4.4144e-02,  7.3498e-01,  3.2126e-01,  5.6918e-01, -8.0092e-01, -6.2647e-01,  1.6764e-01]],

     [[ 5.2072e-01,  1.1937e+00, -1.7208e+00, -4.1489e-02,  4.2971e-01, -9.1894e-01,  7.6229e-01, -3.6712e+00],
      [ 7.1708e-01, -1.3355e+00, -1.4969e+00,  7.4230e-01,  3.6362e-01, -9.8982e-01, -5.9379e-01, -7.5932e-02],
      [-7.5645e-01,  1.8496e+00,  2.9696e-01,  1.8725e+00, -1.2990e-01, -6.1142e-01, -1.2302e-01, -4.6452e-01],
      [-7.7145e-01, -9.0256e-01, -2.4801e-02, -1.8810e+00, -3.4829e-01, -3.8776e-01,  1.1578e+00,  2.4906e+00],
      [ 2.1190e-01,  1.2547e+00, -9.1765e-01, -4.6094e-01, -5.0987e-01, 5.1489e-01, -2.1729e+00, -5.8477e-01],
      [-2.1946e-01,  2.3897e-01, -1.5202e+00, -2.5811e-01, -1.5526e+00, 7.0831e-01,  5.6649e-01,  2.1645e+00],
      [ 1.3613e+00,  1.7665e+00,  3.4939e-01,  4.1595e-01,  4.6268e-01, 3.4609e-01, -1.4659e+00, -1.2181e-01],
      [-1.5236e+00,  5.0945e-01, -1.6540e-01,  5.9664e-01, -9.2929e-01, -8.7995e-01, -3.7354e-01,  1.1070e+00]],

     [[ 1.5772e+00, -2.2197e-01, -3.6867e-01, -1.2687e-01, -1.1753e-01, 5.4921e-01, -2.1954e-01,  7.6966e-01],
      [-3.5975e+00,  4.4909e-01, -2.9015e-01,  1.5555e+00, -1.9549e-01, 7.3934e-01,  6.8679e-01,  7.0042e-01],
      [ 6.7567e-01, -3.4986e-01,  4.0205e-01, -2.8944e-01,  1.2894e-01, 1.6419e+00,  6.0937e-02, -1.7255e+00],
      [ 8.7944e-02,  1.2367e+00,  1.1951e+00, -9.6385e-01,  2.6750e-01, -9.5480e-01, -1.0992e+00,  1.7059e-02],
      [ 7.5747e-01,  2.9670e-01,  3.9995e-01,  1.6000e-01,  3.6096e-02, -1.3798e+00, -2.3600e-01, -2.1575e-01],
      [ 9.7623e-01,  7.5056e-01, -4.2539e-01,  7.9969e-01, -4.4800e-01, 1.4304e+00, -8.6471e-01, -4.5115e-01],
      [-8.6758e-01, -8.5689e-01, -5.6123e-02,  2.0360e+00, -1.1832e-01, 3.2289e-01, -1.0777e+00,  2.5039e+00],
      [ 8.0382e-01, -1.2733e+00, -1.4468e+00, -3.8532e-01, -6.8383e-01, -8.3728e-01, -3.0767e+00,  8.8572e-01]]]], device='cuda:0')

      seed=3216071920, seed_from=GPU, device=0
      
      tensor([[[[-2.0973e-02, -9.5477e-01, -1.2929e+00,  1.0567e-01, -1.1612e+00, -3.2241e-01,  3.6962e-01,  1.3641e+00],
      [ 8.2025e-01,  1.0569e-01, -1.4120e+00,  1.3820e-01, -7.5404e-02, -8.0880e-02,  1.4795e+00,  1.3630e+00],
      [ 5.8766e-02, -8.7644e-01, -2.4227e-01,  1.4345e+00,  5.8446e-01, -8.8624e-01, -1.5455e-01,  2.0149e-01],
      [-6.0064e-01, -3.2008e-02,  1.4102e+00, -3.8238e-01, -1.0307e+00, -7.2646e-01,  3.7912e-01, -4.4667e-01],
      [ 7.0947e-01, -1.8725e-01, -2.6721e-01,  5.2522e-01,  7.8085e-01, -6.9582e-02,  1.7253e-01, -1.0892e-01],
      [-7.7526e-01,  7.4732e-02, -6.6871e-01, -1.3161e+00, -7.0381e-01, -1.5031e-01,  5.7754e-01, -1.5003e+00],
      [ 7.5601e-01, -1.4983e-01,  4.8849e-01, -8.0075e-01,  1.6008e+00, 1.1788e+00,  1.0147e+00, -7.8732e-01],
      [ 7.0080e-01,  1.2617e+00, -5.7344e-01,  5.3507e-01,  2.3166e-01, 3.0196e-01,  1.5931e+00,  5.3913e-01]],

     [[ 1.3080e+00, -5.7686e-02,  3.0104e+00, -4.3914e-01, -3.5036e-01, 1.8226e+00, -1.5981e+00, -1.5272e+00],
      [ 2.4509e-01,  9.4291e-03,  1.0970e-01, -6.1462e-01, -1.2813e+00, -1.0110e+00,  2.7343e-01,  1.3736e+00],
      [-3.7923e-01,  9.7695e-03, -1.2901e+00,  1.1176e+00,  1.0331e+00, -1.1651e+00,  6.0505e-01,  1.2480e+00],
      [ 1.2434e+00, -2.1891e-01, -2.2144e-01,  2.7791e-01, -8.2933e-01, -1.9455e-01,  9.9274e-01,  5.7786e-01],
      [-8.0363e-01, -2.3639e-03,  2.2417e+00, -4.8913e-01,  8.0453e-01, 1.5394e-01,  2.8283e-01, -6.6359e-01],
      [-4.3254e-01,  2.6585e-01, -4.7375e-01,  1.9378e+00, -1.4525e+00, 1.0557e+00, -1.8277e+00,  3.1335e-01],
      [ 5.4438e-01,  3.6345e-01,  5.7134e-01, -7.9616e-01, -1.7190e+00, 3.6922e-01,  5.0843e-01, -1.2864e-01],
      [-1.5426e-01, -4.4144e-02,  7.3498e-01,  3.2126e-01,  5.6918e-01, -8.0092e-01, -6.2647e-01,  1.6764e-01]],

     [[ 5.2072e-01,  1.1937e+00, -1.7208e+00, -4.1489e-02,  4.2971e-01, -9.1894e-01,  7.6229e-01, -3.6712e+00],
      [ 7.1708e-01, -1.3355e+00, -1.4969e+00,  7.4230e-01,  3.6362e-01, -9.8982e-01, -5.9379e-01, -7.5932e-02],
      [-7.5645e-01,  1.8496e+00,  2.9696e-01,  1.8725e+00, -1.2990e-01, -6.1142e-01, -1.2302e-01, -4.6452e-01],
      [-7.7145e-01, -9.0256e-01, -2.4801e-02, -1.8810e+00, -3.4829e-01, -3.8776e-01,  1.1578e+00,  2.4906e+00],
      [ 2.1190e-01,  1.2547e+00, -9.1765e-01, -4.6094e-01, -5.0987e-01, 5.1489e-01, -2.1729e+00, -5.8477e-01],
      [-2.1946e-01,  2.3897e-01, -1.5202e+00, -2.5811e-01, -1.5526e+00, 7.0831e-01,  5.6649e-01,  2.1645e+00],
      [ 1.3613e+00,  1.7665e+00,  3.4939e-01,  4.1595e-01,  4.6268e-01, 3.4609e-01, -1.4659e+00, -1.2181e-01],
      [-1.5236e+00,  5.0945e-01, -1.6540e-01,  5.9664e-01, -9.2929e-01, -8.7995e-01, -3.7354e-01,  1.1070e+00]],

     [[ 1.5772e+00, -2.2197e-01, -3.6867e-01, -1.2687e-01, -1.1753e-01, 5.4921e-01, -2.1954e-01,  7.6966e-01],
      [-3.5975e+00,  4.4909e-01, -2.9015e-01,  1.5555e+00, -1.9549e-01, 7.3934e-01,  6.8679e-01,  7.0042e-01],
      [ 6.7567e-01, -3.4986e-01,  4.0205e-01, -2.8944e-01,  1.2894e-01, 1.6419e+00,  6.0937e-02, -1.7255e+00],
      [ 8.7944e-02,  1.2367e+00,  1.1951e+00, -9.6385e-01,  2.6750e-01, -9.5480e-01, -1.0992e+00,  1.7059e-02],
      [ 7.5747e-01,  2.9670e-01,  3.9995e-01,  1.6000e-01,  3.6096e-02, -1.3798e+00, -2.3600e-01, -2.1575e-01],
      [ 9.7623e-01,  7.5056e-01, -4.2539e-01,  7.9969e-01, -4.4800e-01, 1.4304e+00, -8.6471e-01, -4.5115e-01],
      [-8.6758e-01, -8.5689e-01, -5.6123e-02,  2.0360e+00, -1.1832e-01, 3.2289e-01, -1.0777e+00,  2.5039e+00],
      [ 8.0382e-01, -1.2733e+00, -1.4468e+00, -3.8532e-01, -6.8383e-01, -8.3728e-01, -3.0767e+00,  8.8572e-01]]]], device='cuda:0')

I use A1111 and Comfy in the same virtual environment.
What can cause these differences?

comfyanonymous May 3, 2023
Maintainer

For comparison try using euler. Differences are normal because the code is different and some samplers amplify those differences more than others.

TheMoye May 3, 2023
Author

It seems that it is the same thing with Euler or Euler a, small variations persist.

Differences are normal because the code is different and some samplers amplify those differences more than others.

Very well, I guess I have to stop trying to reproduce the image that I liked in A1111. 🤪
Thank you!

OffixialK Jun 30, 2023

syestem stop

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

'Starting / Ending Control Step' option in ControlNet. #605

Uh oh!

{{title}}

Uh oh!

Replies: 1 comment 4 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

'Starting / Ending Control Step' option in ControlNet. #605

Uh oh!

TheMoye May 2, 2023

Replies: 1 comment · 4 replies

Uh oh!

comfyanonymous May 3, 2023 Maintainer

Uh oh!

TheMoye May 3, 2023 Author

Uh oh!

comfyanonymous May 3, 2023 Maintainer

Uh oh!

TheMoye May 3, 2023 Author

Uh oh!

OffixialK Jun 30, 2023

TheMoye
May 2, 2023

Replies: 1 comment 4 replies

comfyanonymous
May 3, 2023
Maintainer

TheMoye May 3, 2023
Author

comfyanonymous May 3, 2023
Maintainer

TheMoye May 3, 2023
Author