Understanding the sampling process and manipulating it from inside a script

[ghost]

Just by looking at code it is almost impossible for me to figure out what is going on. I spent hours trying to manipulate the inpainting mask just to give up in the end because I dont know what the code is doing. I know it is inverting something, and then it gets very cryptic with syntax I havent seen as a beginner. I love to make progress in my python coding skills but I am at a standstill here right now.

Scripting and manipulating parameters from iteration to iteration is no problem for me.

I would really like to manipulate what the Sampler is doing on a step to step basis from inside a script. Is there any ressource that can teach me what it is really doing every step? I know the basics (going from noise to image via embeddings and weights) but thats about it. And even if I would know it in detail, I probably wouldnt be able to apply my knowledge to the python code thats already there.

I code a lot since I started learning python but this is like a wall for me that I cant break. It is too complex. Even throwing some code lines and module (.py) at me might help greatly in me understanding what is going on.

[grexzen]

I would check out clip retrieval.

https://github.com/rom1504/clip-retrieval

Rom has made so many of the underlining tools that made this possible is not even funny, including openclip that now is the top clip model out there (H/14).

[ghost]

Thanks, wow this is a lot to take in. I have seen this before, but I quickly closed it because of the overwhelming complexity. I should invest some time in this.

I found this when I was searching for a solution to iterate images and ask CLIP for a score. For example I would give clip the word "dog" and it gives me a score for every image how much of "dog" is in there.

I failed because it is so complex. I would really like to do so much more stuff

[Dement242]

Maybe this will be of help:

Lesson 10: Deep Learning Foundations to Stable Diffusion, 2022
Stable diffusion pipeline code walkthrough
https://www.youtube.com/watch?v=6StU6UtZEbU&t=2033s

[ghost]

From first glance I think this is EXACTLY what I need. Thank you