Using the new Magic Prompt Batch Size

[Hugo-Matias]

Perhaps I'm misusing this feature or something isn't quite functional on my installation but I don't think this is working as I understand it would.

If for instance I set both generation parameters of Batch Size and Batch Count to 1 and set Magic Prompt Batch Size to 8 shouldn't that generate 8 prompts for a single image? Does enabling I'm Feeling Lucky as well interfere somehow in the output? I've set my WebUI with the mentioned settings and the csv output is a single generated prompt for a single image.

I'm particularly interested in consuming the script via the api's alwayson_scripts arg, one small caveat though, is that images aren't saved after inference as with using the webui, we must handle that using the returned object containing the relevant info and base64 image data, however, the object does contain the all_prompts / all_negative_prompts properties populated after initializing the prompt_writer here and I'm assuming that's the information written into the csv right? If so we should be able to replicate the default script's behavior.

This means that, even if we have the setting enabled for saving to csv, this callback is never triggered because I believe the api doesn't call that at the moment. Either way, as I said previously, the returned object info and the saved csv seem to be in parity and the problem must elsewhere. If it helps, I can also provide an example payload and response, as well as a regular csv and the WebUI settings for a regular generation.

EDIT: Ok I think I understand it now, I think that the Magic Prompt Batch Size is relevant if we have more than 1 generation, for instance, if we set a Batch Size of 8 and a Magic Prompt Batch Size of 4 it will generate 8 prompts in 2 batches of 4 but we still need to set the generation count using the Batch Size/Batch Count parameters.

Unfortunately, after a quick test I don't think that the returned object from the api is populated properly, doing a batch of 4 images and disabling image generation, the all_prompts list contains only 1 string, presumably the one for the single generated image. It's odd because all_negative_prompts (disabled Magic Prompt on negative prompts as well), all_seeds and all_subseeds contain 4 elements, it's just the promts that are returning a single one.

[adieyal]

I think that I should move batch_size into the settings since its not something you would typically change for every run. What batch size means in this context, is how many prompts to create in a single batch. The default is 1. Increasing it can make a big difference in how quickly your magic prompts are created, but honestly, that time is dwarfed by the time it takes to generate the images themselves. It's probably only useful if you want to generate prompts without images. This code should hopefully make it clear:

# benchmark.py
from datetime import datetime
from dynamicprompts.generators import RandomPromptGenerator
from dynamicprompts.generators.magicprompt import MagicPromptGenerator

device = 0

generator = MagicPromptGenerator(RandomPromptGenerator(), device=device)
t1 = datetime.now()
prompts = generator.generate("A {red|blue|green} {car|boat|plane|truck}", 32)
t2 = datetime.now()
print(f"Time taken with batch_size=1: {t2-t1}")

generator = MagicPromptGenerator(RandomPromptGenerator(), batch_size=32, device=device)
t1 = datetime.now()
prompts = generator.generate("A {red|blue|green} {car|boat|plane|truck}", 32)
t2 = datetime.now()
print(f"Time taken with batch_size=32: {t2-t1}")
print(t2-t1)

python benchmark.py
First load of MagicPrompt may take a while.
Time taken with batch_size=1: 0:00:10.962923
Time taken with batch_size=32: 0:00:00.583754

With a batch size of 1, it takes around 11 seconds to produce 32 prompts. With a batch size of 32, it took just over half a second.

Does enabling I'm Feeling Lucky as well interfere somehow in the output? I've set my WebUI with the mentioned settings and the csv output is a single generated prompt for a single image.

No, but in any case, I don't think it makes sense to run both Magic Prompt and I'm Feeling Lucky at the same time.

I'm particularly interested in consuming the script via the api's alwayson_scripts arg, one small caveat though, is that images aren't saved after inference as with using the webui, we must handle that using the returned object containing the relevant info and base64 image data, however, the object does contain the all_prompts / all_negative_prompts properties populated after initializing the prompt_writer here and I'm assuming that's the information written into the csv right? If so we should be able to replicate the default script's behavior.

I wasn't aware that the API lets you run the alwayson scripts separately. If so, then yes, the ultimate output of DP is all_prompts and all_negative_prompts. It doesn't do anything else. And yes, that's what the PromptWriter writes to the csv file.

EDIT: Ok I think I understand it now, I think that the Magic Prompt Batch Size is relevant if we have more than 1 generation, for instance, if we set a Batch Size of 8 and a Magic Prompt Batch Size of 4 it will generate 8 prompts in 2 batches of 4 but we still need to set the generation count using the Batch Size/Batch Count parameters.

The prompts are all generated before a single image is generated so Batch Size and Magic Prompt Batch Size don't interact at all.

Unfortunately, after a quick test I don't think that the returned object from the api is populated properly, doing a batch of 4 images and disabling image generation, the all_prompts list contains only 1 string, presumably the one for the single generated image. It's odd because all_negative_prompts (disabled Magic Prompt on negative prompts as well), all_seeds and all_subseeds contain 4 elements, it's just the promts that are returning a single one.

That's weird. What happens if you use the same settings in the ui itself (instead of the API)?

As an aside, you might be interested in using the dynamicprompts library directly rather than go through auto1111's API since I don't think you get additional value and it one more layer that could cause problems. Have a look here: https://github.com/adieyal/dynamicprompts/

[Hugo-Matias]

No, but in any case, I don't think it makes sense to run both Magic Prompt and I'm Feeling Lucky at the same time.

Ah, I see, I was under the impression that we should enable Magic Prompt for I'm Feeling Lucky to be enabled as well since they are in the same expansion panel in the ui.

The prompts are all generated before a single image is generated so Batch Size and Magic Prompt Batch Size don't interact at all.

So my initial assumption was right? We could set both batch values to 1 and set Magic Prompt batch size to the number of prompts we want to generate? That doesn't seem to happen even through the webui, I only get the number of prompts written to the csv equals to the number of batched images. This is with Don't generate images enabled.

That's weird. What happens if you use the same settings in the ui itself (instead of the API)?

The csv is written as expected. I debugged the script in VSCode and found that this is the line that's purging the extra info from the response. I'm still not sure why that's required, perhaps to comply some WebUI requirement but that seems to be the problem when enabling the Don't generate images bool. It makes sense because the prompt_writer is already initialized by that time and the csv is written using it's properties, not the final p.all_prompts that is sent in the response. Could we bypass that purge in some way and still make it work? EDIT: I commented out that line and the whole process finished successfully. I'm not sure if there is a potential for a butterfly effect elsewhere but it seems to be sending the, in this case, 4 prompts in all_prompts is fine. I will keep the line commented for now and see if it causes some issue.

Thanks for the suggestion, since we discussed here I actually started to implement pythonnet into my Blazor app, but I don't think the added complexity is worth it for this case and pythonnet can be a bit clunky to work with at times. It's a simple app to integrate a few quality of life features into my workflow, implementing the api has been quite fast for most of my needs so far.

[adieyal]

Ah, I see, I was under the impression that we should enable Magic Prompt for I'm Feeling Lucky to be enabled as well since they are in the same expansion panel in the ui.

It's of course possible to run them together but I'm Feeling Lucky will dominate. I'm not sure off-hand what the order is, but if Magic Prompt is run first, then Lexica's search engine will run a semantic search and return a prompt that is probably quite different to the original prompt generated by Magic Prompt. If Magic Prompt is run last, then it won't have many tokens available to add any modifiers since I'm Feeling Lucky would probably return a long prompt.

So my initial assumption was right? We could set both batch values to 1 and set Magic Prompt batch size to the number of prompts we want to generate? That doesn't seem to happen even through the webui, I only get the number of prompts written to the csv equals to the number of batched images. This is with Don't generate images enabled.

No, batch_size * num_batches will determine how many images will be generated. Magic Prompt batch_size simply tells the MagicPromptGenerator how many to generate at once. It's really only for efficiency. Regardless of what you set it to, you output will be the same.

The csv is written as expected. I debugged the script in VSCode and found that this is the line that's purging the extra info from the response. I'm still not sure why that's required, perhaps to comply some WebUI requirement but that seems to be the problem when enabling the Don't generate images bool.

If you use the interface, auto1111 expects there to be at least one prompt (which makes sense). If I check Don't generate images, I need to provide at least one prompt, which is why you will generate one image, even though that checkbox is checked. It would probably be better to do that outside of the main generation cycle, perhaps we can re-arrange the Wildcards Manager tab to have a section for generating prompts. It's also a good example of how using auto1111's api isn't necessarily the best way to go in this case because there might be user interface considerations that aren't applicable to the api.

In this case, it doesn't seem like a big deal if you're happy to keep the line commented out and deal with any conflicts when merging new versions.

[Hugo-Matias]

If you use the interface, auto1111 expects there to be at least one prompt (which makes sense). If I check Don't generate images, I need to provide at least one prompt, which is why you will generate one image, even though that checkbox is checked.

I understand but the issue here is with the extra prompts. Running the api call seems to only generate a single image even with that line commented out and all_prompts properly populated but the WebUI will generate all the prompts in that variable, quite odd. I'm surely overlooking some internal logic that triggers that behavior but if the api works that's fine by me. I've considered the switch to the library but using pythonnet isn't without issues either and weighting both cases, from a C# point of view, consuming the api is still simpler and less clunky to work with.

perhaps we can re-arrange the Wildcards Manager tab to have a section for generating prompts

That's kinda what i'm trying to achieve. I have a small sqlite db where I store prompt presets, among other things. If I have some default parameters set to 1 inference step and extremely low resolution, I can discard all the api response information and just keep the prompts for further manipulation. It's quite fast and properly implemented can provide a seamless experience when prompt prototyping.

[adieyal]

I'd be keen to see what you come up with when you have it running.

[Hugo-Matias]

Still early days but this is kinda what I had in mind with a prompt generator/helper. I'm yet unsure about the workflow but it's kinda hard to manipulate the DOM with Blazor without relying on JSInterop (something I wan't to keep at a minimum in the project), the prompt text field is changed via buttons and global component variables like _template for the current template.

Code isn't exactly shareable at this moment because it's catered to my particular setup on some places but here is a demo of the current implementation.

https://streamable.com/rwc2sk

On component initialization, I walk through the wildcards tree and define DirectoryInfo's and FileInfo's to make things easy. With that I can now populate the _template textfield accordingly; if it's a dir I use your globbing (nice feature!) __{dir}/*__, if it's a file I do __{dir}/{file}__. Then on the right side I have my manipulation area where i can build a prompt using the template.

For the txt2img call I use most values at default but change the following:

SD:

• BatchSize = amount_of_prompts (this value should have a slider later on, for now its hardcoded at 10)
• Steps = 1,
• Width, Height = 16, (was running into some issues with lower but it doesn't seem to matter much with a step count so low, it's just to minimize discarded info from the response's base64 image string, 32, 64, 128, etc. work fine and, perceptively. just as fast)
• Prompt = "{manipulated prompt}"

Dynamic Prompts:

• IsEnabled = true,
• NoImageGeneration = true

I still want to implement a textfield to change the wildcard file (a bit like you do in WebUI) and the button to send the prompt will most likely change into an inline menu to be able to send it into txt2img or img2img and to prompt or negative prompt fields, as well as an option to save it as a preset.