I want to write a prompt modifying extension and need help understanding how

[TheHorscht]

I'm trying to write an extension that modifies the prompt before it gets sent to the txt2img pipeline, kind of like how dynamic prompts works (which I took a look at but the code is way too complex for me to understand). I also tried to look at the https://github.com/AUTOMATIC1111/stable-diffusion-webui-wildcards extension but there are some issues which I don't quite understand.

First of all, it seems that in my script's process(self, p) when I modify p.all_prompts, only the first element [0] seems to get used and all the others ignored, not so in before_process_batch(self, p), where it seems to work correctly? But the wildcards extension uses process, so wouldn't that also be broken?
I tried this with batch count 2, batch size 2:

def before_process_batch(self, p, is_enabled: bool, *script_args, **kwargs):
    if is_enabled:
        p.all_prompts[0] = "girl wearing a purple dress standing in a park"
        p.all_prompts[1] = "girl wearing a red dress standing in a park"
        p.all_prompts[2] = "girl wearing a green dress standing in a park"
        p.all_prompts[3] = "girl wearing a yellow dress standing in a park"

which correctly replaces the prompts and gives me 4 images with a purple, red, green and yellow dress.
However, some time later, when using this method in a real scenario (with a function that actually modifies the prompt randomly based on p.all_seeds), I could not replicate the results anymore and modifying p.all_prompts had no effect anymore. But it DID show in the generation params in the infotext under the gallery view, while still using the original prompt for the actual image generation.

So that's where I'm really confused, what's the difference between these and which do I need to modify when? Which one gets written to the metadata? Which one gets output in the infotext and which one gets used for the actual generation?

p.prompt
p.prompts
p.prompt_for_display
p.main_prompt
p.all_prompts

Are there any more that I forgot?
I'm not even sure if the prompt I modified affected the the actual image generation, or just the text in the infotext, and if the modified prompt was written correctly into the image metadata or not. No idea where the non-reproducability issue lies. Please help, thanks :)