Batch hires fix?

[jredwards]

The Extras tab in the UI has an option for batch upscaling, but the upscaling that it does here is substantially different from what happens with a hires fix in the txt2img tab. I believe the difference is that it doesn't do any denoising, but someone may correct me if there are other differences.

In my opinion, the upscaled images generated this way are not as good as images generated via hires fix. As a result, I tend to load an image into the png info tab, send it to the txt2img tab with the proper seed, enable hires fix, and regenerate it entirely. If I want to do this to many images, it's time consuming.

I'd like to know if there's a way to do a hires fix on an image after it has been generated. Especially in batch. This seems like an option that could be added to the batch upscaling under Extras.

I'm hoping that I've completely missed something and someone can tell me that there's already an extension for this, but I haven't found one.

[missionfloyd]

This can be automated with the API.

import requests
import base64
import glob
import os

url = "http://127.0.0.1:7860"

# Find files
files = glob.glob('input/*.png')
os.makedirs("output", exist_ok=True)

for file in files:
    # Get PNG Info
    with open(file, "rb") as f:
        img = base64.b64encode(f.read()).decode('utf-8')

    response = requests.post(url=f'{url}/sdapi/v1/png-info', json={"image": img})
    params = response.json()["parameters"]

    # Generate image
    payload = {
        "prompt": params["Prompt"],
        "negative_prompt": params["Negative prompt"],
        "styles": params.get("Styles array", []),
        "seed": params["Seed"],
        "steps": params["Steps"],
        "cfg_scale": params["CFG scale"],
        "sampler_name": params["Sampler"],
        "width": params["Size-1"],
        "height": params["Size-2"],
        "enable_hr": True,
        "hr_scale": 2.0,            # Upscale by
        "hr_upscaler": "Latent",    # Upscaler
        "hr_second_pass_steps": 0,  # Hires steps
        "denoising_strength": 0.7,  # Denoising strength
    }

    response = requests.post(url=f'{url}/sdapi/v1/txt2img', json=payload)
    r = response.json()

    # Save image
    with open(os.path.join("output", os.path.basename(file)), 'wb') as f:
        f.write(base64.b64decode(r['images'][0]))

Takes PNGs from input folder, gets pnginfo, and sends it to txt2img. Saves results to output folder.

[w-e-w]

Of course I saw it. Just try it and you will see it doesn't work

the thing is I see it being opposite I see it's working

now first you got to realize I don't use ADetailer myself
so there may be something really obvious that I'm missing

what I did was (and this is what I did before my last comment and I also test again just now)
once I installed ADetailer and generally image in the UI
I copied the API payload json generated by API payload display and literally copy and paste it into my API test cleint which takes the json input as payload without any modification and generate an image

I then compare the result of it from the UI and the API and fund them to be identical
which shows that the API works

so from what I can see there is no difference instructing webui using UI or API

from information available to me you either are posting only parts of your payload or your payload is incomplete which was suggest that you're missing something

also that's your markdown post is broken doesn't help

since from my testing that if you just use the exact payload it will generate exact same output I only conclusion is that you made some user error and your modifications

---

now there is an extra possibility which if it's the case then you have to read the documentation of ADetailer
which is that if ADetailer implemented special options that are only accessible via API it is possible you need to enable those options to get it working the way you want it to
if they do have these type options this means that you will not be able to follow the general API usage which is what I've been basing my responses on
you would need to read ADetailer docs

---

from what I can guess without going into ADetailer codebase is that if you specify a certain model that it will be in infotex
whether or not the model is actually used is controlled by the threshold

---

leaving a particular parameter blank generally means you're using the default
but the thing is you need to watch out is that the default of the API is not necessary the same as the default of the UI
basically means that you don't want to leave certain parameters empty unless you wish to leave it to whatever the dev decides to be the default

[sdobserver]

I copied the API payload json generated by API payload

i know nothing about Py coding and etc. May you post the whole code?

[w-e-w]

i know nothing about Py coding and etc. May you post the whole code?

well I kind of already did
https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/API#example-script
this is my code

except that I swap out the payload from a dict in python to reading from json file

import json

# all my other stuff

if __name__ == '__main__':
    with open('api test txt2img payload.json', 'r', encoding='utf-8') as file:
        payload = json.load(file)
    call_txt2img_api(**payload)

also it's an API
you don't have to use python if you don't like python
use whatever language you like

[sdobserver]

i meant whole code of working API with adetailer. I'm already starting to think you're kidding me. It makes me depressed.

you don't have to use python if you don't like python use whatever language you like

I'm just a regular auto1111 user. I'm not a programmer.

[w-e-w]

I'm already starting to think you're kidding me. It makes me depressed.

"yes definitely I have nothing better to do than to trolling people, definitely not a waste of my time"

when someone is asking about how to program with the API,
we would generally expect whoever is asking has a certain level of competence, because API is Application Programming Interface,
when working with an API one is expected to write their own client / payload / calls / workflows
and to troubleshoot whatever issues that may come out of it

often in these type of conversations we won't provide actual code for good reason
the code is expected to be implemented by the user as the use case would be different not to mention it may be implemented in different languages on different platforms
also they're possible issues with set a configuration so the exact code including payload provided may not work for someone else
not to mention if there's an update to component that changes a critical aspect things might break so the information is expected to be irrelevant

so providing code is waste of time in this type of conversations

that being said if you really want to see the full code here is it
and it is as I described aboce, it is just my code that is linked on the wiki with minor modifications to the sorce of the payload. from a hardcoded dict to read from json file
https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/API#example-script

there's still no guarantee that this code will work for you because they might be some differences between your configuration set up extensions with mine setup
and the payload is literally just from api payload display without any modifications

the output from the API and the output from the UI is identical (when the same seed is used)

in here as the seat is set to random

Full Code

txt2img_api_payload_tester.py

from datetime import datetime
import urllib.request
import base64
import json
import time
import os

webui_server_url = 'http://127.0.0.1:7860'

out_dir = 'api_out'
out_dir_t2i = os.path.join(out_dir, 'txt2img')
out_dir_i2i = os.path.join(out_dir, 'img2img')
os.makedirs(out_dir_t2i, exist_ok=True)
os.makedirs(out_dir_i2i, exist_ok=True)


def timestamp():
    return datetime.fromtimestamp(time.time()).strftime("%Y%m%d-%H%M%S")


def encode_file_to_base64(path):
    with open(path, 'rb') as file:
        return base64.b64encode(file.read()).decode('utf-8')


def decode_and_save_base64(base64_str, save_path):
    with open(save_path, "wb") as file:
        file.write(base64.b64decode(base64_str))


def call_api(api_endpoint, **payload):
    data = json.dumps(payload).encode('utf-8')
    request = urllib.request.Request(
        f'{webui_server_url}/{api_endpoint}',
        headers={'Content-Type': 'application/json'},
        data=data,
    )
    response = urllib.request.urlopen(request)
    return json.loads(response.read().decode('utf-8'))


def call_txt2img_api(**payload):
    response = call_api('sdapi/v1/txt2img', **payload)
    for index, image in enumerate(response.get('images')):
        save_path = os.path.join(out_dir_t2i, f'txt2img-{timestamp()}-{index}.png')
        decode_and_save_base64(image, save_path)


def call_img2img_api(**payload):
    response = call_api('sdapi/v1/img2img', **payload)
    for index, image in enumerate(response.get('images')):
        save_path = os.path.join(out_dir_i2i, f'img2img-{timestamp()}-{index}.png')
        decode_and_save_base64(image, save_path)


if __name__ == '__main__':

    import json
    if __name__ == '__main__':
        with open('payload.json', 'r', encoding='utf-8') as file:
            payload = json.load(file)
        call_txt2img_api(**payload)

payload.json

{
    "alwayson_scripts": {
        "ADetailer": {
            "args": [
                true,
                false,
                {
                    "ad_cfg_scale": 7,
                    "ad_checkpoint": "Use same checkpoint",
                    "ad_clip_skip": 1,
                    "ad_confidence": 0.3,
                    "ad_controlnet_guidance_end": 1,
                    "ad_controlnet_guidance_start": 0,
                    "ad_controlnet_model": "None",
                    "ad_controlnet_module": "None",
                    "ad_controlnet_weight": 1,
                    "ad_denoising_strength": 0.4,
                    "ad_dilate_erode": 4,
                    "ad_inpaint_height": 512,
                    "ad_inpaint_only_masked": true,
                    "ad_inpaint_only_masked_padding": 32,
                    "ad_inpaint_width": 512,
                    "ad_mask_blur": 4,
                    "ad_mask_k_largest": 0,
                    "ad_mask_max_ratio": 1,
                    "ad_mask_merge_invert": "None",
                    "ad_mask_min_ratio": 0,
                    "ad_model": "face_yolov8n.pt",
                    "ad_negative_prompt": "",
                    "ad_noise_multiplier": 1,
                    "ad_prompt": "",
                    "ad_restore_face": false,
                    "ad_sampler": "DPM++ 2M Karras",
                    "ad_steps": 28,
                    "ad_use_cfg_scale": false,
                    "ad_use_checkpoint": false,
                    "ad_use_clip_skip": false,
                    "ad_use_inpaint_width_height": false,
                    "ad_use_noise_multiplier": false,
                    "ad_use_sampler": false,
                    "ad_use_steps": false,
                    "ad_use_vae": false,
                    "ad_vae": "Use same VAE",
                    "ad_x_offset": 0,
                    "ad_y_offset": 0,
                    "is_api": []
                },
                {
                    "ad_cfg_scale": 7,
                    "ad_checkpoint": "Use same checkpoint",
                    "ad_clip_skip": 1,
                    "ad_confidence": 0.3,
                    "ad_controlnet_guidance_end": 1,
                    "ad_controlnet_guidance_start": 0,
                    "ad_controlnet_model": "None",
                    "ad_controlnet_module": "None",
                    "ad_controlnet_weight": 1,
                    "ad_denoising_strength": 0.4,
                    "ad_dilate_erode": 4,
                    "ad_inpaint_height": 512,
                    "ad_inpaint_only_masked": true,
                    "ad_inpaint_only_masked_padding": 32,
                    "ad_inpaint_width": 512,
                    "ad_mask_blur": 4,
                    "ad_mask_k_largest": 1,
                    "ad_mask_max_ratio": 1,
                    "ad_mask_merge_invert": "None",
                    "ad_mask_min_ratio": 0,
                    "ad_model": "None",
                    "ad_negative_prompt": "",
                    "ad_noise_multiplier": 1,
                    "ad_prompt": "",
                    "ad_restore_face": false,
                    "ad_sampler": "DPM++ 2M Karras",
                    "ad_steps": 28,
                    "ad_use_cfg_scale": false,
                    "ad_use_checkpoint": false,
                    "ad_use_clip_skip": false,
                    "ad_use_inpaint_width_height": false,
                    "ad_use_noise_multiplier": false,
                    "ad_use_sampler": false,
                    "ad_use_steps": false,
                    "ad_use_vae": false,
                    "ad_vae": "Use same VAE",
                    "ad_x_offset": 0,
                    "ad_y_offset": 0,
                    "is_api": []
                }
            ]
        },
        "API payload": {
            "args": []
        },
        "Comments": {
            "args": []
        },
        "Extra options": {
            "args": [
                0,
                0,
                false
            ]
        },
        "Hypertile": {
            "args": []
        },
        "Refiner": {
            "args": [
                false,
                "",
                0.8
            ]
        },
        "Seed": {
            "args": [
                914560564,
                false,
                -1,
                0,
                0,
                0
            ]
        }
    },
    "batch_size": 1,
    "cfg_scale": 7,
    "comments": {},
    "denoising_strength": 0.5,
    "disable_extra_networks": false,
    "do_not_save_grid": false,
    "do_not_save_samples": false,
    "enable_hr": false,
    "height": 1024,
    "hr_negative_prompt": "",
    "hr_prompt": "",
    "hr_resize_x": 0,
    "hr_resize_y": 0,
    "hr_scale": 2,
    "hr_second_pass_steps": 0,
    "hr_upscaler": "R-ESRGAN 4x+ Anime6B",
    "n_iter": 1,
    "negative_prompt": "",
    "override_settings": {
        "face_restoration": false,
        "token_merging_ratio": 0,
        "token_merging_ratio_hr": 0
    },
    "override_settings_restore_afterwards": true,
    "prompt": "1girl",
    "restore_faces": false,
    "s_churn": 0,
    "s_min_uncond": 0,
    "s_noise": 1,
    "s_tmax": null,
    "s_tmin": 0,
    "sampler_name": "DPM++ 2M Karras",
    "script_args": [],
    "script_name": null,
    "seed": 914560564,
    "seed_enable_extras": true,
    "seed_resize_from_h": -1,
    "seed_resize_from_w": -1,
    "steps": 20,
    "styles": [],
    "subseed": -1,
    "subseed_strength": 0,
    "tiling": false,
    "width": 1024
}

[w-e-w]

have you tried using the built-in batch mode?

[jredwards]

Unless I'm missing something, this doesn't perform the hires pass. Resizing is not the same.

[w-e-w]

hires fix is just
txt2img -> upscale (resize) -> img2img

you can choose which upscaler that will be used when doing img2img in setting

you can also add the setting to the main UI using https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/User-Interface-Customizations#additional-options-for-txt2img--img2img

using these batch opts makes webui reads the infotext like prompt seed or model from the image

so as far as I can tell this "Batch" hires fix

[missionfloyd]

If I'm reading it correctly, doesn't hires fix operate on the latent without first converting it to an image and back? It does seem to produce different results than img2img.

Original image.

Regenerated with hires fix.

Img2img with same settings.

[w-e-w]

TL;DR
img2img = hires fix with none-latent upscaller

---

doesn't hires fix operate on the latent

not exactly, it depends on the type of upscaler you're using
there are two types of upscalers atent and image upscalers
depends on which type of upscaler you're using it does different things

stable-diffusion-webui/modules/processing.py

Lines 1328 to 1363 in b7f45e6

	if self.latent_scale_mode is not None:
	for i in range(samples.shape[0]):
	save_intermediate(samples, i)
	samples = torch.nn.functional.interpolate(samples, size=(target_height // opt_f, target_width // opt_f), mode=self.latent_scale_mode["mode"], antialias=self.latent_scale_mode["antialias"])
	# Avoid making the inpainting conditioning unless necessary as
	# this does need some extra compute to decode / encode the image again.
	if getattr(self, "inpainting_mask_weight", shared.opts.inpainting_mask_weight) < 1.0:
	image_conditioning = self.img2img_image_conditioning(decode_first_stage(self.sd_model, samples), samples)
	else:
	image_conditioning = self.txt2img_image_conditioning(samples)
	else:
	lowres_samples = torch.clamp((decoded_samples + 1.0) / 2.0, min=0.0, max=1.0)
	batch_images = []
	for i, x_sample in enumerate(lowres_samples):
	x_sample = 255. * np.moveaxis(x_sample.cpu().numpy(), 0, 2)
	x_sample = x_sample.astype(np.uint8)
	image = Image.fromarray(x_sample)
	save_intermediate(image, i)
	image = images.resize_image(0, image, target_width, target_height, upscaler_name=self.hr_upscaler)
	image = np.array(image).astype(np.float32) / 255.0
	image = np.moveaxis(image, 2, 0)
	batch_images.append(image)
	decoded_samples = torch.from_numpy(np.array(batch_images))
	decoded_samples = decoded_samples.to(shared.device, dtype=devices.dtype_vae)
	if opts.sd_vae_encode_method != 'Full':
	self.extra_generation_params['VAE Encoder'] = opts.sd_vae_encode_method
	samples = images_tensor_to_samples(decoded_samples, approximation_indexes.get(opts.sd_vae_encode_method))
	image_conditioning = self.img2img_image_conditioning(decoded_samples, samples)

---

and if your processing the Hr pass in different jobs using the new HR button (dev branch)
if you are using a latent upscalers as it doesn't keep latent between jobs it first have to encode the image back to latent space
since they might be some accuracy issues with VAE this might result in some differences in the latent space and so the output might slightly differ

stable-diffusion-webui/modules/processing.py

Lines 1328 to 1363 in b7f45e6

	if self.latent_scale_mode is not None:
	for i in range(samples.shape[0]):
	save_intermediate(samples, i)
	samples = torch.nn.functional.interpolate(samples, size=(target_height // opt_f, target_width // opt_f), mode=self.latent_scale_mode["mode"], antialias=self.latent_scale_mode["antialias"])
	# Avoid making the inpainting conditioning unless necessary as
	# this does need some extra compute to decode / encode the image again.
	if getattr(self, "inpainting_mask_weight", shared.opts.inpainting_mask_weight) < 1.0:
	image_conditioning = self.img2img_image_conditioning(decode_first_stage(self.sd_model, samples), samples)
	else:
	image_conditioning = self.txt2img_image_conditioning(samples)
	else:
	lowres_samples = torch.clamp((decoded_samples + 1.0) / 2.0, min=0.0, max=1.0)
	batch_images = []
	for i, x_sample in enumerate(lowres_samples):
	x_sample = 255. * np.moveaxis(x_sample.cpu().numpy(), 0, 2)
	x_sample = x_sample.astype(np.uint8)
	image = Image.fromarray(x_sample)
	save_intermediate(image, i)
	image = images.resize_image(0, image, target_width, target_height, upscaler_name=self.hr_upscaler)
	image = np.array(image).astype(np.float32) / 255.0
	image = np.moveaxis(image, 2, 0)
	batch_images.append(image)
	decoded_samples = torch.from_numpy(np.array(batch_images))
	decoded_samples = decoded_samples.to(shared.device, dtype=devices.dtype_vae)
	if opts.sd_vae_encode_method != 'Full':
	self.extra_generation_params['VAE Encoder'] = opts.sd_vae_encode_method
	samples = images_tensor_to_samples(decoded_samples, approximation_indexes.get(opts.sd_vae_encode_method))
	image_conditioning = self.img2img_image_conditioning(decoded_samples, samples)

---

there is limitation to this method of batch hires fix
currently I believe there's no option to use latent upscaler with img2img