Add support for Res-Adapter (just released)

[CCpt5]

https://github.com/bytedance/res-adapter?tab=readme-ov-file

Would love to see this implemented into Forge/Controlnet.

---

ResAdapter: Domain Consistent Resolution Adapter for Diffusion Models
Jiaxiang Cheng, Pan Xie*, Xin Xia, Jiashi Li, Jie Wu, Yuxi Ren, Huixia Li, Xuefeng Xiao, Min Zheng, Lean Fu (*Corresponding author)

AutoML, ByteDance Inc.

We propose ResAdapter, a plug-and-play resolution adapter for enabling diffusion models of arbitrary style domains to generate resolution-free images: no additional training, no additional inference and no style transfer.

[CCpt5]

The developers of "res-adapter" indicated they are working on integration w/ A1111 (Forge I assume) and Comfy. On their repo today:

bytedance/res-adapter#1

[leomaxwell973]

so you could say... change resolution to 256x384 or 256x256 even and not have to hunt around for a good seed for an hour? :D
down for that. using those resolutions is the only way I can stand AniDiff SD1.5. ((Turbo)SDXL MMs are few, far, and fall short, so far, imo, sadly)

[CCpt5]

Update:

yesterday (March 12) they released their v1.5 model that covers up to 1024x1024. The model weight is available on HF, but still not sure how exactly to use the LoRA along w/ the "normalization" file:

[2024/03/12] We release the resadapter-sdv15 that supports 128~1024 resolution.

https://huggingface.co/jiaxiangc/res-adapter/tree/main/sd1.5

[jiaxiangc]

@CCpt5 Hi, i am learning how to coding webui extension. I only need to load lora and normalization for sd_model.model.diffusion_model. But i do not know how to realize this in webui. Do you have any suggestions?

[CCpt5]

Edit: Nevermind, just realized YOU are the developer of Res-adapter, hahah. Thanks for your work! I'm sorry I don't have information for you, but I hope in time there is a Forge/WebUI Res-Adapter extension. If you're looking for someone to talk to w/ knowledge of Forge/A1111 in general (Re: how to load models etc), I have high regard for https://github.com/wkpark who coded an expert level model merging tool (Model MIxer), and is very activate w/ upddates/discussions on their github repos.

Original reply where I thought you were an amateur user trying to figure Res-Adapter out:

---

Sorry, I don't. I'm not a developer myself. The best I can point you to is a discussion on the ComfyUI node implementation which an amateur developer coded out on their own before the official group was able to share this own.

See this discussion where the user "Bleeping" talks about figured this out on their own for Comfy:

bytedance/res-adapter#14

Thei ComfyUI node Repo:
https://github.com/blepping/ComfyUI-ApplyResAdapterUnet

if you manage to code an extension for A1111/Fogre that loads the Res-Adapter models and enables it's functionality please come back and @ me! I'd love a way to do it in the Webui.

Best of lucj!!

[CCpt5]

I commented on an old thread on @wkpark's Model Mixer repo to see if they can provide Webui model loading assistance for you. They are extremely talented/knowledgeable and friendly. I'm sure if you're seeking this assistance (and I didn't misread) they could likely help.

wkpark/sd-webui-model-mixer#120 (comment)

[jiaxiangc]

haha, thanks for your help.