Getting big (> 2 MP) images from forge / flux without out of memory crash

[rfrazier888]

Hi all. My name is Ron and I'm new here. Please be nice if I say anything silly or make a mistake. I'm also only on the forum very intermittently. I have been using AI image generation for over a year, first with Stable Diffusion XL and Fooocus, now with Flux Dev and Forge. I've also posted hundreds of images on Civitai and have done a number of image generations with Flux Ultra via their website.

While it's nice to be able to post smaller images online, sometimes I want to break out of the 2 MP box, which as I understand it, is the limit for Flux Dev native. So, I've been playing around with the Hires Fix option in Forge. However, almost every time I try that, the system crashes with an out of memory error. I've come up with some observations here which may be useful to some people. Or maybe you all already know this and I'm the only one who didn't. We'll see. Your mileage may vary.

My system is a Linux Mint computer with 128 GB of RAM and 2 ea Nvidia 4060 TI GPU's with 16 GB VRAM ea. I only use the Forge GUI or the pop out web browser GUI for setup and am not doing any python scripting directly. I'm running Pinokio, and Forge on top of that. The system seems to only ever use 1 GPU, but I can live with it. I'm running the Flux Dev fp8 model.

So, I've been able to generate 2 MP images just fine such as 1448 x 1448 or 1920 x 1080. I've tried some 2x upscaling and that worked but if I tried to go bigger, it would crash with out of memory errors. I played around with the GPU Weights setting a bit, but wasn't ever able to get results. Then I noticed that when I adjust the number on the GUI, it would tell me in the terminal how much memory was left for computation. I THOUGHT that higher GPU weights was better and kept trying to nudge it up, eventually leaving no room for computation. Some of you are probably laughing, but I didn't know what I didn't know. Eventually, I tried to do just the opposite. I set the GPU weights to zero to see what would happen. Well, that worked, and I was able to break out of the 2 MP box.

In my latest experiment, I was able to start with a 1920 x 1080 (2 MP) base image, and set hires fix to 3X, which ends up being 5760 x 3240 (18 MP). It ran without failing, although very slowly. The image took about 30 minutes to generate. I'm running NVTOP to monitor the GPU's. I believe the resizer was taking about 12.9 GB of VRAM without any weights being stored in there. This size of image is big enough to be useful for printing, and I could make a 36" x 20" poster and still get about 160 DPI on the print. For most online use, it doesn't need to be that big.

Now, I wasn't actually happy with the quality of the image. The prompt involved a beautiful woman next to a grove of trees. With the latent upscaler, I'm getting block artifacts on her skin and clothes and vertical line artifacts on the trees. I'm experimenting with other upscalers, but at 30 minutes each, it's very slow. I may not do this very much, but it's nice to know I can.

So, I hope this is helpful to some. If you're trying to do big images, and you have 12 GB - 16 GB of VRAM, and you have a good amount of system RAM, and you're getting out of memory crashes, maybe reducing GPU weights way down will help. I'm glad to answer basic questions about my setup. But, even though I have an Electronics degree and have been using personal computers since they were invented pretty much, I don't have detailed knowledge of python or C programming or knowledge of what's going on under the hood here. All input is welcome. Enjoy.

Ron

[abzaloff]

You're doing everything wrong. Do not use Hires-Fix. Install the Ultimate SD Upscale and upscale in to at least 4K images.
https://github.com/Coyote-A/ultimate-upscale-for-automatic1111

[rfrazier888]

You're doing everything wrong. Do not use Hires-Fix. Install the Ultimate SD Upscale and upscale in to at least 4K images. https://github.com/Coyote-A/ultimate-upscale-for-automatic1111

Thanks for the link. 8-)

[megvadulthangya]

This can work with a potatoe PC? only 4GB Vram? High Res fix does give me out of mermory cuda error something something... no matter what setting is use...
BTW im using SDXL...

[Silvicultor]

Your VRAM is to small for a SDXL model to fit in. But it (might) be still possible to use SDXL however. Granted you have enough system RAM you can offload with the build-in NeverOOM extension. But it will be very slow even at 1024x1024.
As long as you have enough system RAM upscaling should also be possible, but with higher resolutions the generation speed will drop extremely.
And using flux (that what this thread is about) is out of the question with your GPU.

[Giribot]

i have a RTX 3050 with 4gb of vram and i use flux .....
No problem. and take the same time than ChatGPT. Lol

NeverOOM Enabled for UNet (always maximize offload)
NeverOOM Enabled for VAE (always tiled)
Skipping unconditional conditioning when CFG = 1. Negative Prompts are ignored.
[Unload] Trying to free 7817.77 MB for cuda:0 with 0 models keep loaded ... Unload model IntegratedAutoencoderKL Done.
CPU Swap Loaded (blocked method): 5225.21 MB, GPU Loaded: 145.52 MB
Moving model(s) has taken 1.04 seconds
Distilled CFG Scale: 3.5
[Unload] Trying to free 17347.11 MB for cuda:0 with 0 models keep loaded ... Unload model JointTextEncoder Done.
CPU Swap Loaded (blocked method): 11350.05 MB, GPU Loaded: 0.04 MB
Moving model(s) has taken 0.07 seconds
30%|███ | 6/20 [02:17<04:18, 18.49s/it]<03:28, 14.89s/it]

[megvadulthangya]

Man that would be a 3 to 5 bussiness day to me on my GTX750Ti i guess... once i tryed flux last year when i installed this headless thing but i dropped... Maybe need to try again couse forge is changed, enough? and i understand more....

SDXL Hyper models are working fine, around 2 min to generate a 896x1152 but with only cfg1