low vram

[coddz]

from yestoday, the ending of forge startup show low vram, 'You just set Forge to use 100% GPU memory (23539.00 MB) to load model weights.'

warning information:
To create a public link, set share=True in launch().
IIB Database file has been successfully backed up to the backup folder.
Startup time: 16.6s (prepare environment: 2.2s, import torch: 4.0s, other imports: 0.3s, load scripts: 3.7s, create ui: 2.2s, gradio launch: 3.3s, app_started_callback: 0.7s).
Environment vars changed: {'stream': False, 'inference_memory': 1024.0, 'pin_shared_memory': False}

[Low VRAM Warning] You just set Forge to use 100% GPU memory (23539.00 MB) to load model weights.
[Low VRAM Warning] This means you will have 0% GPU memory (0.00 MB) to do matrix computation. Computations may fallback to CPU or go Out of Memory.
[Low VRAM Warning] In many cases, image generation will be 10x slower.
[Low VRAM Warning] Make sure that you know what you are testing.

information of forge and env:
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: f2.0.1v1.10.1-previous-395-g0d8eb4c5
Commit hash: 0d8eb4c
Launching Web UI with arguments: --theme dark --ckpt-dir E:/SDWebUI/sd-webui-aki-v4.4/models/Stable-diffusion --hypernetwork-dir E:/SDWebUI/sd-webui-aki-v4.4/models/hypernetworks --embeddings-dir E:/SDWebUI/sd-webui-aki-v4.4/embeddings --vae-dir E:/SDWebUI/sd-webui-aki-v4.4/models/vae --lora-dir E:/SDWebUI/sd-webui-aki-v4.4/models/Lora
Total VRAM 24564 MB, total RAM 65349 MB
pytorch version: 2.3.1+cu121
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4090 : native

[ostap667inbox]

Lol. Set the 'GPU Weights (MB)' control to a lower value.
You have set this setting to maximum. This means that you have allocated all 24Gb for model storage and 0 for calculations. That's why you see this message about not enough VRAM.

[coddz]

'GPU Weights (MB)' control to a lower value, the message changed. thank you very much.

[CptBlood1981]

Hey, having the same issue, how do you set this? Sorry relatively new to forge...

[CptBlood1981]

lol, nvm, I was in SD mode not xl like I should have been found and fixed!

[aggregate15]

I don't get the point of this setting. on my 8GB VRAM when I set GPU Weights to their Maximum limit of 8187 MB - I get generation time of about 40 Seconds. When I set it to Default Flux Setting of 7163 MB, it tells me that enough space is reserved for calculations (and stops giving me low VRAM warnings) --- but my Generation time shoots up to 58 Secs. Can someone clarify what the benefit of setting up free space for calculations is ? For me it gives demonstrably worse generation times for the same settings/quality.

[ostap667inbox]

There can't be specific recommendations here like “set this setting to such and such a value”.
For each specific task, image size etc, you can try out different balances and choose the best one for you.
You strike a balance of assigning the use of fast VRAM for computation and for model storage. Everything more that is needed for work will be loaded into the slow system RAM. And if it is not enough, the slow swap disk will be used.
If you have slowed down image generation while allocating enough VRAM for computation, it means that data exchange was more critical for this task, for which slow access to system RAM was used.
For my 12Gb VRAM I set it to about 50% and I'm happy with the result. For your 8Gb VRAM, it might be better to set this setting to around 30% and see if that is optimal.