Flux Lora Patching crashes - add CPU for Lora patching possible?

[markdee3]

Using Flux NF4 with 4GB VRAM 16GB RAM and presumably running out of vram.
is it possible to use CPU when Lora patching instead of just GPU?

50mb and 35mb Loras work. but 18mb loras crash?
Any loras that have 304 steps during patching crash, but loras with 74 steps patching work fine.
for the lora patching i get negative GPU remaining so wondering if there is a way around this?

[Memory Management] Required Inference Memory: 1024.00 MB
[Memory Management] Estimated Remaining GPU Memory: -2820.25 MB
[Memory Management] Loaded to GPU for backward capability: 73.14 MB
[Memory Management] Loaded to CPU Swap: 3446.00 MB (blocked method)
[Memory Management] Loaded to GPU: 1780.99 MB
Moving model(s) has taken 6.86 seconds
Distilled CFG Scale: 3.5
To load target model KModel
Begin to load 1 model
[Unload] Trying to free 9144.89 MB for cuda:0 with 0 models keep loaded ...
[Unload] Current free memory is 986.94 MB ...
[Unload] Unload model JointTextEncoder
[Memory Management] Current Free GPU Memory: 3325.14 MB
[Memory Management] Required Model Memory: 6246.84 MB
[Memory Management] Required Inference Memory: 1024.00 MB
[Memory Management] Estimated Remaining GPU Memory: -3945.70 MB
Patching LoRAs for KModel:  76%|████████████████████████████████████████▍            | 232/304 [01:55<00:54,  1.33it/s]
Press any key to continue . . . ```

[Thorgrimar]

Not sure if that will help but you can set in Forge Diffusion in low Bits to Automatic (LoRa in fp16) to skip patching loras and they will have better quality. This will takes more vram but what is 18 mb for 4GB.

[dunctait]

Can confirm this finally let me use flux Loras with 6gb vram! Thank you

[vehanlankaja]

i have 8gb vram and 24gb ram, but when i load lora in forge it takes a massive amount if time, im using a gguf model for generation.