- Flux Tools - ENHANCEMENT - Memory Management - Unload Memory - To each Tool in Flux Tools

[danilomaiaweb]

Please add the memory management of forge in models flux tools...

When we are using any model to generate an image in Forge, it has a memory management feature that blocks and releases memory with each image generation or model change.

I noticed that if we have used forge to create an image before using flux tools, the memory is blocked, and this leaves little processing resource for flux tools, blocking the loading of pipeline components.

It would be great if when using any flux tools resource, any of the 4 models (depth, canny, redux or fill), had the same memory management, freeing up blocked memory to use its full potential.

See how Forge behaves when generating an image:

[Unload] Trying to free 7988.00 MB for cuda:0 with 0 models keep loaded ... Done.
[Memory Management] Target: JointTextEncoder, Free GPU: 11234.33 MB, Model Require: 5154.62 MB, Previously Loaded: 0.00 MB, Inference Require: 1287.00 MB, Remaining: 4792.71 MB, All loaded to GPU.
Moving model(s) has taken 18.93 seconds
Distilled CFG Scale will be ignored for Schnell
[Unload] Trying to free 9400.91 MB for cuda:0 with 0 models keep loaded ... Current free memory is 5964.12 MB ... Unload model JointTextEncoder Done.
[Memory Management] Target: KModel, Free GPU: 11191.22 MB, Model Require: 6241.47 MB, Previously Loaded: 0.00 MB, Inference Require: 1287.00 MB, Remaining: 3662.75 MB, All loaded to GPU.
Moving model(s) has taken 27.20 seconds
0%| | 0/4 [00:00<?, ?it/s]TeaCache is enabled!
100%|#########################################################################################################################################################################################################################################| 4/4 [00:10<00:00, 2.69s/it]
[Unload] Trying to free 3474.84 MB for cuda:0 with 0 models keep loaded ... Current free memory is 5260.63 MB ... Done.███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:08<00:00, 2.38s/it]
[Memory Management] Target: IntegratedAutoencoderKL, Free GPU: 5260.63 MB, Model Require: 159.87 MB, Previously Loaded: 0.00 MB, Inference Require: 1287.00 MB, Remaining: 3813.76 MB, All loaded to GPU.
Moving model(s) has taken 0.16 seconds
Total progress: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:15<00:00, 3.86s/it]
Total progress: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:15<00:00, 2.38s/it]

First, it checks memory usage, frees up any blocked memory to use its full potential.
This feature would be very interesting in Fluix Tools.

[Unload] Trying to free 8082.24 MB for cuda:0 with 0 models keep loaded ... Current free memory is 5091.51 MB ... Unload model KModel Current free memory is 10946.35 MB ... Done.
[Memory Management] Target: JointTextEncoder, Free GPU: 10946.35 MB, Model Require: 5227.11 MB, Previously Loaded: 0.00 MB, Inference Require: 1287.00 MB, Remaining: 4432.24 MB, All loaded to GPU.
Moving model(s) has taken 2.67 seconds
Distilled CFG Scale will be ignored for Schnell
[Unload] Trying to free 9400.86 MB for cuda:0 with 0 models keep loaded ... Current free memory is 5714.49 MB ... Unload model IntegratedAutoencoderKL Current free memory is 5874.36 MB ... Unload model JointTextEncoder Done.
[Memory Management] Target: KModel, Free GPU: 11101.47 MB, Model Require: 6241.43 MB, Previously Loaded: 0.00 MB, Inference Require: 1287.00 MB, Remaining: 3573.04 MB, All loaded to GPU.

In my case, I have to restart Forge every time I need to use Flux Tools. This is because Flux always leaves the memory blocked every time a model is loaded and only frees the memory again every time a new model is generated.

Sorry if I didn't explain my thoughts correctly, but I think you get the idea.
Thank in advance mate