FLUX IN FORGE TAKES 15 TO 20 MINUTES TO GENERATE AN IMAGE 🙋‍♂️🙋‍♂️ (FORGE IS A FRESH INSTALL)

[tmprabubiz]

---

WINDOWS 10 PRO / RTX 2060 SUPER - 8GB / CPU: 64GB🚴‍♀️🚴‍♀️

THIS IS THE FIRST FEW LINES OF COMMAND PROMPT👇👇👇

Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: f2.0.1v1.10.1-previous-252-g00f1cd36
Commit hash: 00f1cd3
Installing xformers
Launching Web UI with arguments: --forge-ref-a1111-home 'C:\Users\REVIVE\Automatic1111\stable-diffusion-webui' --gfpgan-model 'C:\Users\REVIVE\Automatic1111\stable-diffusion-webui\models\GFPGAN' --always-normal-vram --always-offload-from-vram --xformers --reinstall-xformers --cuda-malloc --cuda-stream --theme dark --ckpt-dir 'C:\Users\REVIVE\Automatic1111\stable-diffusion-webui\models\Stable-diffusion' --vae-dir 'C:\Users\REVIVE\Automatic1111\stable-diffusion-webui\models\VAE' --hypernetwork-dir 'C:\Users\REVIVE\Automatic1111\stable-diffusion-webui\models\hypernetworks' --embeddings-dir 'C:\Users\REVIVE\Automatic1111\stable-diffusion-webui\embeddings' --lora-dir 'C:\Users\REVIVE\Automatic1111\stable-diffusion-webui\models\lora' --controlnet-dir 'C:\Users\REVIVE\Automatic1111\stable-diffusion-webui\models\ControlNet' --controlnet-preprocessor-models-dir 'C:\Users\REVIVE\Automatic1111\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads'
Using cudaMallocAsync backend.
Total VRAM 8192 MB, total RAM 65447 MB
pytorch version: 2.4.0+cu124

THEN I GET THIS ERROR MESSAGE🧨🧨

ONCE I CLICK 'OK', THE FOLLOWING LINES APPEAR (WITH ERROR MESSAGES)👇👇

WARNING:xformers:WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:
PyTorch 2.3.1+cu121 with CUDA 1201 (you have 2.4.0+cu124)
Python 3.10.11 (you have 3.10.6)
Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)
Memory-efficient attention, SwiGLU, sparse and more won't be available.
Set XFORMERS_MORE_DETAILS=1 for more details
J:\FORGE\system\python\lib\site-packages\xformers\ops\fmha\flash.py:211: FutureWarning: torch.library.impl_abstract was renamed to torch.library.register_fake. Please use that instead; we will remove torch.library.impl_abstract in a future version of PyTorch.
@torch.library.impl_abstract("xformers_flash::flash_fwd")
J:\FORGE\system\python\lib\site-packages\xformers\ops\fmha\flash.py:344: FutureWarning: torch.library.impl_abstract was renamed to torch.library.register_fake. Please use that instead; we will remove torch.library.impl_abstract in a future version of PyTorch.
@torch.library.impl_abstract("xformers_flash::flash_bwd")
J:\FORGE\system\python\lib\site-packages\xformers\ops\swiglu_op.py:128: FutureWarning: torch.cuda.amp.custom_fwd(args...) is deprecated. Please use torch.amp.custom_fwd(args..., device_type='cuda') instead.
def forward(cls, ctx, x, w1, b1, w2, b2, w3, b3):
J:\FORGE\system\python\lib\site-packages\xformers\ops\swiglu_op.py:149: FutureWarning: torch.cuda.amp.custom_bwd(args...) is deprecated. Please use torch.amp.custom_bwd(args..., device_type='cuda') instead.
def backward(cls, ctx, dx5):
xformers version: 0.0.27
Set vram state to: NORMAL_VRAM
Always offload VRAM
Device: cuda:0 NVIDIA GeForce RTX 2060 SUPER : cudaMallocAsync
VAE dtype preferences: [torch.float32] -> torch.float32
CUDA Using Stream: True
J:\FORGE\system\python\lib\site-packages\transformers\utils\hub.py:127: FutureWarning: Using TRANSFORMERS_CACHE is deprecated and will be removed in v5 of Transformers. Use HF_HOME instead.
warnings.warn(
Using pytorch cross attention
Using pytorch attention for VAE
ControlNet preprocessor location: C:\Users\REVIVE\Automatic1111\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads
J:\FORGE\webui\modules_forge\patch_basic.py:35: FutureWarning: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
result = original_loader(*args, **kwargs)
2024-08-16 18:06:44,659 - ControlNet - INFO - ControlNet UI callback registered.
Model selected: {'checkpoint_info': {'filename': 'C:\Users\REVIVE\Automatic1111\stable-diffusion-webui\models\Stable-diffusion\Flux\flux1DevV1V2Flux1_flux1DevBNBNF4V2.safetensors', 'hash': 'f0770152'}, 'additional_modules': ['J:\FORGE\webui\models\text_encoder\t5xxl_fp16.safetensors'], 'unet_storage_dtype': 'nf4'}
Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch().
Startup time: 36.0s (prepare environment: 6.7s, launcher: 2.1s, import torch: 12.8s, initialize shared: 0.1s, other imports: 0.8s, load scripts: 2.0s, initialize extra networks: 7.6s, create ui: 2.3s, gradio launch: 1.5s).
Environment vars changed: {'stream': False, 'inference_memory': 1024.0, 'pin_shared_memory': False}
Model selected: {'checkpoint_info': {'filename': 'C:\Users\REVIVE\Automatic1111\stable-diffusion-webui\models\Stable-diffusion\Flux\flux1DevV1V2Flux1_flux1DevBNBNF4V2.safetensors', 'hash': 'f0770152'}, 'additional_modules': ['J:\FORGE\webui\models\text_encoder\t5xxl_fp16.safetensors'], 'unet_storage_dtype': None}
Model selected: {'checkpoint_info': {'filename': 'C:\Users\REVIVE\Automatic1111\stable-diffusion-webui\models\Stable-diffusion\Flux\flux1DevV1V2Flux1_flux1DevBNBNF4V2.safetensors', 'hash': 'f0770152'}, 'additional_modules': ['J:\FORGE\webui\models\text_encoder\t5xxl_fp16.safetensors'], 'unet_storage_dtype': 'nf4'}

PLEASE GUIDE ME TO RECTIFY THE ISSUE

[Yashpal2004]

Try running without any "VAE / Text Encoder"

[tmprabubiz]

UPDATING MADE A WONDER
IT TOOK 4 MINUTES ONLY
HOWEVER, FOR EVERY GENERATION IT CHECKS ALL THE EMBEDDINGS
I AM GOING TO DO A FRESH INSTALLATION
THANK YOU

[danilomaiaweb]

Hi mate,

When I use the flux1-dev-bnb-nf4-v2.safetensors version, the image generation also takes a while. As I use it, the time decreases, I don't understand why.

So, I'm testing a merged version that I found on huggingface https://huggingface.co/drbaph/FLUX.1-schnell-dev-merged-fp8-4step and I'm liking the results.

With this merged version, I can generate very good images with only 4 steps. The time to generate each image is approximately 20s.

My Setup is:
Windows 11 Pro - Raizen 5 3500X
32GB RAM
8GB VRAM RTX 3050
Torch 2.3.1+cu121 autocast half
cuda: 12.1
cudnn: 8907
driver: 560.70
diffusers: 0.29.2
transformers: 4.44.0
python: 3.10.6

See some results:

All these images were generated using the FLUX.1-schnell-dev-merged-fp8-4step model, with only 4 steps, without VAE, with BNB-NF4. The average time of each generation was only 20s.

The first loading takes a while, but after loading the module, from the second creation onwards, the time is reduced considerably.
See more details below:

Stable Diffusion PATH: F:\ForgeFlux\webui
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: f2.0.1v1.10.1-previous-310-g695aad95
Commit hash: 695aad95e45a1cd24c016d3232d2be08b7faad17
CUDA 12.1
Launching Web UI with arguments: --precision full --opt-split-attention --always-batch-cond-uncond --no-half --skip-torch-cuda-test --pin-shared-memory --cuda-malloc --cuda-stream --ckpt-dir 'F:\ModelsForge\Checkpoints' --lora-dir 'F:\ModelsForge\Loras'
Using cudaMallocAsync backend.
Total VRAM 8191 MB, total RAM 32705 MB
pytorch version: 2.3.1+cu121
Set vram state to: NORMAL_VRAM
Always pin shared GPU memory
Device: cuda:0 NVIDIA GeForce RTX 3050 : cudaMallocAsync
VAE dtype preferences: [torch.bfloat16, torch.float32] -> torch.bfloat16
CUDA Using Stream: True
Using pytorch cross attention
Using pytorch attention for VAE
ControlNet preprocessor location: F:\ForgeFlux\webui\models\ControlNetPreprocessor
[-] ADetailer initialized. version: 24.8.0, num models: 10
sd-webui-prompt-all-in-one background API service started successfully.
17:32:07 - ReActor - STATUS - Running v0.7.1-a1 on Device: CUDA
2024-08-16 17:32:09,701 - ControlNet - INFO - ControlNet UI callback registered.
Model selected: {'checkpoint_info': {'filename': 'F:\ModelsForge\Checkpoints\FLUX1-SchnellDev-Merged-fp8-4step.safetensors', 'hash': '9e0fb423'}, 'additional_modules': [], 'unet_storage_dtype': 'nf4'}
Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch().
IIB Database file has been successfully backed up to the backup folder.
Startup time: 35.7s (prepare environment: 12.8s, launcher: 2.4s, import torch: 3.8s, initialize shared: 0.1s, other imports: 1.1s, opts onchange: 0.8s, list SD models: 0.4s, load scripts: 5.5s, create ui: 5.0s, gradio launch: 2.7s, app_started_callback: 1.0s).
Environment vars changed: {'stream': False, 'inference_memory': 1024.0, 'pin_shared_memory': False}
Loading Model: {'checkpoint_info': {'filename': 'F:\ModelsForge\Checkpoints\FLUX1-SchnellDev-Merged-fp8-4step.safetensors', 'hash': '9e0fb423'}, 'additional_modules': [], 'unet_storage_dtype': 'nf4'}
[Unload] Trying to free 953674316406250018963456.00 MB for cuda:0 with 0 models keep loaded ...
StateDict Keys: {'transformer': 776, 'vae': 244, 'text_encoder': 198, 'text_encoder_2': 220, 'ignore': 0}
Using Default T5 Data Type: torch.float16
Working with z of shape (1, 16, 32, 32) = 16384 dimensions.
K-Model Created: {'storage_dtype': 'nf4', 'computation_dtype': torch.bfloat16}
Model loaded in 93.1s (unload existing model: 0.2s, forge model load: 92.8s).
Skipping unconditional conditioning when CFG = 1. Negative Prompts are ignored.
To load target model JointTextEncoder
Begin to load 1 model
[Unload] Trying to free 13465.80 MB for cuda:0 with 0 models keep loaded ...
[Memory Management] Current Free GPU Memory: 7184.00 MB
[Memory Management] Required Model Memory: 9570.62 MB
[Memory Management] Required Inference Memory: 1024.00 MB
[Memory Management] Estimated Remaining GPU Memory: -3410.62 MB
[Memory Management] Loaded to GPU for backward capability: 73.14 MB
[Memory Management] Loaded to CPU Swap: 4790.00 MB (blocked method)
[Memory Management] Loaded to GPU: 4852.99 MB
Moving model(s) has taken 18.67 seconds
Distilled CFG Scale will be ignored for Schnell
To load target model KModel
Begin to load 1 model
[Unload] Trying to free 9137.91 MB for cuda:0 with 0 models keep loaded ...
[Unload] Current free memory is 1637.82 MB ...
[Unload] Unload model JointTextEncoder
[Memory Management] Current Free GPU Memory: 7121.19 MB
[Memory Management] Required Model Memory: 6241.47 MB
[Memory Management] Required Inference Memory: 1024.00 MB
[Memory Management] Estimated Remaining GPU Memory: -144.28 MB
[Memory Management] Loaded to CPU Swap: 1426.84 MB (blocked method)
[Memory Management] Loaded to GPU: 4814.55 MB
Moving model(s) has taken 151.16 seconds
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:32<00:00, 8.12s/it]
To load target model IntegratedAutoencoderKL██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:18<00:00, 4.55s/it]
Begin to load 1 model
[Unload] Trying to free 2730.93 MB for cuda:0 with 0 models keep loaded ...
[Unload] Current free memory is 2037.39 MB ...
[Unload] Unload model KModel
[Memory Management] Current Free GPU Memory: 7114.03 MB
[Memory Management] Required Model Memory: 159.87 MB
[Memory Management] Required Inference Memory: 1024.00 MB
[Memory Management] Estimated Remaining GPU Memory: 5930.16 MB
Moving model(s) has taken 1.79 seconds
Total progress: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:28<00:00, 7.11s/it]
Skipping unconditional conditioning when CFG = 1. Negative Prompts are ignored.███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:28<00:00, 4.55s/it]
To load target model JointTextEncoder
Begin to load 1 model
[Unload] Trying to free 13560.04 MB for cuda:0 with 0 models keep loaded ...
[Unload] Current free memory is 6944.74 MB ...
[Unload] Unload model IntegratedAutoencoderKL
[Memory Management] Current Free GPU Memory: 7104.61 MB
[Memory Management] Required Model Memory: 9643.11 MB
[Memory Management] Required Inference Memory: 1024.00 MB
[Memory Management] Estimated Remaining GPU Memory: -3562.50 MB
[Memory Management] Loaded to GPU for backward capability: 145.52 MB
[Memory Management] Loaded to CPU Swap: 4998.00 MB (blocked method)
[Memory Management] Loaded to GPU: 4789.86 MB
Moving model(s) has taken 1.51 seconds
Distilled CFG Scale will be ignored for Schnell
To load target model KModel
Begin to load 1 model
[Unload] Trying to free 9137.91 MB for cuda:0 with 0 models keep loaded ...
[Unload] Current free memory is 1796.03 MB ...
[Unload] Unload model JointTextEncoder
[Memory Management] Current Free GPU Memory: 7100.03 MB
[Memory Management] Required Model Memory: 6241.47 MB
[Memory Management] Required Inference Memory: 1024.00 MB
[Memory Management] Estimated Remaining GPU Memory: -165.44 MB
[Memory Management] Loaded to CPU Swap: 1446.64 MB (blocked method)
[Memory Management] Loaded to GPU: 4794.75 MB
Moving model(s) has taken 2.27 seconds
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:13<00:00, 3.41s/it]
To load target model IntegratedAutoencoderKL██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:10<00:00, 2.79s/it]
Begin to load 1 model
[Unload] Trying to free 2730.93 MB for cuda:0 with 0 models keep loaded ...
[Unload] Current free memory is 2040.43 MB ...
[Unload] Unload model KModel
[Memory Management] Current Free GPU Memory: 7099.45 MB
[Memory Management] Required Model Memory: 159.87 MB
[Memory Management] Required Inference Memory: 1024.00 MB
[Memory Management] Estimated Remaining GPU Memory: 5915.57 MB
Moving model(s) has taken 1.00 seconds
Total progress: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:12<00:00, 3.02s/it]
Skipping unconditional conditioning when CFG = 1. Negative Prompts are ignored.███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:12<00:00, 2.79s/it]
To load target model KModel
Begin to load 1 model
[Unload] Trying to free 9137.91 MB for cuda:0 with 0 models keep loaded ...
[Unload] Current free memory is 6939.57 MB ...
[Unload] Unload model IntegratedAutoencoderKL
[Memory Management] Current Free GPU Memory: 7099.45 MB
[Memory Management] Required Model Memory: 6241.47 MB
[Memory Management] Required Inference Memory: 1024.00 MB
[Memory Management] Estimated Remaining GPU Memory: -166.02 MB
[Memory Management] Loaded to CPU Swap: 1446.64 MB (blocked method)
[Memory Management] Loaded to GPU: 4794.75 MB
Moving model(s) has taken 1.66 seconds
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:13<00:00, 3.42s/it]
To load target model IntegratedAutoencoderKL██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:10<00:00, 2.78s/it]
Begin to load 1 model
[Unload] Trying to free 2730.93 MB for cuda:0 with 0 models keep loaded ...
[Unload] Current free memory is 1995.33 MB ...
[Unload] Unload model KModel
[Memory Management] Current Free GPU Memory: 7098.29 MB
[Memory Management] Required Model Memory: 159.87 MB
[Memory Management] Required Inference Memory: 1024.00 MB
[Memory Management] Estimated Remaining GPU Memory: 5914.42 MB
Moving model(s) has taken 1.00 seconds
Total progress: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:12<00:00, 3.01s/it]
Total progress: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:12<00:00, 2.78s/it]

See, only 17s to generate the image:

Result in 4 steps, 17s...

Feel free to test it too
Best Regards here from Brazil

[nitinmukesh]

Interesting information, Thank you.

[dddddssa]

Is it possible that WIN10 can also affect image generation time? My Win101070ti 8G model takes five minutes to generate an NF4 model, while my friend's 20808G model takes more than 60 seconds

[juujaab]

1070ti is slower because AI performance is weak

[talon87]

Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
Version: f2.0.1v1.10.1-previous-507-g3dd5e19c
Commit hash: 3dd5e19
Launching Web UI with arguments: --medvram --xformers
Arg --medvram is removed in Forge.
Now memory management is fully automatic and you do not need any command flags.
Please just remove this flag.
In extreme cases, if you want to force previous lowvram/medvram behaviors, please use --always-offload-from-vram
Total VRAM 12287 MB, total RAM 16273 MB
pytorch version: 2.3.1+cu121
WARNING:xformers:A matching Triton is not available, some optimizations will not be enabled
Traceback (most recent call last):
File "C:\Users\Ratolouco\Documents\webui_forge_cu121_torch231\webui\venv\lib\site-packages\xformers_init_.py", line 57, in _is_triton_available
import triton # noqa
ModuleNotFoundError: No module named 'triton'
xformers version: 0.0.27
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3060 : native
Hint: your device supports --cuda-malloc for potential speed improvements.
VAE dtype preferences: [torch.bfloat16, torch.float32] -> torch.bfloat16
CUDA Using Stream: False
Using xformers cross attention
Using xformers attention for VAE
ControlNet preprocessor location: C:\Users\Ratolouco\Documents\webui_forge_cu121_torch231\webui\models\ControlNetPreprocessor
2024-09-06 12:08:06,143 - ControlNet - INFO - ControlNet UI callback registered.
Model selected: {'checkpoint_info': {'filename': 'C:\Users\Ratolouco\Documents\webui_forge_cu121_torch231\webui\models\Stable-diffusion\flux1-dev-fp8.safetensors', 'hash': 'be9881f4'}, 'additional_modules': [], 'unet_storage_dtype': None}
Using online LoRAs in FP16: True
Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch().
Startup time: 11.6s (prepare environment: 1.9s, import torch: 5.0s, other imports: 0.3s, load scripts: 1.4s, create ui: 1.9s, gradio launch: 0.9s).
Environment vars changed: {'stream': False, 'inference_memory': 1024.0, 'pin_shared_memory': True}
[GPU Setting] You will use 91.67% GPU memory (11263.00 MB) to load weights, and use 8.33% GPU memory (1024.00 MB) to do matrix computation.
Environment vars changed: {'stream': False, 'inference_memory': 1024.0, 'pin_shared_memory': False}
[GPU Setting] You will use 91.67% GPU memory (11263.00 MB) to load weights, and use 8.33% GPU memory (1024.00 MB) to do matrix computation.
Model selected: {'checkpoint_info': {'filename': 'C:\Users\Ratolouco\Documents\webui_forge_cu121_torch231\webui\models\Stable-diffusion\flux1-dev-fp8.safetensors', 'hash': 'be9881f4'}, 'additional_modules': [], 'unet_storage_dtype': None}
Using online LoRAs in FP16: False
Environment vars changed: {'stream': False, 'inference_memory': 1024.0, 'pin_shared_memory': True}
[GPU Setting] You will use 91.67% GPU memory (11263.00 MB) to load weights, and use 8.33% GPU memory (1024.00 MB) to do matrix computation.
Model selected: {'checkpoint_info': {'filename': 'C:\Users\Ratolouco\Documents\webui_forge_cu121_torch231\webui\models\Stable-diffusion\flux1-dev-fp8.safetensors', 'hash': 'be9881f4'}, 'additional_modules': [], 'unet_storage_dtype': None}
Using online LoRAs in FP16: True
Loading Model: {'checkpoint_info': {'filename': 'C:\Users\Ratolouco\Documents\webui_forge_cu121_torch231\webui\models\Stable-diffusion\flux1-dev-fp8.safetensors', 'hash': 'be9881f4'}, 'additional_modules': [], 'unet_storage_dtype': None}
[Unload] Trying to free all memory for cuda:0 with 0 models keep loaded ... Done.

Can anyone help me with this error...........

[Crocodilian12]

If you are having trouble with configuration, I recommend installing Forge with either Stability Matrix or Pinokio. Either will set up your local Webui Forge installation with correct environments, paths and dependencies.

It will save you a lot of troubleshooting . . .

https://github.com/LykosAI/StabilityMatrix