Brandon/flux model loading#6739

Merged

brandonrising merged 113 commits intomainfrom

brandon/flux-model-loading

Aug 27, 2024

Commits on Aug 26, 2024

Bump diffusers version to include FLUX support.

RyanJDick
authored and
brandonrising
committed
Update imports for compatibility with bumped diffusers version.

RyanJDick
authored and
brandonrising
committed
Update HF download logic to work for black-forest-labs/FLUX.1-schnell.

RyanJDick
authored and
brandonrising
committed
First draft of FluxTextToImageInvocation.

RyanJDick
authored and
brandonrising
committed
Add sentencepiece dependency for the T5 tokenizer.

RyanJDick
authored and
brandonrising
committed
Use the FluxPipeline.encode_prompt() api rather than trying to run the two text encoders separately.

RyanJDick
authored and
brandonrising
committed
Got FLUX schnell working with 8-bit quantization. Still lots of rough edges to clean up.

RyanJDick
authored and
brandonrising
committed
Minor improvements to FLUX workflow.

RyanJDick
authored and
brandonrising
committed
Make 8-bit quantization save/reload work for the FLUX transformer. Reload is still very slow with the current optimum.quanto implementation.

RyanJDick
authored and
brandonrising
committed
Add support for 8-bit quantizatino of the FLUX T5XXL text encoder.

RyanJDick
authored and
brandonrising
committed
Make float16 inference work with FLUX on 24GB GPU.

RyanJDick
authored and
brandonrising
committed
WIP - experimentation

RyanJDick
authored and
brandonrising
committed
Make quantized loading fast.

RyanJDick
authored and
brandonrising
committed
Make quantized loading fast for both T5XXL and FLUX transformer.

RyanJDick
authored and
brandonrising
committed
Split a FluxTextEncoderInvocation out from the FluxTextToImageInvocation. This has the advantage that we benfit from automatic caching when the prompt isn't changed.

RyanJDick
authored and
brandonrising
committed
wip

RyanJDick
authored and
brandonrising
committed
NF4 loading working... I think.

RyanJDick
authored and
brandonrising
committed
NF4 inference working

RyanJDick
authored and
brandonrising
committed
Clean up NF4 implementation.

RyanJDick
authored and
brandonrising
committed
LLM.int8() quantization is working, but still some rough edges to solve.

RyanJDick
authored and
brandonrising
committed
More improvements for LLM.int8() - not fully tested.

RyanJDick
authored and
brandonrising
committed
WIP on moving from diffusers to FLUX

RyanJDick
authored and
brandonrising
committed
Setup flux model loading in the UI
brandonrising
committed
Remove changes to v1 workflow
brandonrising
committed
Manage quantization of models within the loader
brandonrising
committed
Run Ruff
brandonrising
committed
Run Ruff
brandonrising
committed
Some UI cleanup, regenerate schema
brandonrising
committed
Add backend functions and classes for Flux implementation, Update the way flux encoders/tokenizers are loaded for prompt encoding, Update way flux vae is loaded
brandonrising
committed
Run ruff, setup initial text to image node
brandonrising
committed
Add nf4 bnb quantized format
brandonrising
committed
Remove unused param on _run_vae_decoding in flux text to image
brandonrising
committed
Working inference node with quantized bnb nf4 checkpoint
brandonrising
committed
Install sub directories with folders correctly, ensure consistent dtype of tensors in flux pipeline and vae
brandonrising
committed
Select dev/schnell based on state dict, use correct max seq len based on dev/schnell, and shift in inference, separate vae flux params into separate config
brandonrising
committed
Fix FLUX output image clamping. And a few other minor fixes to make inference work with the full bfloat16 FLUX transformer model.

RyanJDick
authored and
brandonrising
committed
Add tqdm progress bar to FLUX denoising.

RyanJDick
authored and
brandonrising
committed
Fix support for 8b quantized t5 encoders, update exception messages in flux loaders
brandonrising
committed
Fix styling/lint
brandonrising
committed
Add t5 encoders and clip embeds to the model manager
brandonrising
committed
Some cleanup of the tags and description of flux nodes
brandonrising
committed
exclude flux models from main model dropdown

Mary Hipp
authored and
brandonrising
committed
add default workflow for flux t2i

Mary Hipp
authored and
brandonrising
committed
Rename t5Encoder -> t5_encoder.

RyanJDick
authored and
brandonrising
committed
Address minor review comments.

RyanJDick
authored and
brandonrising
committed
Update doc string for import_local_model and remove access_token since it's only usable for local file paths
brandonrising
committed
Switch inheritance class of flux model loaders
brandonrising
committed
Various styling and exception type updates
brandonrising
committed
More flux loader cleanup
brandonrising
committed
Remove duplicate log_time(...) function.

RyanJDick
authored and
brandonrising
committed
Add docs to the requantize(...) function explaining why it was copied from optimum-quanto.

RyanJDick
authored and
brandonrising
committed
Move requantize.py to the quatnization/ dir.

RyanJDick
authored and
brandonrising
committed
update flux_model_loader node to take a T5 encoder from node field instead of hardcoded list, assume all models have been downloaded

maryhipp
authored and
brandonrising
committed
add case for clip embed models in probe

maryhipp
authored and
brandonrising
committed
add FLUX schnell starter models and submodels as dependenices or adhoc download options

maryhipp
authored and
brandonrising
committed
fix(ui): only exclude flux main models from linear UI dropdown, not model manager list

Mary Hipp
authored and
brandonrising
committed
fix(ui): pass base/type when installing models, add flux formats to MM badges

Mary Hipp
authored and
brandonrising
committed
feat(ui): create new field for t5 encoder models in nodes

Mary Hipp
authored and
brandonrising
committed
tsc and lint fix

Mary Hipp
authored and
brandonrising
committed
fix schema

Mary Hipp
authored and
brandonrising
committed
update default workflow

maryhipp
authored and
brandonrising
committed
fix(worker) fix T5 type

maryhipp
authored and
brandonrising
committed
add better workflow description

maryhipp
authored and
brandonrising
committed
add better workflow name

maryhipp
authored and
brandonrising
committed
Fix bug in InvokeInt8Params that was causing it to use double the necessary VRAM.

RyanJDick
authored and
brandonrising
committed
Update load_flux_model_bnb_llm_int8.py to work with a single-file FLUX transformer checkpoint.

RyanJDick
authored and
brandonrising
committed
Add docs to the quantization scripts.

RyanJDick
authored and
brandonrising
committed
Fix max_seq_len field description.

RyanJDick
authored and
brandonrising
committed
Remove automatic install of models during flux model loader, remove no longer used import function on context
brandonrising
committed
Run ruff
brandonrising
committed
Undo changes to the v2 dir of frontend types
brandonrising
committed
added FLUX dev to starter models

maryhipp
authored and
brandonrising
committed
Don't install bitsandbytes on macOS
brandonrising
committed
Attribute black-forest-labs/flux for much of the flux code
brandonrising
committed
Mark FLUX nodes as prototypes.

RyanJDick
authored and
brandonrising
committed
Make FLUX get_noise(...) consistent across devices/dtypes.

RyanJDick
authored and
brandonrising
committed
Tidy is_schnell detection logic.

RyanJDick
authored and
brandonrising
committed
Add comment about incorrect T5 Tokenizer size calculation.

RyanJDick
authored and
brandonrising
committed
Rename field positive_prompt -> prompt.

RyanJDick
authored and
brandonrising
committed
Move prepare_latent_image_patches(...) to sampling.py with all of the related FLUX inference code.

RyanJDick
authored and
brandonrising
committed
Run FLUX VAE decoding in the user's preferred dtype rather than float32. Tested, and seems to work well at float16.

RyanJDick
authored and
brandonrising
committed
Update macos test vm to macOS-14
brandonrising
committed
Load and unload clip/t5 encoders and run inference separately in text encoding
brandonrising
committed
Only import bnb quantize file if bitsandbytes is installed
brandonrising
committed
Switch flux to using its own conditioning field
brandonrising
committed
Add script for quantizing a T5 model.

RyanJDick
authored and
brandonrising
committed
Fixes to the T5XXL quantization script.

RyanJDick
authored and
brandonrising
committed
Update the T5 8-bit quantized starter model to use the BnB LLM.int8() variant.

RyanJDick
authored and
brandonrising
committed
Remove all references to optimum-quanto and downgrade diffusers.

RyanJDick
authored and
brandonrising
committed
Update docs for T5 quantization script.

RyanJDick
authored and
brandonrising
committed
Move quantization scripts to a scripts/ subdir.

RyanJDick
authored and
brandonrising
committed
Downgrade revert torch version after removing optimum-qanto, and other minor version-related fixes.

RyanJDick
authored and
brandonrising
committed
Update t5 encoder formats to accurately reflect the quantization strategy and data type
brandonrising
committed
Switch the CLIP-L start model to use our hosted version - which is much smaller.

RyanJDick
authored and
brandonrising
committed
Replace swish() with torch.nn.functional.silu(h). They are functionally equivalent, but in my test VAE deconding was ~8% faster after the change.

RyanJDick
authored and
brandonrising
committed
Setup scaffolding for in progress images and add ability to cancel the flux node
brandonrising
committed
Remove dependency on flux config files
brandonrising
committed
ruff

RyanJDick
authored and
brandonrising
committed
Remove flux repo dependency

RyanJDick
authored and
brandonrising
committed
Downgrade accelerate and huggingface-hub deps to original versions.

RyanJDick
authored and
brandonrising
committed
ruff format

RyanJDick
authored and
brandonrising
committed
Remove outdated TODO.

RyanJDick
authored and
brandonrising
committed
Only install starter models if not already installed
brandonrising
committed
Remove in progress images until we're able to make the valuable
brandonrising
committed
Remove no longer used code in the flux denoise function
brandonrising
committed
Fix type error in tsc
brandonrising
committed
Run ruff
brandonrising
committed
Rename params for flux and flux vae, add comments explaining use of the config_path in model config
brandonrising
committed
update default workflow for flux

Mary Hipp
authored and
brandonrising
committed
remove prompt

Mary Hipp
authored and
brandonrising
committed
Run ruff
brandonrising
committed
default workflow: add steps to exposed fields, add more notes

Mary Hipp
authored and
brandonrising
committed
Update starter model size estimates.
RyanJDick
committed