Would really love to see HiDream and Chroma model support for Forge.

[aiartalchemy]

HiDream
https://huggingface.co/HiDream-ai/HiDream-I1-Full

Chroma
https://huggingface.co/lodestones/Chroma

Both of these models are huge improvements over SD models.
Anyone out there able to integrate these into Forge please?

[HMRMike]

Chroma is supported via an easy patch for now:
https://github.com/croquelois/forgeChroma

But LORA support is broken due to some code which can be modified without much trouble, this will allow Chroma to work with regular Flux LORAs (compatibility issues may remain just because Chroma is not really Flux anymore)
croquelois/forgeChroma#4

[dhosein]

Would love to see HiDream support. I've long since given up on ever understanding ComfyUI well enough to make it work. Forge is about as far as my comfort/understanding level can stretch.

[DrSid42]

How is that supposed to work ? It says "git apply forge.patch" .. that doesn't do anything, as the file doesn't exist ?
It seems I need to checkout the patch as well ? Into the same directory ? Is that even possible in git ? Confusing ..

[HMRMike]

forge.patch should be placed with webui-user.bat. this is the only requirement for running the patch command.
All other files should be copied manually to the folders per steps 2 through 4.

[Bailey2]

Thanks for the answer.
I followed your instructions exactly and moved all the necessary files. But after using the git apply forge.patch command, I got this message:

C:\AI\Forge\webui>git apply forge.patch
error: patch failed: backend/loader.py:22  
error: backend/loader.py: patch does not apply  
error: patch failed: backend/operations.py:12  
error: backend/operations.py: patch does not apply  
error: patch failed: backend/sampling/condition.py:102  
error: backend/sampling/condition.py: patch does not apply  
error: patch failed: backend/text_processing/t5_engine.py:17  
error: backend/text_processing/t5_engine.py: patch does not apply

[HMRMike]

This usually happens when the patch has already been applied once. Most likely a good sign. Place the rest of the files as instructed.
Then fire up Forge and try running a Chroma model (the latest V35 Detail Calibrated is awesome) remember to use the Flux VAE, and a T5XXL encoder. No need to load CLIP G or L at all, CFG~5-7, Distilled CFG=0, Euler Simple or Beta at 25-30 steps (other samplers/schedulers also work but it's safe to start with these).

[Bailey2]

I launched Forge, tried generating an image, and got this message:
AssertionError: You do not have CLIP state dict!

[HMRMike]

I don't see why Chroma's code would complain about CLIP. Are you sure you selected a Chroma checkpoint?
Also to make sure- Revert the patch (git apply -R forge.patch), no need to remove the manually placed files, update Forge, re-apply the patch, and then repeat the apply command just to generate this bunch of "patch does not apply" errors- which should indicate it was applied well already.

At worst- just select to load the CLIP encoders as well, just to see what happens.

[wzwow]

any chance this might happen?

[angelika3377]

please! hidream for forge :<

[aiartalchemy]

I'm just going to add a plug here for HiDream. I've spent a great deal of time using it, Forge and various XL models. There are also quants for HiDream now and it can be run on just 8gb of vram.

Pros of HiDream

• It uses quad clip model including Llama8b instruct. THE BEST PROMPT ADHERENCE of any local opensource model. It is freaking amazing. You can actually prompt EXTREMELY complex situational prompts and it will do it. You can also prompt in prose and it works. This is crazy good.
• High Aesthetic output - has to do with the way the pruned their training data set. This thing just chucks out banger beautiful images one after the other. Almost everything that comes out of the model is extremely pleasing to look out.
• Minimal Censored - It's less censored than Flux.
• extreme coherence in images - highly detailed rooms, objects and clothing details are all very coherent. like all the little belts and buttons are in the right place and object scale in room are generally right.
• Good hands - really nice hands, better than flux. it sometimes makes an extra finger but otherwise great with hands.
• More artists and art styles trained than Flux. not as nearly as many as SDXL but it much easier to control art styles as opposed to flux which really leans into photorealism. HiDream does excellent photorealism as well.
• Capable of EXTEMELY detailed images from a simple prompt.

Negative

• Big slow model. The number of parameters is overkill in my opinion. I wish someone would just train a nice 8 billion model with the original SDXL dataset but with better clip.
• Not nearly as many artists and art styles as SDXL but still way better than Flux.
• Artists and art styles aren't weighted heavy enough. If your prompt is long or complex, you lose the art style and it leans into photorealism. SDXL does not do this nearly as badly.

HiDream is stunning model

[dhosein]

I would really love to try your model. But trying to figure out ComfyUI is beyond me.
Is there currently any other way to use your model as yet?