Using Flux checkpoints vs. diff model + VAE + text enc.

[mbac]

Hey Robert,

I have a feature or advice request: there's at least one Flux finetune that I'm interested in (CivitAI, but absolutely SFW) that is offered as a checkpoint (or a GGUF, but that's lower precision). Simply changing the loading nodes won't let me use your Continuum Workflow because there's at least one downstream incompatibility: the ModelSamplingFlux node, for example. Is there a quick and dirty way to replace these for a Flux checkpoint to be used? I know how to use bypass/mute relays/repeaters from the rgthree package, in addition to the Get/Set nodes you used.

Thanks!

[robertvoy]

Hello! Sure you can. Just load it up like this.

[robertvoy]

And instead of using the checkpoint you can use this instead and just load into into the original load model node:
https://civitai.com/models/652009?modelVersionId=733431

[mbac]

Thanks for the quick reply!

Unfortunately, the UNET model is an older version and the author says there's a significant improvement in the latest edition.

I got very good results from the solution you proposed, so first of all, thank you! However, my understanding is that this model might have a finetuned CLIP, so by not using it I may not be leveraging the full model. What do you think?

The incompatibility I referred to is with the CLIP encoders. Both the Flux and 'normal' ones complain that:

  File "/workspace/ComfyUI/nodes.py", line 66, in encode
    tokens = clip.tokenize(text)
             ^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'tokenize'

This is how I tried to implement the more 'complex' change:

Again, thanks!

[robertvoy]

I haven't used Flux checkpoints, but I just tried a basic setup with one and I'm getting the same error.

Let me know if you figure out how to use it in just a basic setup and I'll help you implement it into Flux Continuum.