diff --git a/docs.json b/docs.json index 344adab9..029c5798 100644 --- a/docs.json +++ b/docs.json @@ -119,6 +119,7 @@ "tutorials/flux/flux1-krea-dev", "tutorials/flux/flux-1-kontext-dev", "tutorials/flux/flux-1-text-to-image", + "tutorials/flux/flux-1-uso", "tutorials/flux/flux-1-fill-dev", "tutorials/flux/flux-1-controlnet" ] @@ -676,6 +677,7 @@ "zh-CN/tutorials/flux/flux1-krea-dev", "zh-CN/tutorials/flux/flux-1-kontext-dev", "zh-CN/tutorials/flux/flux-1-text-to-image", + "zh-CN/tutorials/flux/flux-1-uso", "zh-CN/tutorials/flux/flux-1-fill-dev", "zh-CN/tutorials/flux/flux-1-controlnet" ] diff --git a/images/tutorial/flux/flux1_uso_reference_image_gen.jpg b/images/tutorial/flux/flux1_uso_reference_image_gen.jpg new file mode 100644 index 00000000..ec1c1241 Binary files /dev/null and b/images/tutorial/flux/flux1_uso_reference_image_gen.jpg differ diff --git a/images/tutorial/flux/flux1_uso_reference_image_gen_style_reference_only.jpg b/images/tutorial/flux/flux1_uso_reference_image_gen_style_reference_only.jpg new file mode 100644 index 00000000..2f8a5ac7 Binary files /dev/null and b/images/tutorial/flux/flux1_uso_reference_image_gen_style_reference_only.jpg differ diff --git a/snippets/tutorials/update-reminder.mdx b/snippets/tutorials/update-reminder.mdx index 18dea64d..1439c861 100644 --- a/snippets/tutorials/update-reminder.mdx +++ b/snippets/tutorials/update-reminder.mdx @@ -7,7 +7,7 @@ Workflows in this guide can be found in the [Workflow Templates](/interface/feat If you can't find them in the template, your ComfyUI may be outdated.(Desktop version's update will delay sometime) If nodes are missing when loading a workflow, possible reasons: -1. Not using the latest ComfyUI version(Nightly version) -2. Using Stable or Desktop version (Latest changes may not be included) +1. You are not using the latest ComfyUI version(Nightly version) +2. You are using Stable or Desktop version (Latest changes may not be included) 3. Some nodes failed to import at startup \ No newline at end of file diff --git a/tutorials/flux/flux-1-uso.mdx b/tutorials/flux/flux-1-uso.mdx new file mode 100644 index 00000000..d69bdabf --- /dev/null +++ b/tutorials/flux/flux-1-uso.mdx @@ -0,0 +1,107 @@ +--- +title: "ByteDance USO ComfyUI Native Workflow example" +description: "Unified Style and Subject-Driven Generation with ByteDance's USO model" +sidebarTitle: "ByteDance USO" +--- + +import UpdateReminder from '/snippets/tutorials/update-reminder.mdx' + +**USO (Unified Style-Subject Optimized)** is a model developed by ByteDance's UXO Team that unifies style-driven and subject-driven generation tasks. +Built on FLUX.1-dev architecture, the model achieves both style similarity and subject consistency through disentangled learning and style reward learning (SRL). + +USO supports three main approaches: +- **Subject-Driven**: Place subjects into new scenes while maintaining identity consistency +- **Style-Driven**: Apply artistic styles to new content based on reference images +- **Combined**: Use both subject and style references simultaneously + +**Related Links** +- [Project Page](https://bytedance.github.io/USO/) +- [GitHub](https://github.com/bytedance/USO) +- [Model Weights](https://huggingface.co/bytedance-research/USO) + +## ByteDance USO ComfyUI Native Workflow + + + +### 1. Workflow and input + +Download the image below and drag it into ComfyUI to load the corresponding workflow. + +![Workflow](https://raw.githubusercontent.com/Comfy-Org/example_workflows/refs/heads/main/flux/bytedance-uso/bytedance-uso.png) + + +

Download JSON Workflow

+
+ +Use the image below as an input image. + +![input](https://raw.githubusercontent.com/Comfy-Org/example_workflows/refs/heads/main/flux/bytedance-uso/input.png) + +### 2. Model links + + +**checkpoints** + +- [flux1-dev-fp8.safetensors](https://huggingface.co/Comfy-Org/flux1-dev/resolve/main/flux1-dev-fp8.safetensors) + +**loras** + +- [uso-flux1-dit-lora-v1.safetensors](https://huggingface.co/Comfy-Org/USO_1.0_Repackaged/resolve/main/split_files/loras/uso-flux1-dit-lora-v1.safetensors) + +**model_patches** + +- [uso-flux1-projector-v1.safetensors](https://huggingface.co/Comfy-Org/USO_1.0_Repackaged/resolve/main/split_files/model_patches/uso-flux1-projector-v1.safetensors) + +**clip_visions** +- [sigclip_vision_patch14_384.safetensors](https://huggingface.co/Comfy-Org/sigclip_vision_384/resolve/main/sigclip_vision_patch14_384.safetensors) + +Please download all models and place them in the following directories: + +``` +📂 ComfyUI/ +├── 📂 models/ +│ ├── 📂 checkpoints/ +│ │ └── flux1-dev-fp8.safetensors +│ ├── 📂 loras/ +│ │ └── uso-flux1-dit-lora-v1.safetensors +│ ├── 📂 model_patches/ +│ │ └── uso-flux1-projector-v1.safetensors +│ ├── 📂 clip_visions/ +│ │ └── sigclip_vision_patch14_384.safetensors +``` + + +### 3. Workflow instructions + +![Workflow instructions](/images/tutorial/flux/flux1_uso_reference_image_gen.jpg) + +1. Load models: + - 1.1 Ensure the `Load Checkpoint` node has `flux1-dev-fp8.safetensors` loaded + - 1.2 Ensure the `LoraLoaderModelOnly` node has `dit_lora.safetensors` loaded + - 1.3 Ensure the `ModelPatchLoader` node has `projector.safetensors` loaded + - 1.4 Ensure the `Load CLIP Vision` node has `sigclip_vision_patch14_384.safetensors` loaded +2. Click `Upload` to upload the input image we provided +3. In the example, we only use the `content reference` image input. If you want to use the `style reference` image input, you can use `Ctrl-B` to bypass the marked node group. +4. Write your prompt or keep default +5. The EasyCache node is for inference acceleration, but it will also sacrifice some quality and details. You can bypass it (Ctrl+B) if you don't need to use it. +6. Click the `Run` button, or use the shortcut `Ctrl(Cmd) + Enter` to run the workflow + +### 4. Additional Notes + +1. Style reference only: + +We also provide a workflow that only uses style reference in the same workflow we provided + +![Workflow](/images/tutorial/flux/flux1_uso_reference_image_gen_style_reference_only.jpg) +The only different is we replaced the `content reference` node and only use an `Empty Latent Image` node. + +2. You can also bypass whole `Style Reference` group and use the workflow as a text to image workflow, which means this workflow has 4 variations +- Only use content (subject) reference +- Only use style reference +- Mixed content and style reference +- As a text to image workflow diff --git a/zh-CN/tutorials/flux/flux-1-uso.mdx b/zh-CN/tutorials/flux/flux-1-uso.mdx new file mode 100644 index 00000000..e0fd3d9f --- /dev/null +++ b/zh-CN/tutorials/flux/flux-1-uso.mdx @@ -0,0 +1,106 @@ +--- +title: "字节跳动 USO ComfyUI 原生工作流示例" +description: "使用字节跳动 USO 模型实现统一风格和主体驱动生成" +sidebarTitle: "字节跳动 USO" +--- + +import UpdateReminder from '/snippets/zh/tutorials/update-reminder.mdx' + +**USO (Unified Style-Subject Optimized)** 是字节跳动 UXO 团队开发的模型,统一了风格驱动和主体驱动生成任务。 +基于 FLUX.1-dev 架构构建,该模型通过解耦学习和风格奖励学习 (SRL) 实现了风格相似性和主体一致性。 + +USO 支持三种主要方法: + +- **主体驱动**:将主体放置到新场景中,同时保持身份一致性 +- **风格驱动**:基于参考图像将艺术风格应用于新内容 +- **组合模式**:同时使用主体和风格参考 + +**相关链接** +- [项目主页](https://bytedance.github.io/USO/) +- [GitHub](https://github.com/bytedance/USO) +- [模型权重](https://huggingface.co/bytedance-research/USO) + +## 字节跳动 USO ComfyUI 原生工作流 + + + +### 1. 工作流和输入 + +下载下方图像并拖拽到 ComfyUI 中以加载对应的工作流。 + +![工作流](https://raw.githubusercontent.com/Comfy-Org/example_workflows/refs/heads/main/flux/bytedance-uso/bytedance-uso.png) + + +

下载 JSON 工作流

+
+ +使用下面的图片作为输入 + +![输入图像](https://raw.githubusercontent.com/Comfy-Org/example_workflows/refs/heads/main/flux/bytedance-uso/input.png) + +### 2. 模型链接 + +**checkpoints** + +- [flux1-dev-fp8.safetensors](https://huggingface.co/Comfy-Org/flux1-dev/resolve/main/flux1-dev-fp8.safetensors) + +**loras** + +- [uso-flux1-dit-lora-v1.safetensors](https://huggingface.co/Comfy-Org/USO_1.0_Repackaged/resolve/main/split_files/loras/uso-flux1-dit-lora-v1.safetensors) + +**model_patches** + +- [uso-flux1-projector-v1.safetensors](https://huggingface.co/Comfy-Org/USO_1.0_Repackaged/resolve/main/split_files/model_patches/uso-flux1-projector-v1.safetensors) + +**clip_visions** +- [sigclip_vision_patch14_384.safetensors](https://huggingface.co/Comfy-Org/sigclip_vision_384/resolve/main/sigclip_vision_patch14_384.safetensors) + +请下载所有模型并将它们放置在以下目录中: + +``` +📂 ComfyUI/ +├── 📂 models/ +│ ├── 📂 checkpoints/ +│ │ └── flux1-dev-fp8.safetensors +│ ├── 📂 loras/ +│ │ └── uso-flux1-dit-lora-v1.safetensors +│ ├── 📂 model_patches/ +│ │ └── uso-flux1-projector-v1.safetensors +│ ├── 📂 clip_visions/ +│ │ └── sigclip_vision_patch14_384.safetensors +``` + + +### 3. 工作流说明 + +![工作流说明](/images/tutorial/flux/flux1_uso_reference_image_gen.jpg) + +1. 加载模型: + - 1.1 确保 `Load Checkpoint` 节点已加载 `flux1-dev-fp8.safetensors` + - 1.2 确保 `LoraLoaderModelOnly` 节点已加载 `dit_lora.safetensors` + - 1.3 确保 `ModelPatchLoader` 节点已加载 `projector.safetensors` + - 1.4 确保 `Load CLIP Vision` 节点已加载 `sigclip_vision_patch14_384.safetensors` +2. 点击 `Upload` 上传我们提供的输入图像 +3. 在示例中,我们只使用 `content reference` 图像输入。如果你想使用 `style reference` 图像输入,可以使用 `Ctrl+B` 绕过标记的节点组。 +4. 编写你的提示词或保持默认设置 +5. EasyCache 节点用于推理加速,但也会牺牲一些质量和细节。如果不需要使用,可以用 `Ctrl+B` 绕过它。 +6. 点击 `Run` 按钮,或使用快捷键 `Ctrl(Cmd) + Enter` 运行工作流 + +### 4. 补充说明 + +1. 仅使用风格参考: + +我们在同一个工作流中也提供了仅使用风格参考的版本 +![工作流](/images/tutorial/flux/flux1_uso_reference_image_gen_style_reference_only.jpg) +唯一的区别是我们替换了 `content reference` 节点,仅使用 `Empty Latent Image` 节点来创建一个我们需要的图像大小 + +2. 你也可以 **绕过(Ctrl+B)** 整个 `Style Reference` 组,将工作流用作文本到图像的工作流,也就是这个文本存在 4 个变体 +- 仅使用内容(主体)参考 +- 仅使用风格参考 +- 混合内容及风格参考 +- 作为文生图工作流