Skip to content

Commit 7dfe378

Browse files
sayakpaula-r-r-o-w
andauthored
Apply suggestions from code review
Co-authored-by: Aryan <[email protected]>
1 parent ecc16ed commit 7dfe378

File tree

3 files changed

+4
-3
lines changed

3 files changed

+4
-3
lines changed

examples/control-lora/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# Training Control LoRA with Flux
22

3-
This (experimental) example shows how train Control LoRA with [Flux](https://huggingface.co/black-forest-labs/FLUX.1-dev) to condition it with additional structural controls (like depth maps, poses, etc.).
3+
This (experimental) example shows how to train Control LoRAs with [Flux](https://huggingface.co/black-forest-labs/FLUX.1-dev) by conditioning it with additional structural controls (like depth maps, poses, etc.).
44

5-
We simply expand the input channels of Flux.1 Dev from 64 to 128 to allow for additional inputs and then train a regular LoRA on top of it. To account for the newly added input channels, we additional append a LoRA on the underlying layer (`x_embedder`). Inference, however, is performed with the `FluxControlPipeline`.
5+
To incorporate additional condition latents, we expand the input features of Flux.1-Dev from 64 to 128. The first 64 channels correspond to the original input latents to be denoised, while the latter 64 channels correspond to control latents. This expansion happens on the `x_embedder` layer, where the combined latents are projected to the expected feature dimension of rest of the network. Inference is performed using the `FluxControlPipeline`.
66

77
> [!NOTE]
88
> **Gated model**

examples/control-lora/train_control_flux.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -188,7 +188,7 @@ def save_model_card(repo_id: str, image_logs=None, base_model=str, repo_folder=N
188188
img_str += f"![images_{i})](./images_{i}.png)\n"
189189

190190
model_description = f"""
191-
# controlnet-{repo_id}
191+
# control-lora-{repo_id}
192192
193193
These are Control weights trained on {base_model} with new type of conditioning.
194194
{img_str}

examples/control-lora/train_control_lora_flux.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -224,6 +224,7 @@ def save_model_card(repo_id: str, image_logs=None, base_model=str, repo_folder=N
224224
"diffusers",
225225
"control-lora",
226226
"diffusers-training",
227+
"lora",
227228
]
228229
model_card = populate_model_card(model_card, tags=tags)
229230

0 commit comments

Comments
 (0)