Skip to content
Open
Changes from 1 commit
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
ec5449f
Support both huggingface_hub `v0.x` and `v1.x` (#12389)
Wauplin Sep 25, 2025
016316a
mirage pipeline first commit
Sep 26, 2025
4ac274b
use attention processors
Sep 26, 2025
904debc
use diffusers rmsnorm
Sep 26, 2025
122115a
use diffusers timestep embedding method
Sep 26, 2025
4588bbe
[CI] disable installing transformers from main in ci for now. (#12397)
sayakpaul Sep 26, 2025
e3fe0e8
remove MirageParams
Sep 26, 2025
85ae87b
checkpoint conversion script
Sep 26, 2025
9a697d0
ruff formating
Sep 26, 2025
9c09445
[docs] slight edits to the attention backends docs. (#12394)
sayakpaul Sep 26, 2025
041501a
[docs] remove docstrings from repeated methods in `lora_pipeline.py` …
sayakpaul Sep 26, 2025
19085ac
Don't skip Qwen model tests for group offloading with disk (#12382)
sayakpaul Sep 29, 2025
0a15111
Fix #12116: preserve boolean dtype for attention masks in ChromaPipe…
akshay-babbar Sep 29, 2025
64a5187
[quantization] feat: support aobaseconfig classes in `TorchAOConfig` …
sayakpaul Sep 29, 2025
ccedeca
[docs] Distributed inference (#12285)
stevhliu Sep 29, 2025
c07fcf7
[docs] Model formats (#12256)
stevhliu Sep 29, 2025
76d4e41
[modular]some small fix (#12307)
yiyixuxu Sep 29, 2025
20fd00b
[Tests] Add single file tester mixin for Models and remove unittest d…
DN6 Sep 30, 2025
0e12ba7
fix 3 xpu failures uts w/ latest pytorch (#12408)
yao-matrix Sep 30, 2025
b596545
Install latest prerelease from huggingface_hub when installing transf…
Wauplin Sep 30, 2025
d7a1a03
[docs] CP (#12331)
stevhliu Sep 30, 2025
cc5b31f
[docs] Migrate syntax (#12390)
stevhliu Sep 30, 2025
34fa9dd
remove dependencies to old checkpoints
Sep 30, 2025
5cc965a
remove old checkpoints dependency
Sep 30, 2025
d79cd8f
move default height and width in checkpoint config
Sep 30, 2025
f2759fd
add docstrings
Sep 30, 2025
394f725
if conditions and raised as ValueError instead of asserts
Sep 30, 2025
54fb063
small fix
Sep 30, 2025
c49fafb
nit remove try block at import
Sep 30, 2025
7e7df35
mirage pipeline doc
Sep 30, 2025
814d710
[tests] cache non lora pipeline outputs. (#12298)
sayakpaul Oct 1, 2025
9ae5b62
[ci] xfail failing tests in CI. (#12418)
sayakpaul Oct 2, 2025
b429796
[core] conditionally import torch distributed stuff. (#12420)
sayakpaul Oct 2, 2025
7242b5f
FIX Test to ignore warning for enable_lora_hotswap (#12421)
BenjaminBossan Oct 2, 2025
941ac9c
[training-scripts] Make more examples UV-compatible (follow up on #12…
linoytsaban Oct 3, 2025
2b7deff
fix scale_shift_factor being on cpu for wan and ltx (#12347)
vladmandic Oct 5, 2025
c3675d4
[core] support QwenImage Edit Plus in modular (#12416)
sayakpaul Oct 5, 2025
ce90f9b
[FIX] Text to image training peft version (#12434)
SahilCarterr Oct 6, 2025
7f3e9b8
make flux ready for mellon (#12419)
sayakpaul Oct 6, 2025
cf4b97b
[perf] Cache version checks (#12399)
cbensimon Oct 6, 2025
0974b4c
[i18n-KO] Fix typo and update translation in ethical_guidelines.md (#…
braintrue Oct 6, 2025
2d69bac
handle offload_state_dict when initing transformers models (#12438)
sayakpaul Oct 7, 2025
de03851
update doc
Oct 7, 2025
a69aa4b
rename model to photon
Oct 7, 2025
1066de8
[Qwen LoRA training] fix bug when offloading (#12440)
linoytsaban Oct 7, 2025
2dc3167
Align Flux modular more and more with Qwen modular (#12445)
sayakpaul Oct 8, 2025
35e538d
fix dockerfile definitions. (#12424)
sayakpaul Oct 8, 2025
345864e
fix more torch.distributed imports (#12425)
sayakpaul Oct 8, 2025
9e099a7
mirage pipeline first commit
Sep 26, 2025
6e10ed4
use attention processors
Sep 26, 2025
866c6de
use diffusers rmsnorm
Sep 26, 2025
4e8b647
use diffusers timestep embedding method
Sep 26, 2025
472ad97
remove MirageParams
Sep 26, 2025
97a231e
checkpoint conversion script
Sep 26, 2025
35d721f
ruff formating
Sep 26, 2025
775a115
remove dependencies to old checkpoints
Sep 30, 2025
1c6c25c
remove old checkpoints dependency
Sep 30, 2025
b0d965c
move default height and width in checkpoint config
Sep 30, 2025
235fe49
add docstrings
Sep 30, 2025
a6ff579
if conditions and raised as ValueError instead of asserts
Sep 30, 2025
3a91503
small fix
Sep 30, 2025
e200cf6
nit remove try block at import
Sep 30, 2025
2ea8976
mirage pipeline doc
Sep 30, 2025
26429a3
update doc
Oct 7, 2025
0abe136
rename model to photon
Oct 7, 2025
fe0e3d5
add text tower and vae in checkpoint
Oct 8, 2025
855b068
update doc
Oct 8, 2025
d2c6bdd
Merge branch 'mirage' of https://github.com/Photoroom/diffusers into …
Oct 8, 2025
89beae8
update photon doc
Oct 8, 2025
2df0e2f
ruff fixes
Oct 8, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 7 additions & 13 deletions docs/source/en/api/pipelines/mirage.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,18 +22,12 @@ Mirage is a text-to-image diffusion model using a transformer-based architecture

Key features:

- **Transformer Architecture**: Uses a modern transformer-based denoising model with attention mechanisms optimized for image generation
- **Flow Matching**: Employs flow matching with Euler discrete scheduling for efficient sampling
- **Simplified MMDIT architecture**: Uses a simplified MMDIT architecture for image generation where text tokens are not updated through the transformer blocks
- **Flow Matching**: Employs flow matching with discrete scheduling for efficient sampling
- **Flexible VAE Support**: Compatible with both Flux VAE (8x compression, 16 latent channels) and DC-AE (32x compression, 32 latent channels)
- **T5Gemma Text Encoder**: Uses Google's T5Gemma-2B-2B-UL2 model for text encoding with strong text-image alignment
- **T5Gemma Text Encoder**: Uses Google's T5Gemma-2B-2B-UL2 model for text encoding offering multiple language support
- **Efficient Architecture**: ~1.3B parameters in the transformer, enabling fast inference while maintaining quality
- **Modular Design**: Text encoder and VAE weights are loaded from HuggingFace, keeping checkpoint sizes small

<Tip>

Make sure to check out the Schedulers [guide](../../using-diffusers/schedulers) to learn how to explore the tradeoff between scheduler speed and quality, and see the [reuse components across pipelines](../../using-diffusers/loading#reuse-a-pipeline) section to learn how to efficiently load the same components into multiple pipelines.

</Tip>

## Loading the Pipeline

Expand All @@ -46,7 +40,7 @@ from diffusers import MiragePipeline
pipe = MiragePipeline.from_pretrained("path/to/mirage_checkpoint")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we'll be able to store the checkpoint on Hugging Face as well, right? If yes, we should not forget to update the paths here to the official one, to make this truly copy-paste and run.

pipe.to("cuda")

prompt = "A digital painting of a rusty, vintage tram on a sandy beach"
prompt = "A vibrant night sky filled with colorful fireworks, with one large firework burst forming the glowing text “Photon” in bright, sparkling light"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haha, awesome!

image = pipe(prompt, num_inference_steps=28, guidance_scale=4.0).images[0]
image.save("mirage_output.png")
```
Expand Down Expand Up @@ -123,11 +117,11 @@ Key parameters for image generation:
```py
# Example with custom parameters
image = pipe(
prompt="A serene mountain landscape at sunset",
prompt="A vibrant night sky filled with colorful fireworks, with one large firework burst forming the glowing text “Photon” in bright, sparkling light",
num_inference_steps=28,
guidance_scale=4.0,
height=1024,
width=1024,
height=512,
width=512,
generator=torch.Generator("cuda").manual_seed(42)
).images[0]
```
Expand Down