Feature/zimage scheduler support#8705
Conversation
Add support for alternative diffusers Flow Matching schedulers: - Euler (default, 1st order) - Heun (2nd order, better quality, 2x slower) - LCM (optimized for few steps) Backend: - Add schedulers.py with scheduler type definitions and class mapping - Modify denoise.py to accept optional scheduler parameter - Add scheduler InputField to flux_denoise invocation (v4.2.0) Frontend: - Add fluxScheduler to Redux state and paramsSlice - Create ParamFluxScheduler component for Linear UI - Add scheduler to buildFLUXGraph for generation
Add support for alternative diffusers Flow Matching schedulers for Z-Image: - Euler (default) - 1st order, optimized for Z-Image-Turbo (8 steps) - Heun (2nd order) - Better quality, 2x slower - LCM - Optimized for few-step generation Backend: - Extend schedulers.py with Z-Image scheduler types and mapping - Add scheduler InputField to z_image_denoise invocation (v1.3.0) - Refactor denoising loop to support diffusers schedulers Frontend: - Add zImageScheduler to Redux state in paramsSlice - Create ParamZImageScheduler component for Linear UI - Add scheduler to buildZImageGraph for generation
|
Euler and Heun are working as expected. However, when I generate with the LCM scheduler, I get the same error as in PR #8704 : I am just using the linear UI at the moment. |
LCM scheduler may have more internal timesteps than user-facing steps, causing user_step to exceed total_steps. This resulted in progress percentage > 1.0, which caused a pydantic validation error. Fix: Only call step_callback when user_step <= total_steps.
…/Pfannkuchensack/InvokeAI into feature/zimage-scheduler-support
|
I can confirm that all three schedulers now run to completion on my system in both the linear and workflow editor views. There is a step count problem. Both Euler and LCM are doing one more step than I ask for (e.g. 9 steps when I ask for 8). Heun is giving me N2-1 steps, i.e. 15 steps when I ask for 8. But maybe this is the expected behavior, because I also get N2-1 when using Heun with SDXL. |
|
Heun N×2-1: Heun is a 2nd-order solver that performs two model evaluations per "user step": |
|
Euler/LCM N+1 Issue: This likely comes from the initial step_callback at step 0: step_callback( |
I'm happy with the explanation for the N+1 steps with Euler and Heun, but the user experience is inconsistent with the SD and SDXL models, which display N steps in the server log. Some users will ask why SDXL shows N steps and Flux/ZImage show N+1 steps. It's a small thing, and if it is too hard or hacky to fix let me know and I'll merge it in. |
…ount Remove the initial step_callback at step=0 to match SD/SDXL behavior. Previously Flux/Z-Image showed N+1 steps (step 0 + N denoising steps), while SD/SDXL showed only N steps. Now all models display N steps consistently in the server log.
|
Now it should show the "right" step count. |
lstein
left a comment
There was a problem hiding this comment.
I'm so sorry, but LCM is still reporting N+1 steps when I request N.
Euler and Heun are reporting the correct number of steps.
- Handle LCM scheduler by using num_inference_steps instead of custom sigmas - Fix progress bar to show user-facing steps instead of internal scheduler steps - Pass scheduler parameter to Z-Image denoise node in graph builder - Add model-aware metadata recall for Flux and Z-Image schedulers
|
But this time. Tested it all. Added the recall and metadata too. |
lstein
left a comment
There was a problem hiding this comment.
Confirmed working as expected. Whewwww!
## Summary Add a new "Denoise - Z-Image + Metadata" node (`ZImageDenoiseMetaInvocation`) that extends the Z-Image denoise node with metadata output for image recall functionality. This follows the same pattern as existing `denoise_latents_meta` (SD1.5/SDXL) and `flux_denoise_meta` (FLUX) nodes. **Captured metadata:** - `width` / `height` - `steps` - `guidance` (guidance_scale) - `denoising_start` / `denoising_end` - `scheduler` - `model` (transformer) - `seed` - `loras` (if applied) ## Related Issues / Discussions Enables metadata recall for Z-Image generated images, similar to existing support for SD1.5, SDXL, and FLUX models. ## QA Instructions 1. Create a workflow using the new "Denoise - Z-Image + Metadata" node 2. Connect the metadata output to a "Save Image" node 3. Generate an image 4. Check that metadata is saved with the image (visible in image info panel) 5. Verify all generation parameters are captured correctly ## Merge Plan Requires `feature/zimage-scheduler-support` #8705 branch to be merged first (base branch). ## Checklist - [x] _The PR has a short but descriptive title, suitable for a changelog_ - [ ] _Tests added / updated (if applicable)_ - [ ] _❗Changes to a redux slice have a corresponding migration_ - [ ] _Documentation added / updated (if applicable)_ - [ ] _Updated `What's New` copy (if doing a release after this PR)_
Summary
Add support for alternative diffusers Flow Matching schedulers for Z-Image models:
The scheduler can be selected in both the Linear UI (Generation Settings → Advanced) and the Workflow Editor (Z-Image Denoise node).
Backend changes:
invokeai/backend/flux/schedulers.pywith Z-Image scheduler types and mappingschedulerInputField toz_image_denoiseinvocation (version 1.2.0 → 1.3.0)num_inference_stepsinstead of custom sigmas (LCM has its own sigma schedule)Frontend changes:
zImageSchedulerto Redux state inparamsSliceParamZImageSchedulercomponent for Linear UI dropdownbuildZImageGraphNote: Only Flow Matching schedulers are compatible with Z-Image. Traditional schedulers like DDIM cannot be used because Z-Image uses velocity prediction (v-prediction) with a linear interpolation schedule, while DDIM expects noise prediction (ε-prediction) with a variance-preserving schedule.
Related Issues / Discussions
QA Instructions
Merge Plan
Standard merge, no special considerations. This PR depends on the Flux scheduler PR being merged first (shared scheduler infrastructure in
schedulers.py).Checklist
What's Newcopy (if doing a release after this PR)