Scheduler sigma index out-of-bounds #10738
Description
Describe the bug
This is a continuation of issue #10266 with related pr #10267 which provided a fix for some, but not all schedulers
specifically, it fixed DPMSolverMultistepInverseScheduler, but NOT DPMSolverMultistepScheduler
Reproduction
sampler configurations that produce error:
class=DPMSolverMultistepScheduler config={'num_train_timesteps': 1000, 'beta_start': 0.00085, 'beta_end': 0.012, 'beta_schedule': 'squaredcos_cap_v2', 'solver_order': 2, 'prediction_type': 'epsilon', 'dynamic_thresholding_ratio': 0.995, 'sample_max_value': 1.0, 'algorithm_type': 'dpmsolver++', 'solver_type': 'midpoint', 'lower_order_final': True, 'use_lu_lambdas': True, 'flow_shift': 1.0, 'final_sigmas_type': 'zero', 'lambda_min_clipped': -inf, 'timestep_spacing': 'linspace', 'steps_offset': 0}
class=DPMSolverMultistepScheduler config={'num_train_timesteps': 1000, 'beta_start': 0.00085, 'beta_end': 0.012, 'beta_schedule': 'squaredcos_cap_v2', 'solver_order': 2, 'prediction_type': 'epsilon', 'dynamic_thresholding_ratio': 0.995, 'sample_max_value': 1.0, 'algorithm_type': 'dpmsolver++', 'solver_type': 'midpoint', 'lower_order_final': True, 'use_karras_sigmas': True, 'flow_shift': 1.0, 'final_sigmas_type': 'zero', 'lambda_min_clipped': -inf, 'timestep_spacing': 'linspace', 'steps_offset': 0}
or shorter, out of all combinations problematic are:
beta_schedule=squaredcos_cap_v2 combined with either use_lu_lambdas or use_karras_sigmas
Logs
│ /home/vlado/dev/sdnext/venv/lib/python3.12/site-packages/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py:1237 in __call__ │
│ │
│ 1236 │ │ │ │ latents_dtype = latents.dtype │
│ ❱ 1237 │ │ │ │ latents = self.scheduler.step(noise_pred, t, latents, **extra_step_kwargs, return_dict=False)[0] │
│ 1238 │ │ │ │ if latents.dtype != latents_dtype: │
│ │
│ /home/vlado/dev/sdnext/venv/lib/python3.12/site-packages/diffusers/schedulers/scheduling_dpmsolver_multistep.py:1102 in step │
│ │
│ 1101 │ │ elif self.config.solver_order == 2 or self.lower_order_nums < 2 or lower_order_second: │
│ ❱ 1102 │ │ │ prev_sample = self.multistep_dpm_solver_second_order_update(self.model_outputs, sample=sample, noise=noise) │
│ 1103 │ │ else: │
│ │
│ /home/vlado/dev/sdnext/venv/lib/python3.12/site-packages/diffusers/schedulers/scheduling_dpmsolver_multistep.py:825 in multistep_dpm_solver_second_order_update │
│ │
│ 824 │ │ sigma_t, sigma_s0, sigma_s1 = ( │
│ ❱ 825 │ │ │ self.sigmas[self.step_index + 1], │
│ 826 │ │ │ self.sigmas[self.step_index], │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
IndexError: index 21 is out of bounds for dimension 0 with size 21System Info
diffusers==0.33.0-dev