Simplify debug code

[dxqb]

Part of #1203

[O-J1]

Looks good to me but we are missing these 2

BaseStableDiffusionSetup.py
BaseStableDiffusionXLSetup.py

[dxqb]

Looks good to me but we are missing these 2

BaseStableDiffusionSetup.py BaseStableDiffusionXLSetup.py

the debug code for diffusion models is different from flow matching models, so I only shared code between flow matching models. you could share code between the diffusion models also, but since the main purpose of these PRs is to make it easier to implement any future models, I chose not to do that. It's unlikely that we implement any new diffusion models.

[O-J1]

you could share code between the diffusion models also, but since the main purpose of these PRs is to make it easier to implement any future models, I chose not to do that. It's unlikely that we implement any new diffusion models.

I'd argue if we are going to refactor we do it for all to ensure we clean up properly, otherwise dont at all, since it just makes it more fractured.

[dxqb]

you could share code between the diffusion models also, but since the main purpose of these PRs is to make it easier to implement any future models, I chose not to do that. It's unlikely that we implement any new diffusion models.

I'd argue if we are going to refactor we do it for all to ensure we clean up properly, otherwise dont at all, since it just makes it more fractured.

PR welcome :)
the intention of all these PRs is to provide generic code where code can be shared, but still allow model-specific implementations. For example, in the dataloader PR I kept most of stable cascade code unchanged because it was too different from the other dataloaders, and I would have broken something.
So in the same way, I don't mind that StableDiffusion debug code is different from the debug code of flow matching models.

Yes, StableDiffusion and StableDiffusionXL could share code between themselves. But that's not worth the effort.