Images may not be ideally distributed into batches

[araleza]

I've discovered something new about how batches are formed when the number of images in a bucket doesn't divide exactly by the batch size. It had been my assumption that PyTorch or sd-scripts would evenly distribute the images into each batch. e.g.:

16 images, batch size 5 ==> actual batch sizes [4, 4, 4, 4]

but now I put that assumption to the test, batches actually are full-sized until the final few images, so reality currently seems to be:

16 images, batch size 5 ==> actual batch sizes [5, 5, 5, 1]

Flux doesn't seem to respond anywhere near as well to batch size 1 compared to higher batch sizes, so I think the [4, 4, 4, 4] arrangement might offer higher quality training than [5, 5, 5, 1]. I'll probably write a custom batch sampler function to try to evenly distribute the images into batches to see if I can get further quality gains.