Different inference results under different batch size

[HelloWorldLTY]

Hi, I noticed that the inference result based on model.predict_on_dataset is slightly different with different batch size. Is it normal? Thanks.

[avantikalal]

Hi, this looks like a small difference but shouldn't be the case. Is it possible for you to give us a reproducible example?

[HelloWorldLTY]

Hi, thanks for your answers. I think I can send you a small set of datasets to reproduce the error via email, as there are some datasets not proper to share publicly, like GTEx. I tried on various datasets and this problem happend again: