Lighting Module Loaded From Checkpoint Generates Different Output Each Time

[xsys-technology]

I'm trying to gain some confidence in a model that seems to be training fine.

As a simple sanity check I'm trying to make sure I can load then test a checkpoint with the same input, expecting to be able to produce the same output each and every time (I'm using the same input and checkpoint each time so I expect the output to be the same).

Unfortunately, I'm observing different output each time I reload the checkpoint.

Here is the essence of what I'm doing:

for n in range(2):
        my_module = MyLightningModule.load_from_checkpoint(ckpt_path)

        my_dataset = MyDataset()
        batch = my_dataset.get_sanity_test_batch()  # confirmed to be the same batch every time

        # this output is different every time (???)
        output = my_module.model.generate(batch, max_length=some_length)

It is also probably worth noting that the model trained/loaded by my_module is a hugginface T5 transformer (T5ForConditionalGeneration )

Please help me figure out how to ensure output is consistent after loading a trained checkpoint.

[xsys-technology]

Turns out that I was doing something a little different in the actual code than:
my_module = MyLightningModule.load_from_checkpoint(ckpt_path)

When I do this exactly, things work as expected :)