When I run .generate function of any of the 3b/7b/12b model, I get continuously tensor sizes printed which looks like torch.Size([1, 510])
torch.Size([1, 40, 509, 128])
torch.Size([1, 1])
torch.Size([1, 511])
torch.Size([1, 40, 510, 128])
torch.Size([1, 1])
.........
I think in code somewhere print( X.shape ) is written which is creating problem. I tried using Huggingface pipeline also but same problem.
I do not encounter this problem while using any other LLM model