Discriminator training: only integer tensors of a single element can be converted to an index

[umberH]

I am getting this error at train_discrim file.
Can you please guide me on how can I fix this issue?

File "train_discrim.py", line 88, in avg_representation
masked_hidden = hidden * mask
TypeError: only integer tensors of a single element can be converted to an index

[deepeshhada]

It looks like the current version of transformers is causing the issue. Can you please check if this error persists with transformers==2.5.1?

[umberH]

I tried multiple versions but the problem is still there.

[zyrmj0212]

I also encountered the same issue, and I think it's caused by the current version of transformer returning a different type of object of the transformer model's outputs. To put it more clear, you can check the type of the above 'hidden' variable, and it's probably string type! not a PyTorch tensor. Here's my corresponding solution (if there is a better solution, please feel free to correct me ):

change the code line 84 in train_discrim file : hidden, _ = self.encoder.transformer(x) to
hidden= self.encoder.transformer(x)[0]

Similar errors occur several times during my training and testing phase, however, in my opinion, they are all caused by the new version of transformer lib having some different properties than the older ones (the one the author used).