The volume of a tensor cannot exceed 2^31-1 (tensorRT 8.0.16)

[773041642]

[Matrix Multiply]_output: tensor volume exceeds (2^31)-1, dimensions are [4,65536,65536])

what should I do?

[ttyio]

@773041642 , tensor size has 2G limitation in TensorRT, could you make your network smaller? Thanks

[773041642]

Whether there is a way to modify this limit, changing the size will affect the accuracy of the model .

[ttyio]

@773041642 , sorry this is not configurable , because many kernel implementations using 32bit when addressing the offset in tensors.

[773041642]

@ttyio, thanks for your help, I will try to reduce my tensor.

[gesanqiu]

@ttyio Any update about this limitation? For stable-diffusion model, it's tensor size can exceed this limitation easily because of the high resolution output.

[DamonsJ]

@ttyio is there any update here ?

or is there any solution for dit models which seqlen is really large

[HanG-CS]

any update?????