Parameter initialized with torch.randn may be not a good choice

[lddsdu]

It seems that the parameter initialized with randn (

xlnet-Pytorch/xlnet.py

Line 119 in cb793a1

self.q_proj_weight = nn.Parameter(torch.randn(self.d_model,

) will lead to low-performance, and I tried xavier_norm and kaiming_uniform, both reach a much higher AUC and F1 score in my task.