what's self(rnn_output, target) meaning?

[EricAugust]

` def forward_with_loss(self, rnn_output, target):

    if self.training:
        target_prob, sample_prob, sample = self(rnn_output, target)
        loss = self.nce_loss(target_prob, sample_prob, target, sample)
        return loss.mean()
    else:
        output = torch.addmm(
            1, self.word_bias.weight.view(-1), 1, rnn_output, self.word_embeddings.weight.t()
        )
        return self.CE(output, target)`

And why addmm rnn output and embedding.weight

[sa1ka]

In the training phase, we need to calculate probabilities for target words and noise words, and then calculate the nce loss. But in the valid phase, in order to get a comparable PPL with other models (like softmax and hierarchical softmax), we use the normal softmax approach. Actually, in the real applications, it is OK to use the unnormalized NCE probability to get faster inference time.

…

On Sat, Jul 14, 2018 at 9:29 PM, EricAugust ***@***.***> wrote: ` def forward_with_loss(self, rnn_output, target): if self.training: target_prob, sample_prob, sample = self(rnn_output, target) loss = self.nce_loss(target_prob, sample_prob, target, sample) return loss.mean() else: output = torch.addmm( 1, self.word_bias.weight.view(-1), 1, rnn_output, self.word_embeddings.weight.t() ) return self.CE(output, target)` And why add rnn output and embedding.weight — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH0cOiJPPKgvVvdxpxTBohS9LUpk6YT0ks5uGfI9gaJpZM4VP247> .