Convert logprobs (sometimes positive) to regular probabilities?

[anatol-grabowski]

Hey. How does one convert logprobs to regular probabilities? I've tried np.exp(token.logprob) expecting them to be always negative. But sometimes they are positive. Need some explanation on how to handle those.

[anatol-grabowski]

Realized that it is impossible to get probability from just one of the logits.
Used a variation of this code to get an estimate for the probability.

[abb128]

I think it may be necessary to add a softmax layer to the output to achieve this. I'll look into it when I get around to training another model