Count Loss with sigmoid

[tsuijenk]

Hello,

I was working out your count loss formula. I realized that if I applied sigmoid to my probability map of 0s and 1s, a 100% match of counts to groundtruth number of emitters would result in a heavier loss than an 80% match of counts.

I've only tried this to a simple case of 3x3 matrix with groundtruth number of emitters = 5. I am not sure yet if this applies to a very large number of emitters.

Any thoughts? Thank you!

[ASpeiser]

I'm not quite sure I understand. If you apply a sigmoid to the probabilities (which are already between 0 and 1) you would get numbers that are all somewhere between 0.5 and 0.6 and applying the loss to that wouldn't make much sense.

[tsuijenk]

I'm not quite sure I understand. If you apply a sigmoid to the probabilities (which are already between 0 and 1) you would get numbers that are all somewhere between 0.5 and 0.6 and applying the loss to that wouldn't make much sense.

Right! That is what I thought.

The reason I am asking is because my colleague used a sigmoid function for count loss while expecting the input to be Bernoulli probability map. But I did many test cases where the loss doesn't make sense in this situation.

I guess he (and myself perhaps) are confused by the sentence in the paper that says "logistic sigmoid activation function for the Bernoulli probability map."

Correct me if I am wrong, but I think what this sentence means is within the UNet class module, you apply the logistic sigmoid function for the Bernoulli probability map output, and apply hyperbolic tangent for the coordinate output etc.? (Instead of the usual ReLU activation function stated in the original UNet paper)

Thank you for replying so quickly! :)

[ASpeiser]

Correct me if I am wrong, but I think what this sentence means is within the UNet class module, you apply the logistic sigmoid function for the Bernoulli probability map output, and apply hyperbolic tangent for the coordinate output etc.?

Yes that is correct. The we apply the sigmoid to the network output such that p is between 0 and 1. For all hidden layers we use a different nonlinearity.

Then we use this to get the count loss:

    p_mean = p.sum(-1).sum(-1)
    p_var = (p - p ** 2).sum(-1).sum(-1)  # var estimate of bernoulli
    p_gauss = distributions.Normal(p_mean, torch.sqrt(p_var))

    log_prob = log_prob + p_gauss.log_prob(mask.sum(-1)) * mask.sum(-1)