02. PyTorch Neural Network Classification - Excercise 6: Tanh()

[jonorl]

I managed to sucessfully do the excercises for the 2nd module, but under the excercises, number 6 asks to replicate the Tanh (hyperbolic tangent) activation function in pure PyTorch. I did it in a similar way to Sigmoid() and ReLU() but I don't understand if the excercise is referncing Tanh because we were supposed to use it in the previous moons excercise.

I tried using Tanh() replacing nn.ReLU() on my moon module with nn.tanh() as well as replacing torch.sigmoid() with torch.tanh() when transforming logits to predictions but in both cases the model needed more epochs to get to the target 96% accuracy. Am I doing something wrong?

[Haji-Suleman]

🧠 Tanh Activation – Exercise Clarification

I’m not doing anything wrong 🙂

• The exercise asks to replicate tanh using pure PyTorch ops, not just replace ReLU with nn.Tanh().
• tanh(x) outputs values in (-1, 1), so learning is usually slower than ReLU → more epochs is normal.
• Unlike sigmoid, tanh is not a probability function.

Important Notes

• If using BCEWithLogitsLoss, do NOT apply tanh/sigmoid during training.
• Use raw logits for loss.
• For predictions with tanh, use:

  preds = y_hat > 0.0