Added onto sphinx thingy

Author: Seilmast

.github/Individual_pages/Magnus_page.md

@@ -35,5 +35,17 @@ The EntropyPrediction class' main job is to take some inputs and return the Shan
 * __returnmetric__ : Returns the collected metric. 
 * __reset__ : Removes all the stored values up until that point. Readies the instance for storing values from a new epoch. 
 
-The class is initialized with a single parameter called "averages". This is inspired from other PyTorch and NumPy implementations and controlls how values from different batches or within batches will be combined. The __init__ method checks the value of this argument with an assertion, which must be one of three string. We only allow "mean", "sum" and "none" as methods of combining the different entropy values. 
-Furtherore, this method will also store the different Shannon Entropy values as we pass values into the __call__ method. 
+The class is initialized with a single parameter called "averages". This is inspired from other PyTorch and NumPy implementations and controlls how values from different batches or within batches will be combined. The __init__ method checks the value of this argument with an assertion, which must be one of three string. We only allow "mean", "sum" and "none" as methods of combining the different entropy values. We'll come back to the specifics here.  
+Furthermore, this method will also store the different Shannon Entropy values as we pass values into the __call__ method. 
+
+In __call__ we get both true labels and model logit scores for each sample in the batch as input. We're calculating Shannon Entropy, not KL-divergence, so the true labels aren't needed. 
+With permission I've used the scipy implementation to calculate entropy here. We apply a softmax over the logit values, then calculate the Shannon Entropy, and make sure to remove any NaN or Inf values which might arise from a perfect guess/distribution.
+
+Next we have the __returnmetric__ method which is used to retrive the stored metric. Here the averages argument comes into play. 
+Depending on what has been chosen as the averaging metric when initializing the class, one of the following operations will be applied to the stored values:
+* Mean: Calculate the mean of the stored entropy values.
+* Sum: Sum the stored entropy values.
+* None: Do nothing with the stored entropy values. 
+Then the value(s) are returned. 
+
+Lastly we have the __reset__ method which simply emptied the variable which stores the entropy values to prepare it for the next epoch. 

utils/metrics/EntropyPred.py

@@ -1,3 +1,4 @@
+import torch as th 
 import torch.nn as nn
 from scipy.stats import entropy
 
@@ -32,51 +33,27 @@ def __call__(self, y_true, y_logits):
             torch.Tensor: The aggregated entropy value(s) based on the specified
                           method ('mean', 'sum', or 'none').
         """
-        entropy_values = entropy(y_logits, axis=1)
+        y_pred = nn.Softmax(dim=1)(y_logits)
+        entropy_values = entropy(y_pred, axis=1)
         entropy_values = th.from_numpy(entropy_values)
 
         # Fix numerical errors for perfect guesses
         entropy_values[entropy_values == th.inf] = 0
         entropy_values = th.nan_to_num(entropy_values)
 
-        if self.averages == "mean":
-            entropy_values = th.mean(entropy_values)
-        elif self.averages == "sum":
-            entropy_values = th.sum(entropy_values)
-        elif self.averages == "none":
-            return entropy_values
-
-        self.stored_entropy_values.append(entropy_values)
-
-        return entropy_values
+        for sample in entropy_values:
+            self.stored_entropy_values.append(sample.item())
+        
 
     def __returnmetric__(self):
         if self.averages == "mean":
             self.stored_entropy_values = th.mean(self.stored_entropy_values)
         elif self.averages == "sum":
             self.stored_entropy_values = th.sum(self.stored_entropy_values)
         elif self.averages == "none":
-            return self.stored_entropy_values
+            pass 
+        return self.stored_entropy_values
 
     def __reset__(self):
         self.stored_entropy_values = []
 
-
-if __name__ == "__main__":
-    import torch as th
-
-    metric = EntropyPrediction(averages="mean")
-
-    true_lab = th.Tensor([0, 1, 1, 2, 4, 3]).reshape(6, 1)
-    pred_logits = th.nn.functional.one_hot(true_lab, 5)
-
-    assert th.abs((th.sum(metric(true_lab, pred_logits)) - 0.0)) < 1e-5
-
-    pred_logits = th.rand(6, 5)
-    metric2 = EntropyPrediction(averages="sum")
-    assert (
-        th.abs(
-            th.sum(6 * metric(true_lab, pred_logits) - metric2(true_lab, pred_logits))
-        )
-        < 1e-5
-    )